Opinion piece on the Swedish Network for Evidence-Based Policy

Cross-posted from the Effec­tive Altru­ism Forum

Fol­low up to: The effec­tive­ness-alone strat­egy and ev­i­dence-based policy

A trans­la­tion of the opinion piece can be found here.

I

Effec­tive al­tru­ism is a great con­cept, but it’s not triv­ial to sell. There are there­fore good rea­sons to ally our­selves with other ra­tio­nal­ist memes to in­crease the level of ra­tio­nal­ity and effec­tive­ness in the world. One pow­er­ful such ra­tio­nal­ist meme is “ev­i­dence-based policy”, which is in­spired by the “ev­i­dence-based medicine” move­ment.

The ex­act mean­ing of ev­i­dence-based policy is some­what dis­puted, but gen­er­ally pro­po­nents of ev­i­dence-based policy de­mand that the stan­dards on which policy is based should be raised. Many be­lieve strongly in ran­dom­ized con­trol tri­als (RCTs) and in the “hi­er­ar­chy of ev­i­dence”, but there is not com­plete agree­ment on the strength of RCTs rel­a­tive to other kinds of stud­ies.

In the US and the UK, there are sev­eral or­ga­ni­za­tions which work on ev­i­dence-based policy, such as the Bri­tish What Works Net­work and the Amer­i­can Coal­i­tion for Ev­i­dence-Based Policy. In­spired by them, I took the ini­ta­tive to start a Swedish net­work for ev­i­dence-based policy at the start of this year. We are by now around 50 (de­pend­ing on how you count) re­searchers, civil ser­vants, jour­nal­ists, con­sul­tants, stu­dents and other ac­tivists in the net­work. Only my­self and a few oth­ers are EA mem­bers, so it’s not an EA or­ga­ni­za­tion, but as I ar­gued in my pre­vi­ous post, I do be­lieve work­ing on this nev­er­the­less is an effec­tively al­tru­is­tic cause.

One differ­ence be­tween us and What Works is that we aim to be a broad cam­paign­ing or­ga­ni­za­tion. We be­lieve that policy not be­ing ev­i­dence-based is not only due to a lack of knowl­edge, but also due to a lack of will, es­pe­cially among poli­ti­ci­ans. Poli­ti­ci­ans of­ten dis­re­gard ex­pert ad­vice (on what poli­cies are the most effec­tive to reach a given set of goals) which goes against their poli­ti­cal prej­u­dices. There­fore we need to put pres­sure on poli­ti­ci­ans—not the least in the me­dia—rather than just work be­hind the scenes as an ex­pert or­ga­ni­za­tion.

II (Most linked replies be­low are in Swedish)

Our ac­tivi­ties were fairly mod­est un­til last Sun­day, when we wrote an opinion piece call­ing for ev­i­dence-based policy (English). The opinion piece was pub­lished in the most widely-read broad­sheet, Da­gens Ny­heter, on DN De­batt—a sort of op-ed fo­rum. DN De­batt has a spe­cial stand­ing in Swedish poli­tics. Every­body reads it and it’s well-re­spected.

Hence we had ex­pected a lot of at­ten­tion, but the re­sults still ex­ceeded them. Ours was the sec­ond most shared DN De­batt-ar­ti­cle in the month of May. We got seven replies in Da­gens Ny­heter, were strongly crit­i­cized in the other main broad­sheet, Sven­ska Dag­bladet (con­ser­va­tive), par­o­died in a pop­u­lar pub­lic ser­vice (equiv­a­lent of BBC) pod­cast, and were also com­mented on in a num­ber of smaller news­pa­pers. The dis­cus­sion on Twit­ter was pretty in­tense. Sub­se­quently, we also pub­lished two replies to replies in Da­gens Ny­heter and Sven­ska Dag­bladet.

It’s hard to tell what the ma­jor­ity opinion on our piece was. Cer­tainly, there was a lot of praise and a lot of Face­book likes, but also some fierce crit­i­cism. This was al­most ex­clu­sively down to mi­s­un­der­stand­ings. I won’t bog you down with all of the de­tails here, but will rather sum­ma­rize my gen­eral con­clu­sions. They could be use­ful for any­one try­ing to write on ev­i­dence-based policy or re­lated con­cepts in other coun­tries.

I should say that “ev­i­dence-based policy” isn’t as en­trenched a con­cept in Swe­den as it is in the US and the UK, which prob­a­bly played to our dis­ad­van­tage.

1) You need to be very clear over the means-ends-dis­tinc­tion. Ev­i­dence-based policy is about mak­ing the meth­ods for reach­ing your poli­ti­cal goals (hap­piness, equal­ity, liberty, etc) more effec­tive by the use of ev­i­dence. It is not about prop­a­gat­ing any par­tic­u­lar set of poli­ti­cal goals. We tried to be clear about this, but partly failed for two rea­sons. Firstly, Da­gens Ny­heter set the head­line, which was mis­lead­ing. Se­cond, we only clar­ified this dis­tinc­tion at the end. It should have been at the top.

2) There is a straw man con­cep­tion of ev­i­dence-based policy, or ex­pert-in­formed policy and poli­ti­cal ra­tio­nal­ity more gen­er­ally, akin to Ju­lia Galef’s “Straw Vul­can” con­cep­tion of ra­tio­nal­ity. Per­haps this varies a bit from coun­try to coun­try, but in Swe­den it’s strong. Let’s call it “Straw Soviet” for now (please come with sug­ges­tions!).

Ac­cord­ing to this con­cep­tion, ev­i­dence-based policy means tech­noc­racy (of a dic­ta­to­rial form, ac­cord­ing to the more ex­treme in­ter­pre­ta­tions), dis­re­gard of non-quan­tifi­able val­ues (cf “Straw Vul­can”), dis­re­gard of emo­tions, “Mad Scien­tist”-con­cep­tion of so­ciety as a labra­tory, etc, etc. You need to ev­ery­thing you can to counter such in­ter­pre­ta­tions. I cer­tainly un­der­es­ti­mated the power of this straw man meme. I should also say that the Straw Soviet is prob­a­bly more vi­cious than the Straw Vul­can, who seems more in­no­cent (per­haps this is partly down to Ju­lia’s plafyul pre­sen­ta­tion of it, though).

For in­stance, Sven­ska Dag­bladet’s crit­i­cism was all about the “Straw Soviet”. We were said to want to “de­sign voter be­havi­our” (this was also partly due to the ar­ti­cle hav­ing been signed by a few nudgers who call them­selves “be­havi­oural en­g­ineers”—a big trig­ger of the Straw Soviet). Here are some more quotes:

It is per­haps not the “en­light­ened despot” who is called for in the opinion piece, but rather Dr Despot. To­day’s most fright­en­ing read­ing came from the re­cently formed “Net­work for Ev­i­dence-Based Policy” (Da­gens Ny­heter 1 June).

//​

Since there prob­a­bly are very few cit­i­zens who base their votes on re­search re­ports, free elec­tions yields re­sults which are not ev­i­dence-based. Ac­cord­ing to the ar­gu­ment in the opinion piece, that means that since we “see the world through par­ti­san lenses”, the elec­tion re­sults are as a rule prob­le­matic or di­rectly harm­ful.

//​

Now if the net­work were cor­rect, true ev­i­dence-based poli­cies would lead to a sin­gle pro­posal, a solu­tion “free from ide­ol­ogy and pop­ulism”. That would in turn mean that all par­ties ar­rived at the same an­swer, and it is ab­solutely im­pos­si­ble why that – though ever so full of ev­i­dence – would be de­sir­able.

A vibrant democ­racy is based on the ex­is­tence of con­flicts of opinion and value, in­tel­lec­tual di­ver­sity and the cit­i­zen’s right to freely ex­press it. The com­plete and ra­tio­nal cit­i­zen is an anomaly, and based on the un­pleas­ant idea that en­light­ened pow­ers can raise, de­sign, a new man.

Para­dox­i­cally, it is pre­cisely highly ide­olog­i­cal regimes which have at­tempted just that. The re­sults have been dev­as­tat­ing.

We got sev­eral other replies along these lines, though we also got a much more pos­i­tive one from Da­gens Ny­heter it­self. A large group of replies treated more tech­ni­cal and hum-drum is­sues con­cern­ing RCTs, prac­ti­cal policy-mak­ing, etc.

3) Con­nected to the Straw Vul­can and the Straw Soviet, there is a “Straw Naive Pos­i­tivist Scien­tist” (again, sug­ges­tions for bet­ter terms are wel­come), who thinks that knowl­edge is eas­ily ob­tain­able even in messy fields like eco­nomics, that it’s easy to reach con­sen­sus if you just don’t lead poli­ti­cal mis­con­cep­tions mis­lead you, that you always eas­ily can in­fer policy-ad­vice from re­search, etc. We got a lot of crit­i­cism which was based on the Straw Naive Pos­i­tivist. Ob­vi­ously, we don’t hold any of those views.

4) Peo­ple read very su­perfi­cially. This is not only true of the man in the street, but also of many jour­nal­ists, poli­ti­ci­ans, etc. At some level I know this, hav­ing my­self writ­ten about re­search on this on my blog, but it’s harder to make full use of that knowl­edge when you write.

Also lots of peo­ple don’t use the prin­ci­ple of char­ity at all. Some of the replies—in­clud­ing one from a philos­o­phy pro­fes­sor—were ex­ceed­ingly un­char­i­ta­ble. Thus don’t ex­pect peo­ple to use the prin­ci­ple of char­ity—es­pe­cially when emo­tional memes like the Straw Soviet are around.

When you fight such pow­er­ful memes, you need to be ex­tremely clear. You need to say the things you re­ally want to get across early, to re­peat them, and to give ex­am­ples. If at all pos­si­ble, you should con­trol the ti­tle, since that sets so much of the tone of the piece (give the pub­lish­ers a juicy sug­ges­tion and they might buy it). Don’t say too much, but fo­cus on get­ting the cen­tral mes­sage across.

This is so differ­ent from writ­ing an aca­demic pa­per. Of course that’s ob­vi­ous, but it’s one thing to get it on an in­tel­lec­tual level, quite an­other to re­ally in­ter­nal­ize it. If you could get a skil­led pub­lic com­mu­ni­ca­tor on board, that would be very use­ful.

I also think it would be good to pre-test ma­jor ar­ti­cles (e.g. on Me­chan­i­cal Turk) to get a clearer pic­ture of whether the mes­sage gets across. If you don’t want the con­tent to leak be­fore­hand, that might not be doable, though.

5) We were prob­a­bly a bit too ex­treme re­gard­ing RCTs, which trig­gered the Straw Soviet and the Straw Naive Pos­i­tivist (for episte­molog­i­cal and eth­i­cal rea­sons). It would have been more tac­ti­cal to em­pha­size other stuff.

6) We would have come off as more con­crete if we had based our opinion piece on a re­search re­port on the state of Swedish policy-mak­ing. It’s great if you can do that, but I don’t think it would have been ra­tio­nal for us (see be­low).

7) We should have stressed how big the move­ment on ev­i­dence-based policy is in the US and the UK. For in­stance, we could have men­tioned that “Obama’s 2016 bud­get calls for an em­pha­sis on ev­i­dence-based ap­proaches at all lev­els of gov­ern­ment”. Obama be­ing pop­u­lar and re­spected in Swe­den, that would have done much to disarm the Straw Soviet.

8) It was a mis­take to men­tion le­gal means as a way of mak­ing poli­tics more ev­i­dence-based, since it strongly trig­gers the Soviet meme. Even those who oth­er­wise sup­ported us crit­i­cized this sug­ges­tion.

III

In our replies, we fo­cused on rec­tify­ing the mi­s­un­der­stand­ings, fo­cus­ing on the claim that we are call­ing for “Dr Despot”. Th­ese replies nor­mally gets much less at­ten­tion, and so it was with ours as well. How­ever, the re­cep­tion also was more unan­i­mously pos­i­tive, es­pe­cially from aca­demics and civil ser­vants who know the field.

I don’t re­gret writ­ing this opinion piece at this early stage. Be­fore I started writ­ing it (I wrote the body of the text, and the oth­ers then made minor tweaks) there wasn’t much ac­tivity in our net­work. Now, we have many more mem­bers, in­clud­ing more se­nior ones. Also, those who already were in the net­work grew much more en­thu­si­as­tic af­ter the pub­li­ca­tion. Thus all-in-all it’s been a ma­jor suc­cess. Still, I think you can learn a lot from things we could have done bet­ter.

I’ll write more later on how the net­work is de­vel­op­ing more gen­er­ally. Also I should add that I’m still di­gest­ing what I’ve learnt, so my con­clu­sions aren’t set in stone. Any com­ments are wel­come.