Are ‘Evidence-based’ policies damaging policymaking?

Both the gov­ern­ment and the pub­lic should be far more scep­ti­cal about poli­cies which are pur­ported to be ‘ev­i­dence-based’, ar­gues new re­search re­leased to­day by the In­sti­tute of Eco­nomic Af­fairs.

In Quack Policy – Abus­ing Science in the Cause of Pa­ter­nal­ism, Jamie Whyte ex­poses how poli­ti­ci­ans pro­mote reg­u­la­tions and taxes un­der the jus­tifi­ca­tion of sci­en­tific ev­i­dence, yet the ex­perts pro­mot­ing these poli­cies of­ten make ba­sic er­rors and have lit­tle or no grasp of eco­nomics.

http://​​www.iea.org.uk/​​in-the-me­dia/​​press-re­lease/​​%E2%80%98ev­i­dence-based%E2%80%99-poli­cies-are-dam­ag­ing-uk-policymaking

Those in favour of ev­i­denced based poli­cies have tended to be dis­mis­sive, ar­gu­ing that this is just a case of lack of ev­i­dence, rather than a prob­lem with ev­i­dence-based poli­cies per se.

https://​​twit­ter.com/​​ben­goldacre/​​sta­tus/​​370159847985381376

I’m not fully con­vinced though. I’ve started read­ing Tak­ing the medicine: a short his­tory of medicine’s beau­tiful idea and our difficulty swal­low­ing it by Druin Burch, and this has plenty of ex­am­ples of peo­ple say­ing some­thing like “Of course, up till now medicine has been to­tally wrong, but I’ve got a new way of do­ing it right”—and then go­ing on to make the same old mis­takes. Maybe ev­i­dence based prac­tice is the same, just a way of con­vinc­ing our­selves that we’ve got it right this time.

As I see it, the trou­ble with ev­i­dence based prac­tice is that what counts as ev­i­dence based to­day doesn’t meet to­mor­rows higher stan­dards. This means

  1. How can we trust ev­i­dence based prac­tice if it might be over­turned to­mor­row?

  2. As less gets to count as ev­i­dence, ev­i­dence based prac­tice gets harder and harder to do.

At the root of this is the in­sis­tence that ev­i­dence must meet some ‘gold stan­dard’. This seems to be too much the fre­quen­tist view­point, but in the end we have to be Bayesi­ans, tak­ing all ev­i­dence into ac­count.