To Be Decided #2

TBD is a quar­terly-ish newslet­ter about de­ploy­ing knowl­edge for im­pact, learn­ing at scale, and mak­ing more thought­ful choices for our­selves and our or­ga­ni­za­tions. This is the sec­ond is­sue, which was origi­nally pub­lished in June 2019. En­joy! --Ian

An In­tro­duc­tion to De­ci­sion Modeling

De­ci­sion-mak­ing is life. Over time, our de­ci­sions carve an iden­tity for our­selves and our or­ga­ni­za­tions, and it is our de­ci­sions, more than any­thing else, that de­ter­mine how we are re­mem­bered af­ter we’re gone. De­spite their im­por­tance, though, we barely pay at­ten­tion to most of the de­ci­sions we make. Biol­ogy has pro­grammed in us a pow­er­ful in­stinct to make de­ci­sions us­ing our in­tu­itions rather than our con­scious selves when­ever pos­si­ble. There are good rea­sons for this; if we had to think about ev­ery lit­tle de­ci­sion we made, we’d never get any­thing done. But com­plex de­ci­sions re­quire us to com­pare the like­li­hood and de­sir­a­bil­ity of many pos­si­ble fu­tures on mul­ti­ple, dis­parate, and of­ten con­flict­ing crite­ria, some­thing our in­tu­itions just aren’t nat­u­rally equipped to do.

Thank­fully, there is a bet­ter way. The se­cret to re­solv­ing com­plex, risky dilem­mas with jus­tified ease and con­fi­dence is to model your de­ci­sions ex­plic­itly. At its best, mod­el­ing our de­ci­sions can help us make the very hu­man ex­er­cise of de­ci­sion-mak­ing not only more likely to lead to the out­comes we want, but more in­stinc­tively satis­fy­ing as well.

(Keep read­ing)

What I’ve Been Reading

Most Fun­ders Ad­mit Their Own Eval­u­a­tions Are Not Use­ful
I re­ally wish that head­line was an ex­ag­ger­a­tion, but it’s not much of one. In 2015, the Cen­ter for Eval­u­a­tion In­no­va­tion and the Cen­ter for Effec­tive Philan­thropy sur­veyed eval­u­a­tion and pro­gram ex­ec­u­tives at 127 US and Cana­dian foun­da­tions with $10 mil­lion or more in an­nual giv­ing. The re­sult­ing re­port, “Bench­mark­ing Foun­da­tion Eval­u­a­tion Prac­tices,” con­tains startling rev­e­la­tions about how lit­tle eval­u­a­tion re­ports are used. Most re­mark­ably, more than three-quar­ters of re­spon­dents re­ported that they have a hard time com­mis­sion­ing eval­u­a­tions that yield mean­ingful in­sights for the field, grantees, or even their own col­leagues!
(Twit­ter thread)

Our Cog­ni­tive Bi­ases Can Tell Us a Lot About the Mean­ing of Life You prob­a­bly know Daniel Kah­ne­man’s clas­sic vol­ume Think­ing, Fast and Slow as a com­pre­hen­sive cat­a­logue of cog­ni­tive bi­ases and er­rors in judg­ment. But it’s more than that: it’s also about the mean­ing of life. In the sec­tion en­ti­tled “Two Selves,” Kah­ne­man re­veals that we re­mem­ber pain and plea­sure differ­ently from how we ex­pe­rience it. You might as­sume that the re­al­ity of our ex­pe­riences is what mat­ters to us. But in fact that’s not true. What we re­ally care about is our mem­o­ries of our ex­pe­riences, and the story those mem­o­ries cause us to tell our­selves and oth­ers about our lives. In other words, nar­ra­tives don’t just mat­ter, nar­ra­tives are ev­ery­thing. This might just seem like a cu­ri­ous arte­fact of the re­search, but it has enor­mous philo­soph­i­cal and prac­ti­cal im­pli­ca­tions for so­cial sec­tor lead­ers. (Twit­ter thread)

Stuff You Should Know About

  • If you’ve ever found your­self look­ing for ex­am­ples of real-life cost-benefit and so­cial re­turn on in­vest­ment (SROI) analy­ses, the So­cial Value In­ter­na­tional net­work has your back. Their UK chap­ter’s database in­cludes more than 800 pub­li­ca­tions rang­ing from “Cost Benefit Anal­y­sis of Early Child­hood In­ter­ven­tion” to “The So­cial Value of Com­mu­nity Pubs.”

  • Have you ever been frus­trated with a re­porter’s treat­ment of a re­search study that seemed to stretch the con­clu­sions much farther than war­ranted? Well, ac­cord­ing to re­search pub­lished in 2014, “most ex­ag­ger­a­tion in health-re­lated sci­ence news is already pre­sent in the press re­leases is­sued by uni­ver­si­ties.” In­trigued by this re­sult, the team got back to­gether for an un­prece­dented ran­dom­ized con­trol­led trial of real-life sci­ence com­mu­ni­ca­tion strate­gies em­ployed by uni­ver­sity press offices, which found that, con­trary to many peo­ple’s as­sump­tions, ton­ing down the hype around re­search find­ings doesn’t nec­es­sar­ily lead to less in­ter­est from jour­nal­ists. As a bonus, in this bonkers 96-part Twit­ter thread, co-au­thor Chris Cham­bers de­tails the amaz­ing back story (and back-stab­bing) be­hind the lat­est study.

  • Speak­ing of mis­rep­re­sent­ing re­search, I’ve re­ally been en­joy­ing Vox.com’s Fu­ture Perfect pro­ject high­light­ing effec­tive al­tru­ism and re­lated top­ics. Staff writer Kel­sey Piper has been cov­er­ing a reg­u­lar re­search in­tegrity beat, and re­cently she wrote about two re­mark­able in­stances in which well-known non­fic­tion au­thors have been caught in the act of badly mi­s­un­der­stand­ing key el­e­ments of the re­search un­der­ly­ing their books. Piper’s take­away? “Don’t trust shock­ing claims with a sin­gle source, even if they’re from a well-re­garded ex­pert. It’s all too easy to mis­read a study, and all too easy for those er­rors to make it all the way to print.”

That’s all for now!

If you en­joyed this edi­tion of TBD, please con­sider for­ward­ing it to a friend. It’s easy to sign up here. See you next time!

No comments.