TBD is a quarterly-ish newsletter about deploying knowledge for impact, learning at scale, and making more thoughtful choices for ourselves and our organizations. This is the second issue, which was originally published in June 2019. Enjoy! --Ian
An Introduction to Decision Modeling
Decision-making is life. Over time, our decisions carve an identity for ourselves and our organizations, and it is our decisions, more than anything else, that determine how we are remembered after we’re gone. Despite their importance, though, we barely pay attention to most of the decisions we make. Biology has programmed in us a powerful instinct to make decisions using our intuitions rather than our conscious selves whenever possible. There are good reasons for this; if we had to think about every little decision we made, we’d never get anything done. But complex decisions require us to compare the likelihood and desirability of many possible futures on multiple, disparate, and often conflicting criteria, something our intuitions just aren’t naturally equipped to do.
Thankfully, there is a better way. The secret to resolving complex, risky dilemmas with justified ease and confidence is to model your decisions explicitly. At its best, modeling our decisions can help us make the very human exercise of decision-making not only more likely to lead to the outcomes we want, but more instinctively satisfying as well.
Most Funders Admit Their Own Evaluations Are Not Useful I really wish that headline was an exaggeration, but it’s not much of one. In 2015, the Center for Evaluation Innovation and the Center for Effective Philanthropy surveyed evaluation and program executives at 127 US and Canadian foundations with $10 million or more in annual giving. The resulting report, “Benchmarking Foundation Evaluation Practices,” contains startling revelations about how little evaluation reports are used. Most remarkably, more than three-quarters of respondents reported that they have a hard time commissioning evaluations that yield meaningful insights for the field, grantees, or even their own colleagues! (Twitter thread)
Our Cognitive Biases Can Tell Us a Lot About the Meaning of Life You probably know Daniel Kahneman’s classic volume Thinking, Fast and Slow as a comprehensive catalogue of cognitive biases and errors in judgment. But it’s more than that: it’s also about the meaning of life. In the section entitled “Two Selves,” Kahneman reveals that we remember pain and pleasure differently from how we experience it. You might assume that the reality of our experiences is what matters to us. But in fact that’s not true. What we really care about is our memories of our experiences, and the story those memories cause us to tell ourselves and others about our lives. In other words, narratives don’t just matter, narratives are everything. This might just seem like a curious artefact of the research, but it has enormous philosophical and practical implications for social sector leaders. (Twitter thread)
Stuff You Should Know About
If you’ve ever found yourself looking for examples of real-life cost-benefit and social return on investment (SROI) analyses, the Social Value International network has your back. Their UK chapter’s database includes more than 800 publications ranging from “Cost Benefit Analysis of Early Childhood Intervention” to “The Social Value of Community Pubs.”
Have you ever been frustrated with a reporter’s treatment of a research study that seemed to stretch the conclusions much farther than warranted? Well, according to research published in 2014, “most exaggeration in health-related science news is already present in the press releases issued by universities.” Intrigued by this result, the team got back together for an unprecedented randomized controlled trial of real-life science communication strategies employed by university press offices, which found that, contrary to many people’s assumptions, toning down the hype around research findings doesn’t necessarily lead to less interest from journalists. As a bonus, in this bonkers 96-part Twitter thread, co-author Chris Chambers details the amazing back story (and back-stabbing) behind the latest study.
Speaking of misrepresenting research, I’ve really been enjoying Vox.com’s Future Perfect project highlighting effective altruism and related topics. Staff writer Kelsey Piper has been covering a regular research integrity beat, and recently she wrote about two remarkable instances in which well-known nonfiction authors have been caught in the act of badly misunderstanding key elements of the research underlying their books. Piper’s takeaway? “Don’t trust shocking claims with a single source, even if they’re from a well-regarded expert. It’s all too easy to misread a study, and all too easy for those errors to make it all the way to print.”
That’s all for now!
If you enjoyed this edition of TBD, please consider forwarding it to a friend. It’s easy to sign up here. See you next time!
To Be Decided #2
TBD is a quarterly-ish newsletter about deploying knowledge for impact, learning at scale, and making more thoughtful choices for ourselves and our organizations. This is the second issue, which was originally published in June 2019. Enjoy! --Ian
An Introduction to Decision Modeling
Decision-making is life. Over time, our decisions carve an identity for ourselves and our organizations, and it is our decisions, more than anything else, that determine how we are remembered after we’re gone. Despite their importance, though, we barely pay attention to most of the decisions we make. Biology has programmed in us a powerful instinct to make decisions using our intuitions rather than our conscious selves whenever possible. There are good reasons for this; if we had to think about every little decision we made, we’d never get anything done. But complex decisions require us to compare the likelihood and desirability of many possible futures on multiple, disparate, and often conflicting criteria, something our intuitions just aren’t naturally equipped to do.
Thankfully, there is a better way. The secret to resolving complex, risky dilemmas with justified ease and confidence is to model your decisions explicitly. At its best, modeling our decisions can help us make the very human exercise of decision-making not only more likely to lead to the outcomes we want, but more instinctively satisfying as well.
(Keep reading)
What I’ve Been Reading
Most Funders Admit Their Own Evaluations Are Not Useful
I really wish that headline was an exaggeration, but it’s not much of one. In 2015, the Center for Evaluation Innovation and the Center for Effective Philanthropy surveyed evaluation and program executives at 127 US and Canadian foundations with $10 million or more in annual giving. The resulting report, “Benchmarking Foundation Evaluation Practices,” contains startling revelations about how little evaluation reports are used. Most remarkably, more than three-quarters of respondents reported that they have a hard time commissioning evaluations that yield meaningful insights for the field, grantees, or even their own colleagues!
(Twitter thread)
Our Cognitive Biases Can Tell Us a Lot About the Meaning of Life You probably know Daniel Kahneman’s classic volume Thinking, Fast and Slow as a comprehensive catalogue of cognitive biases and errors in judgment. But it’s more than that: it’s also about the meaning of life. In the section entitled “Two Selves,” Kahneman reveals that we remember pain and pleasure differently from how we experience it. You might assume that the reality of our experiences is what matters to us. But in fact that’s not true. What we really care about is our memories of our experiences, and the story those memories cause us to tell ourselves and others about our lives. In other words, narratives don’t just matter, narratives are everything. This might just seem like a curious artefact of the research, but it has enormous philosophical and practical implications for social sector leaders. (Twitter thread)
Stuff You Should Know About
If you’ve ever found yourself looking for examples of real-life cost-benefit and social return on investment (SROI) analyses, the Social Value International network has your back. Their UK chapter’s database includes more than 800 publications ranging from “Cost Benefit Analysis of Early Childhood Intervention” to “The Social Value of Community Pubs.”
Have you ever been frustrated with a reporter’s treatment of a research study that seemed to stretch the conclusions much farther than warranted? Well, according to research published in 2014, “most exaggeration in health-related science news is already present in the press releases issued by universities.” Intrigued by this result, the team got back together for an unprecedented randomized controlled trial of real-life science communication strategies employed by university press offices, which found that, contrary to many people’s assumptions, toning down the hype around research findings doesn’t necessarily lead to less interest from journalists. As a bonus, in this bonkers 96-part Twitter thread, co-author Chris Chambers details the amazing back story (and back-stabbing) behind the latest study.
Speaking of misrepresenting research, I’ve really been enjoying Vox.com’s Future Perfect project highlighting effective altruism and related topics. Staff writer Kelsey Piper has been covering a regular research integrity beat, and recently she wrote about two remarkable instances in which well-known nonfiction authors have been caught in the act of badly misunderstanding key elements of the research underlying their books. Piper’s takeaway? “Don’t trust shocking claims with a single source, even if they’re from a well-regarded expert. It’s all too easy to misread a study, and all too easy for those errors to make it all the way to print.”
That’s all for now!
If you enjoyed this edition of TBD, please consider forwarding it to a friend. It’s easy to sign up here. See you next time!