I’ve been thinking a bunch about this study (direct link) of CBT-style techniques used to try and get criminals onto a better life path in Liberia that showed what looks like spectacular results, curious if anyone wants to share thoughts. If it would replicate seems like it crosses the EA-worthiness threshold.
I’ve been thinking more about the topic of this post, which is the cash transfer study in Chicago. That worked out less well. It seems that small cash transfers to those in need here in America made them feel worse and maybe even actually makes them worse off somehow? (paper)
This prediction seems very reasonable, and I think I would have agreed with it if I’d thought about this in advance. My model would have said that obviously cash is good and cash is helpful, in the short term getting some of it will be helpful. In the long term I’d expect some positive impact from the wealth effect. And in some cases, that money will have a particular jam that it solves or opportunity it enables (e.g. fix your car, security deposit on a new apartment, maybe even give opportunity to start a small business). In those cases it would have a permanent impact, so I would predict some overall positive impact. But in my experience there are enough implicit taxes and poverty traps for such people, and such a cultural lack of the idea of hanging onto money, that it is tough to hang onto funds, so mostly they’d be back where they started.
It looks like within about three weeks the money was mostly gone for both groups. That is not what would happen for an economically rational agent, unless they had a lot of opportunities for very high return on capital or they were paying down debt.
Elizabeth had a similar reaction to mine.
There’s something very interesting in this graph. The three groups have completely converged by the end of the 180 day period, but the average bank balance is now considerably higher. They all started out with ~$200 and now they have ~$400. That feels like a meaningfully different average amount.
This happened during Covid, so I am guessing that this is due to the Covid checks or other similar factors. This certainly complicates the experiment.
Those are pretty different things to happen. If you spend the money on bills, that’s paying down debt. It makes sense that this should happen right away. Spending on food and drink in a way you wouldn’t have otherwise is true consumption, the opposite of that.
Note of course this is ‘what they intended to do’ with the money, not what actually happened. Gambling money tends to end up bigger than expected and all that.
I’d ask less ‘what do poor people spend money on?’ and more ‘what do poor people spend more money on when they get a windfall?’
This tells a very clear story. The control group is spending a lot on groceries, but the treatment groups are spending less. I don’t think they are splurging all that much here, more like they are spending money there anyway and money is fungible. This makes sense, this is smart.
Where they are spending it is on bills, utilities (more bills), and especially on housing. Also small on general debt, education and clothes.
Notice that non-durable household goods spending is stronger in the control group, and durable household goods spending is stronger in the treatment group.
Note that it’s not easy to spend only $500 or even $2000 on housing, in a way that you weren’t already spending it, other than paying debt. So my guess is the bulk of this is ‘was behind on rent or mortgage’ and now they aren’t, but the size of this shift says that perhaps some of them did use this to get a new place.
So I interpret this as these people’s balance sheets are shot and they used the money to shore up their balance sheets and mostly didn’t spend the money beyond that.
But that turns out not to match what we find in the next chart.
My understanding is that this has been oriented such that higher numbers are always good and lower numbers are bad. So giving people money caused them to have more debt, less income, be less happy and in worse mental health, get less sleep and have less food security.
This didn’t merely backfire, it backfired completely. People’s financial situations actively got worse, both subjectively and objectively.
This is true, but everything got worse, so it’s thin. It makes sense they felt more burdened by the needs of others but their own needs felt like they got worse too. Again, even food security.
The paper is long, but there’s certainly good reasons to consider reading it.
Comparisons to Other Programs
Comparing this grant to the one in Kenya by GiveWell, in Kenya the typical grant is $1,000, and median monthly income is $76, so that’s a little over a year’s income. The median income in Chicago is about $34,000, so $2,000 is good for about three weeks for the median earner, although the paper says it was actually more like two months. This was a poor group. I am still skeptical this is the right comparison, as the earnings in this group sound pretty depressed from normal.
Or, as Sarah puts it:
I wouldn’t have expected this to be life changing money very often, but I would have expected some positive outcome some of the time and the average values to go up rather than down.
Here’s a first-hand report of a similar spot.
Bank balance being higher and bills being covered still counts as better in my book.
The small size of the grants makes it very difficult to say that there is something structurally different going on in these two situations. It is highly plausible that in Chicago $2,000 simply wasn’t enough money to escape the poverty trap and allow investment, whereas $1,000 in Kenya often was enough. The PPP gap alone makes the grants in Kenya functionally bigger.
It is also quite plausible that financial engineering and the nature of debt makes a big difference here. It’s also less concentrated giving, so the recipients can’t serve as customers for each other, the people around you still have needs that fall upon you and there is no cultural expectation that you must ‘make something of’ this opportunity.
Unhints: Need Versus Perception of Need
The thing that I couldn’t stop thinking about was the idea that getting resources increased scarcity because people ‘became more aware of all the needs they cannot address.’
This is a fundamental mismatch or confusion somewhere. Are needs changing, or are needs staying the same?
If needs are changing, then this isn’t about being aware of your needs so much as the gift changing your needs. People (your family or otherwise) see you get a windfall, they expect you to step up, increasing your needs. It’s not about being aware of the new needs so much as them existing.
If the needs aren’t changing, then what changed was both your level of needs (slightly better off) and your awareness of your additional needs (better information, which is a mixed bag). You might feel worse off, but you are actually better off other than the feeling and the resulting stress. Is ignorance bliss?
Increasingly it seems like these two things are being conflated, in many contexts. IN this model, you have a problem if and only if you are aware of the problem.
This explains why people think things are so much worse in America today than ever, along dimensions where this plainly isn’t true. But it is true if you only consider the problem to be the gap between reality and expectations, or alternatively the gap between reality and what is considered acceptable.
In the excellent Schrodenger’s Cat Trilogy this is referred to as the revolution of rising expectations, except in the trilogy the timelines where times were good meant everyone adjusted and had high expectations and this led to good outcomes, whereas in the timelines where times were bad everyone adjusted, lowered expectations and became small and petty. That is certainly another important dynamic, but here it is the thing where (and I need to formalize this more another time) our expectations are rising faster than our reality, and also shifting in ways incompatible with maintaining that reality, causing the reality to break down as people forget how nice our things are and cut out the ways in which those things are created and maintained.
This is an underestimated general problem in the world.
It seems plausible that in Chicago people’s marginal consumption options are things they’d like to consume but not things that one cannot do without, whereas in Kenya marginal consumption is considerably more vital – by comparison even poor people in Chicago are vastly richer and have much higher consumption. Structurally, I am also guessing their problems are much less likely to be primarily caused by lack of funds, as opposed to other structural problems including lack of human or social capital.
Anyway, the dilemma here is that by shifting from no money to temporarily some money, people suddenly are forced to be aware of their financial doom and needs, and this ends up making them feel worse. It does this to such a large extent, in this theory, that such people end up overspending the endowment, which we know because their debt levels increased. One might hope this was due to buying real estate (a few people buying a house or apartment goes a long way in an average) but the other context makes this seem unlikely.
Reading the paper, I find it interesting that almost everyone had previously gotten a payment for $500 from the same organization, which potentially caused strange expectation setting.
All participants had previously applied to the organization for COVID-19 relief funds (and thus, presumably needed money). The ways in which people learned about and came to apply to the non-profit varied, with some being referred by municipal agencies (e.g., housing authorities) or social service agencies, and others conducting internet searches. About 43% of our participants (2,237 people) received funds from the non-profit before the trial began. The overwhelming majority (99%) of these participants received $500, with a median of 243 days between a participant’s pre-trial payment and the start of the trial. Less than 1% (45 participants) received a payment in under four months of when they began the trial.
It also is a big selection effect. Applying to an organization for Covid-19 relief funds means you are probably in need, sufficiently that it is highly unlikely a few thousand can solve your problems. It also means that you responded to that need by applying for relief funds. One could argue it means you’re more likely to be on the edge of disaster, but the flip side is that anyone with the wherewithal to apply successfully probably isn’t as likely to end up with a true disaster. That is a particular mentality and approach, that suggests a lack of hope for transformational effects. It even suggests potential substitution effects with other sources of funds.
There is also the dynamic where getting the money made people work less hard and less effectively. That makes sense, and is one easy way for this to backfire. Especially during Covid it makes sense that those with the option to work less would do so, but they also quickly spent all the money and there wasn’t much there to begin with. Also especially with a group that applies for funding assistance, which suggests differentially doing labor to get dollars in response to a shortage of dollars.
Additionally, I wonder if the very act of taking the surveys helped force awareness of financial problems.
Copenhagen Once Again
That all takes us back to the Copenhagen Interpretation of Ethics. Not interacting with a problem means the problem can be ignored. Most of the financial problems of people in modern society are a combination of long term doom (which can of course be happily ignored, often for an arbitrarily long time, before the music stops) and the problem of rising expectations and wanting things (in the sense that a properly Buddhist attitude would largely help, whether or not that would be wise or possible).
So ‘don’t look at it, maybe it’ll go away’ is a less stupid strategy than it appears. Stressing about money is painful, and perhaps is a large percentage of the cost of being short of money, so one can hope not to realize it by not interacting with or not noticing one’s ‘needs’ aren’t being met, even if they are indeed needs and even if that need is ‘not eventually running out of money.’ And also choices are bad even in good times in good spots, it’s that much worse with no good answers.
This matches my experience. On reflection it is totally plausible that help that is actually helpful, but not adequate and that forces attention and choice, could end up being a net negative in practice, even if it would be very good in theory if the choice got taken away. This could be an argument for some sort of in-kind benefit, but if that freed up capital in turn because money is fungible then it doesn’t help either.
We can contrast this with the stimulus checks and the expanded child tax credit. It seems very clear, with comparisons to Europe and by looking at people’s experiences generally, even without a control group, that these checks did quite a lot of short term good for the people who needed them, regardless of the unfortunate side effects of the extra spending or disincentives towards work. And both took place during the pandemic.
Yet the stimulus checks and expanded credit were similar in size to the $500 or $2,000 payments. Does anyone think people would have not been worse off (again, in short term) without those checks? The child tax credit in particular seems to have been very effective at relieving child poverty (although perhaps we should verify the extent to which this is a number versus a real effect?).
Thus I go back to being reasonably confused by the whole thing. I see all the cute arguments, but if money actually was harmful then a better full explanation is required. These stories of how it might backfire aren’t nothing, but they aren’t much.
Another thing I’m wondering about is whether there are long term positive benefits to being forced to confront your needs, if that is indeed what is largely happening. One could tell a story where this has a very positive effect, including on earnings, which would be the opposite of what the data shows.
Explanations that come to mind there are that people, especially during the pandemic but increasingly so in general, have lost the core belief that the future exists and is real and matters and you can change it, which makes windfalls greatly reducing of effort in ways that are bad in the long term, and also that a windfall teaches people that life is arbitrary and doesn’t reward work or punish lack of work, so they work less.
And maybe grants of different sizes with different contexts send people very different messages, and we need to take the importance of such messaging more seriously.
Future Expectations
As I’ve been making an effort to explore and model how voters think, I’ve noticed that polls are useful, but focus groups have been vital to being able to build models that make any sense. You need true ‘market contact’ that isn’t about taking numbers, but actually looking at individual data points. My guess is the way to understand the situation better is to try and actually interview people in depth.
The experimenters are next going to try $500/month for a few years. That’s still not on the level of the GiveWell grants in Kenya, but it is a big step up in size. I don’t expect transformational effects but I do expect a noticeable and positive one. If that still lacks a positive effect, I would be very confused.
How much did this make me less excited about giving people money writ large? Somewhat, but not too much because I don’t expect the salient alternatives to offer better prospects. The hope might be that in-kind offerings don’t increase people’s perception of need in the same way as cash, but they do that mostly by not being fungible and also not being what is actually desired. If there were public goods options that were already almost good, the observations here might push them over the edge by comparison but in practice I don’t find such decisions are that likely to be close.
Mostly I conclude that small one-time transfers are likely to get offset due to a combination of poverty traps, including perception of need but centrally hyperbolic discounting. When you give a one-time payment or even simply have money in the bank at all, most cultural norms involve acting richer than you are, and heavy pressure to spend, usually not efficiently. There isn’t proper appreciation for how much that money will be worth to you when things get tough again, but then again that is only valuable to you to the extent you can resist the pressures to spend. Part of the trap is that once you’re in the trap trying and failing to get out of it doesn’t help you much, so traits that would help in abundance don’t have a hill they can climb.
Thus, one wants to focus on either giving people money continuously, in ways that don’t create a false wealth effect that causes overspending, or to give them enough at once to outright escape the trap. Either way seems plausible.
This also makes it that much more likely that the stimulus checks got spent and acted as proper stimulus, but also didn’t do as much good for those who got them as we might have liked.
Important update from reading the paper: Figure A3 (the objective and subjective outcomes chart) is biased against the cash-receiving groups and can’t be taken at face value. Getting money did not make everything worse. The authors recognize this; it’s why they say there was no effect on the objective outcomes (I previously thought they were just being cowards about the error bars).
The bias is from an attrition effect: basically, control-group members with bad outcomes disproportionately dropped out of the trial. Search for ‘attrition’ in the paper to see their discussion on this.
This doesn’t erase the study; the authors account for this and remain confident that the cash transfers didn’t have significant positive impacts. But they conclude that most or all of the apparent negative impacts are probably illusory.
20% of the control group dropped out of the study while only 10% or 12% of those receiving cash did. This is a major between-groups attrition effect! I’m not as sure that there were no positive effects: even a very small proportion dropping out due to severe negative effects of poverty would completely reverse the effects they were able to measure.
This probably doesn’t affect the finding that the amount of cash received ($500 or $2000) did not greatly affect measurable outcomes, though. The attrition rate was very similar between those two groups.
I notice that the majority of extra spending during the trial was not even approximately tracked, as it fell into the uninformative category of “transfers”. Not nearly enough to verify whether the previously stated intended use of the money was actually borne out in practice. That said, the major intended spending categories that were reported substantially more often in the payment groups than control (housing and bills, about +10% each) almost certainly do fall under “debt”.
I’m really curious if their count of debt included money owed but not yet tracked by credit bureaus (e.g. modestly late utilities or medical payments), informal loans (e.g. from family), or informal loans the recipient had perhaps given up on until they found out you had money).
EDIT: interesting discussion of this on twitter, getting into situations that could never have been tracked as debt per se, but in the end are claims on your money by other people that your extended social network will view as legitimate and punish you if you don’t fill (and that punishment is worse than losing the money). I think an individual or nuclear family might just be the wrong unit of intervention or measurement.
(Epistemic status: vague recollections that I’m not going to bother to look up, also I skimmed this post rather than reading it thoroughly)
I remember people in the GiveDirectly programs in Africa spending the money on things like replacing a thatched roof with a tin roof, or buying a cow. These are investments that can improve your well-being long-term (e.g. because your roof used to leak every time it rained, and now it’s not going to leak again for five years). I can’t think of anything comparable that a person in the US could do with a ~comparable lump sum.
The tin roof thing seems roughly comparable to buying a house outright (so that you don’t have to deal with recurring payments), but that’s orders of magnitude more expensive. I guess maybe you could… buy an RV/trailer to live in? But you can’t easily live in an RV in the city. The buying a cow thing seems roughly comparable to getting a better-paying job (the one-time investment gives you the chance to increase your quality of life, but you still have to put in the work to maintain it). Maybe $500-$2000 could buy you enough time off of work to look for a better job? But that’s not as straightforward as buying a cow.
Also, do the transfer recipients in Kenya generally have debts they need to pay off? That seems like a major and important difference
One hypothesis is that one ends up being stuck in a first-order phase transition where the quantitative changes are not visible until enough energy is poured into the system (or removed from it). You can get all kinds of small and spurious effects, but nothing meaningful. If so, one would expect a discontinuous effect once the cash infusions become large enough or frequent enough to finish the transition and go into a more linear regime.
This feels right to me, and I think matches up well with:
The discussion in Elizabeth’s comment, about informal debts and other invisible money-sinks in the local environment;
Mingyuan’s comment about … something like, the relative straightforwardness, in the two environments, of turning cash into permanent value.
Unfortunately, it feels like the phase transition might be “escaping the local community”, in the model where (as discussed in Elizabeth’s twitter thread) the local community expects resources to be shared, and represents roughly a bottomless sink on individual resources. (As well as a source, when needed—it acts like a buffer with capacity far exceeding what you could give an individual in a program like this as a one-shot payment.)
This vaguely suggests that “enough money to move somewhere with better opportunities” ought to be a major threshold where effects should start showing up, if they haven’t already. Both because it separates someone from their existing community (removing the community-buffer factor), and because it overcomes the problem of limited opportunities to invest the money in something of durable value.
Wasn’t the groups selected for having currently low income? Shouldn’t we expect some regression towards the mean i.e. an increase in average bank balance? Was there any indication for if the observed effect was larger or smaller than expected?
Note that after day 120 or so, all three groups’ balances decline together. Not sure what that’s about.
The control group goes up, and then back down. The payments were made in waves, and it doesn’t say the exact dates. But it’s possible that they were on average made a little bit before a stimulus payment.
The predicted effect sizes (.16-.65 SDs) seem too large, compared to (e.g.) the size of the linear relationship between log(income) and well-being that many studies find. I would’ve expected a positive effect rather than zero or a negative effect, but probably on the low end of that range or below it, depending on the specific question.
In the Rich Dad/Poor Dad frame, some people, when they have money, spend that money multiple times (eg, with a bank balance of $500 they charge up $300 on thier credit card, and withdraw $300 in cash, and commit to a monthly $20 subscription box, and plan a party for next month that will cost $300).