It made me think about the way that classic biases are often explained by constructing money pumps. A money pump is taken to be a clear, knock-down demonstration of irrationality, since “clearly” no one would want to lose arbitrarily large amounts of money. But in fact any money pump could be rational if the agent just enjoyed making the choices involved. If I greatly enjoyed anchoring on numbers presented to me, I might well pay a lot of extra money to get anchored; this would be like buying a kind of enjoyable product. Likewise someone might just get a kick out of making choices in intransitive loops, or hyperbolic discounting, or whatever. (In the reverse direction, if you didn’t know I enjoyed some consumer good, you might think I was getting “money pumped” by paying for it again and again.)
So there is a missing step here, and to supply the step we need psychology. The reason these biases are biases and not values is “those aren’t the sort of things we care about,” but to formalize that, we need an account of “the sort of things we care about” which, as you say, can’t be solved for from policy data alone.
This seems like a very good perspective to me.
It made me think about the way that classic biases are often explained by constructing money pumps. A money pump is taken to be a clear, knock-down demonstration of irrationality, since “clearly” no one would want to lose arbitrarily large amounts of money. But in fact any money pump could be rational if the agent just enjoyed making the choices involved. If I greatly enjoyed anchoring on numbers presented to me, I might well pay a lot of extra money to get anchored; this would be like buying a kind of enjoyable product. Likewise someone might just get a kick out of making choices in intransitive loops, or hyperbolic discounting, or whatever. (In the reverse direction, if you didn’t know I enjoyed some consumer good, you might think I was getting “money pumped” by paying for it again and again.)
So there is a missing step here, and to supply the step we need psychology. The reason these biases are biases and not values is “those aren’t the sort of things we care about,” but to formalize that, we need an account of “the sort of things we care about” which, as you say, can’t be solved for from policy data alone.
Thanks, that’s helping me clarify the issues.