I like this exploration, but I think you need to go a little deeper into the difference between utility and outcome. You probably shouldn’t entertain non-transitive preferences and still call it “rational”—that’s provably inefficient. But you really need to include non-linear valuation of world-states.
It’s perfectly consistent with rationality and utility maximization to value one life more than you value a distribution of 0 or 1000 that’s mostly zeros. This implies a great value to you of saving one person, and a lesser value for saving numbers 2-1000. There’s a LOT of cases where declining marginal value kicks in like this. It’s a common assumption that many things have logarithmic utility, and there’s some interesting math about how to maximize expectation in that case—https://en.wikipedia.org/wiki/Kelly_criterion .
Note that the expected utility framework is often reverse-engineered, not forward-enforced. If your actions imply a consistent utility function, then you are exhibiting the underlying axioms of rational decisions. It’s about consistency of decision, not about the internal process. In that sense, the rock is rationally pursuing a preference to just sit there. It never takes an action inconsistent with that utility curve. See https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem for the axioms—transitivity is the most common one for humans to violate, showing their irrationality.
Side-note, not changing the main point: If the lives are close to fungible for you (they’re distant strangers) and reputation effects don’t come into play (nobody will find out about your decision), then it’s hard to credit the implication that you care so much about the one that you’ll let the expected 99 you could have saved die—that’s probably scope insensitivity rather than reasoned valuation of outcomes. One of the best outcomes of learning this topic well is the ability to recognize inconsistency in yourself, as areas of personal growth to improve. You might improve by understanding your utility and valuation more clearly (you’re imagining you know the one, and don’t know the 999, or you’re thinking about credit and how you’re perceived, or many other factors). Or by realizing that some of your intuitions are wrong, and re-training yourself, in order to get better at improving the state of the world (according to your preferences).
I like this exploration, but I think you need to go a little deeper into the difference between utility and outcome. You probably shouldn’t entertain non-transitive preferences and still call it “rational”—that’s provably inefficient. But you really need to include non-linear valuation of world-states.
It’s perfectly consistent with rationality and utility maximization to value one life more than you value a distribution of 0 or 1000 that’s mostly zeros. This implies a great value to you of saving one person, and a lesser value for saving numbers 2-1000. There’s a LOT of cases where declining marginal value kicks in like this. It’s a common assumption that many things have logarithmic utility, and there’s some interesting math about how to maximize expectation in that case—https://en.wikipedia.org/wiki/Kelly_criterion .
Note that the expected utility framework is often reverse-engineered, not forward-enforced. If your actions imply a consistent utility function, then you are exhibiting the underlying axioms of rational decisions. It’s about consistency of decision, not about the internal process. In that sense, the rock is rationally pursuing a preference to just sit there. It never takes an action inconsistent with that utility curve. See https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem for the axioms—transitivity is the most common one for humans to violate, showing their irrationality.
Side-note, not changing the main point: If the lives are close to fungible for you (they’re distant strangers) and reputation effects don’t come into play (nobody will find out about your decision), then it’s hard to credit the implication that you care so much about the one that you’ll let the expected 99 you could have saved die—that’s probably scope insensitivity rather than reasoned valuation of outcomes. One of the best outcomes of learning this topic well is the ability to recognize inconsistency in yourself, as areas of personal growth to improve. You might improve by understanding your utility and valuation more clearly (you’re imagining you know the one, and don’t know the 999, or you’re thinking about credit and how you’re perceived, or many other factors). Or by realizing that some of your intuitions are wrong, and re-training yourself, in order to get better at improving the state of the world (according to your preferences).