It occurs to me that the precedent of humans being misaligned is more of a mixed bag than the argument admits.
For one thing, modern humans still consume plenty of calories and reproduce quite a lot. And when we do avoid calories, we may be defying evolution’s mandate to consume, but we are complying with evolution’s mandate to survive and be healthy. Just as “eat tasty things” is a misaligned inner objective relative to “consume calories”, “eat calories” is a misaligned inner objective relative to “survive”. When we choose survival over calories, in some sense we’re correcting our own misalignment.
And then there’s the case of human morality, where we’ve done the exact opposite. “Do the right thing” is an inner objective relative to “survive”, and it too is misaligned. After all, the right thing sometimes includes acts of altruism that don’t help us survive or help our genes reproduce. How exactly this evolved is, of course, a matter of long debate. Now that it has, though, we could in theory correct the misalignment by doing things that make us feel good (innermost objective) but aren’t actually altruistic (middle objective), instead prioritizing our own survival (outermost objective). But for the most part, we don’t. Sure, there are lots of useless activities that hijack our altruism, ranging from virtual pets to moe anime. Society sanctions those – but only because they’re morally neutral. That sanction doesn’t extend to activities that are morally harmful, no matter how good they make us feel.
Even though that’s ultimately a case of misalignment, in some ways it’s a good sign. Morality is something that evolved well before intelligence; thus, it’s analogous to behaviors we currently observe in AI. When intelligence came, instead of subverting it, humans took great pains to extend it as faithfully as possible to the new types of choices we could make, choices far more complex and abstract than what we evolved for. Sometimes we claim we’re doing it for the sake of hallucinated gods, sometimes not, but the principles themselves change little. Often people do things that other people consider immoral, but neither position is necessarily misaligned with the evolutionary origin of morality, which was always flexible and open to violence.
I think you have extended the point of the analogy into unrelated territory. My understanding is the analogy is simply to get people to understand mesa optimization.
Your analysis seems interesting, but very distant from the topic of mesa optimization.
It occurs to me that the precedent of humans being misaligned is more of a mixed bag than the argument admits.
For one thing, modern humans still consume plenty of calories and reproduce quite a lot. And when we do avoid calories, we may be defying evolution’s mandate to consume, but we are complying with evolution’s mandate to survive and be healthy. Just as “eat tasty things” is a misaligned inner objective relative to “consume calories”, “eat calories” is a misaligned inner objective relative to “survive”. When we choose survival over calories, in some sense we’re correcting our own misalignment.
And then there’s the case of human morality, where we’ve done the exact opposite. “Do the right thing” is an inner objective relative to “survive”, and it too is misaligned. After all, the right thing sometimes includes acts of altruism that don’t help us survive or help our genes reproduce. How exactly this evolved is, of course, a matter of long debate. Now that it has, though, we could in theory correct the misalignment by doing things that make us feel good (innermost objective) but aren’t actually altruistic (middle objective), instead prioritizing our own survival (outermost objective). But for the most part, we don’t. Sure, there are lots of useless activities that hijack our altruism, ranging from virtual pets to moe anime. Society sanctions those – but only because they’re morally neutral. That sanction doesn’t extend to activities that are morally harmful, no matter how good they make us feel.
Even though that’s ultimately a case of misalignment, in some ways it’s a good sign. Morality is something that evolved well before intelligence; thus, it’s analogous to behaviors we currently observe in AI. When intelligence came, instead of subverting it, humans took great pains to extend it as faithfully as possible to the new types of choices we could make, choices far more complex and abstract than what we evolved for. Sometimes we claim we’re doing it for the sake of hallucinated gods, sometimes not, but the principles themselves change little. Often people do things that other people consider immoral, but neither position is necessarily misaligned with the evolutionary origin of morality, which was always flexible and open to violence.
I think you have extended the point of the analogy into unrelated territory. My understanding is the analogy is simply to get people to understand mesa optimization.
Your analysis seems interesting, but very distant from the topic of mesa optimization.