And in the same stroke that its capabilities leap forward, its alignment properties are revealed to be shallow, and to fail to generalize. The central analogy here is that optimizing apes for inclusive genetic fitness (IGF) doesn’t make the resulting humans optimize mentally for IGF. Like, sure, the apes are eating because they have a hunger instinct and having sex because it feels good—but it’s not like they could be eating/fornicating due to explicit reasoning about how those activities lead to more IGF. They can’t yet perform the sort of abstract reasoning that would correctly justify those actions in terms of IGF. And then, when they start to generalize well in the way of humans, they predictably don’t suddenly start eating/fornicating because of abstract reasoning about IGF, even though they now could. Instead, they invent condoms,
This is a super bizarre argument to make about humans considering that we are orders of magnitude more successful by any IGF metric than other apes, and clearly one of the most IGF-successful species ever.
Optimizing apes for inclusive genetic fitness did make the resulting humans optimize mentally for IGF (through various mixes of implicit habitual and explicit model-based reasoning, as is computationally efficient). Across history humans have consciously planned to further their bloodlines, and arguably that was the primary goal of most typical elites/nobles throughout most of history. This analogy supports the opposite of your point.
Invention of condoms is hardly evidence that humanity as a whole stopped optimizing for IGF: having unwanted children often doesn’t maximize IGF. Today there are some humans who are leveraging modern technology to better explicitly optimize for IGF, allowing them to have as many children as kings of old.
Evolution proceeds by variation followed by selection. Our current enormous success entails we are in a variation dominated regime, so it’s only expected that there are a wide variety of strategies being explored, most of which won’t optimize well for IGF. But given time eventually the few that do will dominate (or they would in worlds without AI and the postbiological transition).
This is a super bizarre argument to make about humans considering that we are orders of magnitude more successful by any IGF metric than other apes, and clearly one of the most IGF-successful species ever.
Optimizing apes for inclusive genetic fitness did make the resulting humans optimize mentally for IGF (through various mixes of implicit habitual and explicit model-based reasoning, as is computationally efficient). Across history humans have consciously planned to further their bloodlines, and arguably that was the primary goal of most typical elites/nobles throughout most of history. This analogy supports the opposite of your point.
Invention of condoms is hardly evidence that humanity as a whole stopped optimizing for IGF: having unwanted children often doesn’t maximize IGF. Today there are some humans who are leveraging modern technology to better explicitly optimize for IGF, allowing them to have as many children as kings of old.
Evolution proceeds by variation followed by selection. Our current enormous success entails we are in a variation dominated regime, so it’s only expected that there are a wide variety of strategies being explored, most of which won’t optimize well for IGF. But given time eventually the few that do will dominate (or they would in worlds without AI and the postbiological transition).