I really appreciated Beren’s post to figure out what I think about this for myself. I think another intuition where I and Beren differ is that it is just fine if 1% of enhanced humans you make is a psychopath or whatever (10% is a bit high though. I don’t expect it to be that high unless you allow the data for you GWAS to have a high % of people cheating on tests or misreporting data (which is a valid concern)). The other enhanced humans can deal with the psychopaths! The human psychopaths would like to solve alignment too and are incentivised to cooperate with the other humans. This is not true with AIs because they are so easily changed and copied. The one project that isn’t careful with their AI’s can then spoil it for everyone else when their Pythia overtakes the universe, while the safe projects work on interp and conceptual foundations. With even mildly superhuman AI (we have superhuman hacking now), you would have to be super paranoid that the AI didn’t poison it’s training data or did anything else mischievous. Meanwhile, as a thought experiment I would feel quite fine retiring to childcare and handing off the future to say ~10 clones of myself that have been enhanced in health and intelligence by ~2-4sd (clones would minimize difference from myself, multiple means less noise in the handover, assuming we did this through editing I could even edit different SNPs for every clone, so there is no consistent misalignment between them and me).
I do think some goal shift from intelligence enhancement is possible and we should be able to get some data on this already from looking at existing humans.
One intuition I have is that if higher intelligence leads to weird things when it comes to goal misgeneralization, we should see more difference in value that start emerging in humans during puberty compared to in early childhood. I’d be curious if someone has compared this between twins (monozygous twins being more different typical puberty related emotions like parental defiance and their sexuality compared to their relation to anger?) or has checked for consistent trends in puberty being more weird in smarter people compared to early childhood development.
I really appreciated Beren’s post to figure out what I think about this for myself. I think another intuition where I and Beren differ is that it is just fine if 1% of enhanced humans you make is a psychopath or whatever (10% is a bit high though. I don’t expect it to be that high unless you allow the data for you GWAS to have a high % of people cheating on tests or misreporting data (which is a valid concern)). The other enhanced humans can deal with the psychopaths! The human psychopaths would like to solve alignment too and are incentivised to cooperate with the other humans. This is not true with AIs because they are so easily changed and copied. The one project that isn’t careful with their AI’s can then spoil it for everyone else when their Pythia overtakes the universe, while the safe projects work on interp and conceptual foundations. With even mildly superhuman AI (we have superhuman hacking now), you would have to be super paranoid that the AI didn’t poison it’s training data or did anything else mischievous. Meanwhile, as a thought experiment I would feel quite fine retiring to childcare and handing off the future to say ~10 clones of myself that have been enhanced in health and intelligence by ~2-4sd (clones would minimize difference from myself, multiple means less noise in the handover, assuming we did this through editing I could even edit different SNPs for every clone, so there is no consistent misalignment between them and me).
I do think some goal shift from intelligence enhancement is possible and we should be able to get some data on this already from looking at existing humans. One intuition I have is that if higher intelligence leads to weird things when it comes to goal misgeneralization, we should see more difference in value that start emerging in humans during puberty compared to in early childhood. I’d be curious if someone has compared this between twins (monozygous twins being more different typical puberty related emotions like parental defiance and their sexuality compared to their relation to anger?) or has checked for consistent trends in puberty being more weird in smarter people compared to early childhood development.