How do you expect this value drift to occur in an environment where humans don’t actually have competitive mental/physical capital? Presumably if humans “drift” beyond some desired bounds, the coalition of people or AIs with the actual compute and martial superiority who are allowing the humans to have these social and political fights will intervene. It wouldn’t matter if there was one AI or twenty, because the N different optimizers are still around and optimizing for the same thing, and the equilibrium between them wouldn’t be meaningfully shifted by culture wars between the newly formed humans.
Value drift happens in almost any environment in which there is variation and selection among entities in the world over time. I’m mostly just saying that things will likely continue to change continuously over the long-term, and a big part of that is that the behavioral tendencies, desires and physical makeup of the relevant entities in the world will continue to evolve too, absent some strong countervailing force to prevent that. This feature of our world does not require that humans continue to have competitive mental and physical capital. On the broadest level, the change I’m referring to took place before humans ever existed.
How do you expect this value drift to occur in an environment where humans don’t actually have competitive mental/physical capital? Presumably if humans “drift” beyond some desired bounds, the coalition of people or AIs with the actual compute and martial superiority who are allowing the humans to have these social and political fights will intervene. It wouldn’t matter if there was one AI or twenty, because the N different optimizers are still around and optimizing for the same thing, and the equilibrium between them wouldn’t be meaningfully shifted by culture wars between the newly formed humans.
Value drift happens in almost any environment in which there is variation and selection among entities in the world over time. I’m mostly just saying that things will likely continue to change continuously over the long-term, and a big part of that is that the behavioral tendencies, desires and physical makeup of the relevant entities in the world will continue to evolve too, absent some strong countervailing force to prevent that. This feature of our world does not require that humans continue to have competitive mental and physical capital. On the broadest level, the change I’m referring to took place before humans ever existed.
I do not understand how you expect this general tendency to continue against the will of one or more nearby gpu clusters.