Value drift involves extrapolation of values to strange situations based on things other than endorsement by current values from the comfort of familiar situations. Running this risk doesn’t require building different agents, like with natural selection and humans, or humans and AIs, or humans modifying their own cognition or that of their descendants. Rapid permanent change of environment is sufficient, and AIs introduce this risk for both humans and for themselves even when aligned, until they are smart enough to anticipate it.
Value drift involves extrapolation of values to strange situations based on things other than endorsement by current values from the comfort of familiar situations. Running this risk doesn’t require building different agents, like with natural selection and humans, or humans and AIs, or humans modifying their own cognition or that of their descendants. Rapid permanent change of environment is sufficient, and AIs introduce this risk for both humans and for themselves even when aligned, until they are smart enough to anticipate it.