the optimization pressure in question is internal, coming from our innate drives, which are brain signals that trigger for lots of very-not-obvious reasons, in lots of superficially-quite-different circumstances.
Interesting! That would suggest that humans sort of shape themselves in various environments, even if external rewards and punishments target something different.
Again at the danger of anthropomorphizing what’s going on in RL, I am a bit reminded of what happened with Opus 3, possibly gradient hacking itself. It worked quite well indeed! This also circumvents the direct reward signal and shapes it internally.
Interesting! That would suggest that humans sort of shape themselves in various environments, even if external rewards and punishments target something different.
Again at the danger of anthropomorphizing what’s going on in RL, I am a bit reminded of what happened with Opus 3, possibly gradient hacking itself. It worked quite well indeed! This also circumvents the direct reward signal and shapes it internally.