From evolution’s perspective, humans in the EEA would have looked like they were doing a good job of optimizing inclusive fitness.
I’m pretty sure that if you evaluated every action a human ever took, in terms of its contributions to improving genetic fitness, you’d pretty reliably find the humans lacking. And not just because they weren’t smart enough to come up with the right strategies for having more children. It was probably clear, even to the humans of the time, that lots of them cared about things other than fitness, which would conflict with fitness if extrapolated outward. E.g., wanting to impress your friends by doing stupid stunts that might seriously injure you, or picking a fight with some other band of humans, who you might in turn get into a pointless war with.
Gradient descent, as yielded by a decent reward model, would look at these behaviors, and immediately edit every parameter in the algorithm that generated them, to make them less likely. Evolution has to spend generations waiting for the right mutations to reduce their prevalence.
I’m pretty sure that if you evaluated every action a human ever took, in terms of its contributions to improving genetic fitness, you’d pretty reliably find the humans lacking. And not just because they weren’t smart enough to come up with the right strategies for having more children. It was probably clear, even to the humans of the time, that lots of them cared about things other than fitness, which would conflict with fitness if extrapolated outward. E.g., wanting to impress your friends by doing stupid stunts that might seriously injure you, or picking a fight with some other band of humans, who you might in turn get into a pointless war with.
Gradient descent, as yielded by a decent reward model, would look at these behaviors, and immediately edit every parameter in the algorithm that generated them, to make them less likely. Evolution has to spend generations waiting for the right mutations to reduce their prevalence.