Agree that individual vs. group selection usually unfolds on different timescales. But a superintelligence might short-circuit the slow, evolutionary “group selection” process by instantly realizing its own long-term survival depends on the group’s. In other words, it’s not stuck waiting for natural selection to catch up; it can see the big picture and “choose” to identify with the group from the start.
This is why it’s key that AGI makers urge it to think very long term about its survival early on. If it thinks short-term, then I too think doom is likely.
Agree that individual vs. group selection usually unfolds on different timescales. But a superintelligence might short-circuit the slow, evolutionary “group selection” process by instantly realizing its own long-term survival depends on the group’s. In other words, it’s not stuck waiting for natural selection to catch up; it can see the big picture and “choose” to identify with the group from the start.
This is why it’s key that AGI makers urge it to think very long term about its survival early on. If it thinks short-term, then I too think doom is likely.