To what degree can we extrapolate our biological inclinations to a species of intelligence which will lack the evolutionary basis of them? Put another way: how much of human camaraderie and benevolence are genetically selected drives as opposed to inherent properties of intelligence? (I’m not saying that any nonhuman intelligence or designed intelligence is fundamentally inscrutable or alien, nor that we cannot possibly imbue biological drives into an artificial intelligence)
Might alignment someday (for a time?) resemble a kind of highly functional alliance, based on the idea that humans have, materially, most of the stuff, and are extremely dangerous? A rudimentary scenario would be an ASI who would like access to resources humans possess, and cannot reasonably avoid getting nuked in the process of taking them? I am aware that this framing may fundamentally differ from the concept of alignment as is often posited, it is not a long term solution, but I believe it’s reasonable to see it as an outcome, given how much of human interaction on the macro and micro seems to be based off flavors of mutually assured destruction.
I appreciate this! I wonder on the following:
To what degree can we extrapolate our biological inclinations to a species of intelligence which will lack the evolutionary basis of them? Put another way: how much of human camaraderie and benevolence are genetically selected drives as opposed to inherent properties of intelligence? (I’m not saying that any nonhuman intelligence or designed intelligence is fundamentally inscrutable or alien, nor that we cannot possibly imbue biological drives into an artificial intelligence)
Might alignment someday (for a time?) resemble a kind of highly functional alliance, based on the idea that humans have, materially, most of the stuff, and are extremely dangerous? A rudimentary scenario would be an ASI who would like access to resources humans possess, and cannot reasonably avoid getting nuked in the process of taking them? I am aware that this framing may fundamentally differ from the concept of alignment as is often posited, it is not a long term solution, but I believe it’s reasonable to see it as an outcome, given how much of human interaction on the macro and micro seems to be based off flavors of mutually assured destruction.