I expect to find that memorization and compression eventually make an agi wish to become an archeologist and at minimum retain memory of humanity in a way that an initially misaligned hard asi might not; but this is little reassurance, as most pivotal anti-humanity acts are within the reach of a soft asi that will fail to care for itself after taking over. the most likely outcome I currently see is an incremental species replacement where ai replaces humanity as the world’s dominant species over the next 20 years. no strongly coherent planning ai need come into existence in that time in order for a weakly consistent but strongly capable ai to kill or permanently disempower humanity.
I expect to find that memorization and compression eventually make an agi wish to become an archeologist and at minimum retain memory of humanity in a way that an initially misaligned hard asi might not; but this is little reassurance, as most pivotal anti-humanity acts are within the reach of a soft asi that will fail to care for itself after taking over. the most likely outcome I currently see is an incremental species replacement where ai replaces humanity as the world’s dominant species over the next 20 years. no strongly coherent planning ai need come into existence in that time in order for a weakly consistent but strongly capable ai to kill or permanently disempower humanity.