In a very real sense, wouldn’t an AGI itself be a descendant of humanity? It’s not obvious, anyway, that there would be big categorical differences between an AGI and humanity 200+ years down the road after we’ve been merged/cyborged/upgraded, etc., to the hilt, all with technologies made possible by the AGI. This goes back to Phil’s point above—it seems a little short-sighted to place undo importance on the preservation of this particular incarnation, or generation, of humanity, when what we really care about is some fuzzy concept of “human intelligence” or “culture.”
In a very real sense, wouldn’t an AGI itself be a descendant of humanity? It’s not obvious, anyway, that there would be big categorical differences between an AGI and humanity 200+ years down the road after we’ve been merged/cyborged/upgraded, etc., to the hilt, all with technologies made possible by the AGI. This goes back to Phil’s point above—it seems a little short-sighted to place undo importance on the preservation of this particular incarnation, or generation, of humanity, when what we really care about is some fuzzy concept of “human intelligence” or “culture.”