I currently estimate a 7% chance AI will kill us all this century.
Does this take into account the transitive AI risk, that it’s AGIs developed by AGIs developed by AGIs… that kill us? It’s not just the AGIs we build that need to be safe, but also the AGIs they build, and the AGIs that those AGIs build, and so on. Most classical AI risk arguments seem to characterize what might go wrong with those indirectly developed AGIs, even if they don’t directly apply to the more likely first AGIs like LLM human imitations.
Does this take into account the transitive AI risk, that it’s AGIs developed by AGIs developed by AGIs… that kill us? It’s not just the AGIs we build that need to be safe, but also the AGIs they build, and the AGIs that those AGIs build, and so on. Most classical AI risk arguments seem to characterize what might go wrong with those indirectly developed AGIs, even if they don’t directly apply to the more likely first AGIs like LLM human imitations.