It’s worth exploring exactly which resources are under competition. Humans have killed orders of magnitude more ants than Neanderthals, but the overlap in resources is much less complete for ants, so they’ve survived.
Grey-goo-like scenarios are scary because resource contention is 100% - there is nothing humans want/need that the goo doesn’t want/need, in ways that are exclusive to human existence. We just don’t know how much resource-use overlap there will be between AI and humans (or some subset of humans), and fast-takeoff is a little more worrisome because there’s far less opportunity to find areas of compromise (where the AI values human cooperation enough to leave some resources to us).
It’s worth exploring exactly which resources are under competition. Humans have killed orders of magnitude more ants than Neanderthals, but the overlap in resources is much less complete for ants, so they’ve survived.
Grey-goo-like scenarios are scary because resource contention is 100% - there is nothing humans want/need that the goo doesn’t want/need, in ways that are exclusive to human existence. We just don’t know how much resource-use overlap there will be between AI and humans (or some subset of humans), and fast-takeoff is a little more worrisome because there’s far less opportunity to find areas of compromise (where the AI values human cooperation enough to leave some resources to us).