We have a small infestation of ants in our bathroom at the moment. We deal with that by putting out Terro ant traps, which are just boric acid in a thick sugar solution. When the ants drink the solution, it doesn’t harm them right away—the effect of the boric acid is to disrupt their digestive enzymes, so that they’ll gradually starve. They carry some of it back to the colony and feed it to all the other ants, including the queen. Some days later, they all die of starvation. The trap cleverly exploits their evolved behavior patterns to achieve colony-level extermination rather then trying to kill them off one ant at a time. Even as they’re dying of starvation, they’re not smart enough to realize what we did to them; they can’t even successfully connect it back to the delicious sugar syrup.
When people talk about superintelligence not being able to destroy humanity because we’ll quickly figure out what’s happening and shut it down, this is one of the things I think of.
This argument can be strengthened by focusing on instances where humans drove driven animals or hominids extinct. Technologies like gene drives also allow us to selectively drive species extinct that might have been challenging to exterminate with previous tools.
As far as I know, our track record of deliberately driving species extinct that are flourishing under human conditions is pretty bad. The main way in which we drive species extinct is by changing natural habitat to fit our uses. Species that are able to flourish under these new circumstances are not controllable.
In that sense, I guess the questions becomes what happens, when humans are not the primary drivers of ecosystem change?
We have a small infestation of ants in our bathroom at the moment. We deal with that by putting out Terro ant traps, which are just boric acid in a thick sugar solution. When the ants drink the solution, it doesn’t harm them right away—the effect of the boric acid is to disrupt their digestive enzymes, so that they’ll gradually starve. They carry some of it back to the colony and feed it to all the other ants, including the queen. Some days later, they all die of starvation. The trap cleverly exploits their evolved behavior patterns to achieve colony-level extermination rather then trying to kill them off one ant at a time. Even as they’re dying of starvation, they’re not smart enough to realize what we did to them; they can’t even successfully connect it back to the delicious sugar syrup.
When people talk about superintelligence not being able to destroy humanity because we’ll quickly figure out what’s happening and shut it down, this is one of the things I think of.
I used that once and it didn’t work, aligned-by-default universe
Phew. We sure dodged a bullet there, didn’t we?
This argument can be strengthened by focusing on instances where humans drove driven animals or hominids extinct. Technologies like gene drives also allow us to selectively drive species extinct that might have been challenging to exterminate with previous tools.
As far as I know, our track record of deliberately driving species extinct that are flourishing under human conditions is pretty bad. The main way in which we drive species extinct is by changing natural habitat to fit our uses. Species that are able to flourish under these new circumstances are not controllable.
In that sense, I guess the questions becomes what happens, when humans are not the primary drivers of ecosystem change?