There is one more point. If number of AI teams is very large, like 1000, when several of them will come to AI always simultaneously and there will be several AI take off simultaneously, which could result in the world divided between different AIs. It may be bad, if they go to war with each other, or good if they will have different value systems, as some of their value systems will be more human friendly than others.
Other AIs will probably take in account values of the most friendly AI, as they will be able to trade with it, like “selling” humans to it or do other beneficial things to humans on their territories in exchange for paperclips (or whatever).
TLDR: if we increase number of AI teams, we will create multipolar AI world and humans will be a currency in it. Profit: some humans survive.
Multipolar worlds are some the hardest to reason about, and it’s not clear whether they will cause FAI to have to sacrifice humans or uFAI to have to go easy on us.
There is one more point. If number of AI teams is very large, like 1000, when several of them will come to AI always simultaneously and there will be several AI take off simultaneously, which could result in the world divided between different AIs. It may be bad, if they go to war with each other, or good if they will have different value systems, as some of their value systems will be more human friendly than others.
Other AIs will probably take in account values of the most friendly AI, as they will be able to trade with it, like “selling” humans to it or do other beneficial things to humans on their territories in exchange for paperclips (or whatever).
TLDR: if we increase number of AI teams, we will create multipolar AI world and humans will be a currency in it. Profit: some humans survive.
Multipolar worlds are some the hardest to reason about, and it’s not clear whether they will cause FAI to have to sacrifice humans or uFAI to have to go easy on us.