New Proposal (although I think I see the flaw already):
-Create x “Friendly” AI, where x is the total number of people in the world. An originator AI is designed to create 1 of each such one equal to the number of humans in the world, then create new ones every time another human comes into being.
-Each “Friendly” AI thus created is “attached” to one person in the world in that it is programmed to constantly adjust it’s utility function to that person’s wants. All of them have equal self-enhancement potential, and have two programs.
-Program A is the “Dominant” program, and has complete control of the utility function but nothing else. Program B cannot defy Program A. Program A is constantly adjusting Program B’s utility function to be identical to that of the person it is “attached” to, and has no other priorities.
-Program B’s priorities are determined by Program A.
One such set of Program A and Program B is ‘attached’ to every individual in the world. Being of equal intelligence, in THEORY they should compromise in order to avoid wasting resuorces.
Some AIs have access to slightly more resources than others, owing perhaps to humans offering varying levels of assistance to their own AIs. Other AIs just get lucky and have good insights into intelligence enhancement before the rest. These differences escalate as the smarter AIs are now able to grab more resources and become even smarter. Within a week one random person has become dictator of earth.
One such set of Program A and Program B is ‘attached’ to every individual in the world. Being of equal intelligence, in THEORY they should compromise in order to avoid wasting resources.
Ecosystems of many cooperating agents only work so long as either they all have similar goals, or there is a suitable balance between offense and defense. This particular example fails if there is any one person in the world who wants to destroy it, because their AI can achieve this goal without having to compromise or communicate with any of the others.
New Proposal (although I think I see the flaw already): -Create x “Friendly” AI, where x is the total number of people in the world. An originator AI is designed to create 1 of each such one equal to the number of humans in the world, then create new ones every time another human comes into being.
-Each “Friendly” AI thus created is “attached” to one person in the world in that it is programmed to constantly adjust it’s utility function to that person’s wants. All of them have equal self-enhancement potential, and have two programs. -Program A is the “Dominant” program, and has complete control of the utility function but nothing else. Program B cannot defy Program A. Program A is constantly adjusting Program B’s utility function to be identical to that of the person it is “attached” to, and has no other priorities. -Program B’s priorities are determined by Program A.
One such set of Program A and Program B is ‘attached’ to every individual in the world. Being of equal intelligence, in THEORY they should compromise in order to avoid wasting resuorces.
Some AIs have access to slightly more resources than others, owing perhaps to humans offering varying levels of assistance to their own AIs. Other AIs just get lucky and have good insights into intelligence enhancement before the rest. These differences escalate as the smarter AIs are now able to grab more resources and become even smarter. Within a week one random person has become dictator of earth.
Ecosystems of many cooperating agents only work so long as either they all have similar goals, or there is a suitable balance between offense and defense. This particular example fails if there is any one person in the world who wants to destroy it, because their AI can achieve this goal without having to compromise or communicate with any of the others.