From actual tractable AI architectures, this is it. This is the best we can do. We build tools that are heavily restricted in what they can remember, their every important output checked by other tools constructed a different way, we don’t give any system a “global” RL counter but have it run in short time episodes where it does what it can towards a goal before time limit expires.
This is the reality. Humans will be able to do anything they want. That’s the best alignment can do.
Asking for a “guardian” AGI that rules the planet instead of humans is just choosing suicide. (and so human groups armed with these tool AIs will resort to as much violence as is necessary—no limits—if someone attempts to build such an AGI and allow it to rule.)
There are unfortunately a large number of routes that lead to a new round of global wars once humans have these tools AI systems.
say more about why it would be suicide? it seems to me that it would only be able to succeed through diplomacy, and would take decades before enough trust had been established. but I see no fundamental reason that war and conflict couldn’t be near completely ended forever, once the diffusion of coprotective strategies is thorough enough.
Certainly pivotal acts are intense acts of war and beget retaliation, as you say.
My definition of a guardian AI : machine allowed to do anything it wants, but we tried to make it “be good”. It has military capabilities, can self improve, and we have no edit access.
From actual tractable AI architectures, this is it. This is the best we can do. We build tools that are heavily restricted in what they can remember, their every important output checked by other tools constructed a different way, we don’t give any system a “global” RL counter but have it run in short time episodes where it does what it can towards a goal before time limit expires.
This is the reality. Humans will be able to do anything they want. That’s the best alignment can do.
Asking for a “guardian” AGI that rules the planet instead of humans is just choosing suicide. (and so human groups armed with these tool AIs will resort to as much violence as is necessary—no limits—if someone attempts to build such an AGI and allow it to rule.)
There are unfortunately a large number of routes that lead to a new round of global wars once humans have these tools AI systems.
say more about why it would be suicide? it seems to me that it would only be able to succeed through diplomacy, and would take decades before enough trust had been established. but I see no fundamental reason that war and conflict couldn’t be near completely ended forever, once the diffusion of coprotective strategies is thorough enough.
Certainly pivotal acts are intense acts of war and beget retaliation, as you say.
My definition of a guardian AI : machine allowed to do anything it wants, but we tried to make it “be good”. It has military capabilities, can self improve, and we have no edit access.
That’s suicide. Murder pills etc.