Basically their idea is that instead of having one agent that optimizes the hell out of its value function and bad things happen, have a collection of smaller components that each are working on a subproblem with limited resources. If you can do that and also aggregate them such that as a unit they are superhuman, you get a lot of the benefits without (at least some of) the big risks.
Cooperative AI Systems/Services? Quick google search isn’t finding it, working on only a vague memory.
Comprehensive AI Services.
Summary, original paper.
Yeah, I shoulda linked that. Fixing shortly, thanks to niplav in the meantime!
Basically their idea is that instead of having one agent that optimizes the hell out of its value function and bad things happen, have a collection of smaller components that each are working on a subproblem with limited resources. If you can do that and also aggregate them such that as a unit they are superhuman, you get a lot of the benefits without (at least some of) the big risks.
Here’s a brief explainer with some objections.