So, if an Everett-branches traveller told me “well, you know, MIRI folks had the best intentions, but in your branch, made the field pay attention to unproductive directions, and this made your civilization more confused and alignment harder” and I had to guess “how?”, one of the top choices would be ongoing strawmanning and misrepresentation of Eric Drexler’s ideas.
</rant>
To me, CAIS thinking seems quite different from the description in the op.
Some statements, without much justifications/proofs
- Modularity is a pretty powerful principle/law of intelligent systems. If you look around the world, you see modularity everywhere. Actually in my view you will see more of “modularity” than of “rational agency”, suggesting gods of modularity are often stronger than gods of rational agency. Modularity would help as a lot, in contrast to integrated homogenous agents ⇒ one of the key directions of AI safety should be figuring out how to summon Gods of modularity
- Modularity helps with interpretability; once you have “modules” and “interfaces”, you have much better shot at understanding what’s going on by looking on the interfaces. (For intuitive feel: Imagine you want to make plotting and scheming of three people much more legible, and you can impose this constrain: they need to make all communication between them on Slack, which you can read)
- Any winning strategy needs to solve global coordination at some point, otherwise people will just call a service to destroy the world. Solutions of the type “your aligned superintelligent agent takes over the world and coordinates everyone” are dangerous and won’t work; superintelligent agent able takes over the world is something you don’t want to bring into existence, and in contrast, you need a security layer to prevent anyone from event attempting that
- There are multiple hard problems; attempting to solve them all at once in one system is not the right tactic. In practice, we want to isolate some problems to separate “modules” or “services”—for example, we want a separate “research and development services”, “security service”, “ontology mapping service”,...
- Many hard problems don’t disappear, but there are also technical solutions for them [e.g. distillation]
<sociology of AI safety rant>
So, if an Everett-branches traveller told me “well, you know, MIRI folks had the best intentions, but in your branch, made the field pay attention to unproductive directions, and this made your civilization more confused and alignment harder” and I had to guess “how?”, one of the top choices would be ongoing strawmanning and misrepresentation of Eric Drexler’s ideas.
</rant>
To me, CAIS thinking seems quite different from the description in the op.
Some statements, without much justifications/proofs
- Modularity is a pretty powerful principle/law of intelligent systems. If you look around the world, you see modularity everywhere. Actually in my view you will see more of “modularity” than of “rational agency”, suggesting gods of modularity are often stronger than gods of rational agency. Modularity would help as a lot, in contrast to integrated homogenous agents ⇒ one of the key directions of AI safety should be figuring out how to summon Gods of modularity
- Modularity helps with interpretability; once you have “modules” and “interfaces”, you have much better shot at understanding what’s going on by looking on the interfaces. (For intuitive feel: Imagine you want to make plotting and scheming of three people much more legible, and you can impose this constrain: they need to make all communication between them on Slack, which you can read)
- Any winning strategy needs to solve global coordination at some point, otherwise people will just call a service to destroy the world. Solutions of the type “your aligned superintelligent agent takes over the world and coordinates everyone” are dangerous and won’t work; superintelligent agent able takes over the world is something you don’t want to bring into existence, and in contrast, you need a security layer to prevent anyone from event attempting that
- There are multiple hard problems; attempting to solve them all at once in one system is not the right tactic. In practice, we want to isolate some problems to separate “modules” or “services”—for example, we want a separate “research and development services”, “security service”, “ontology mapping service”,...
- Many hard problems don’t disappear, but there are also technical solutions for them [e.g. distillation]