Old-school companies, like Anthropic, see the danger in AI superorganisms. Where single AI agents are hard to align, multi-agent superorganisms are near impossible. These companies will choose to slow advancement, favoring smaller corporation-like AI structures with well-tested alignment.
Hmm, I’d argue that multi-agent superorganisms might actually be easier to align, due to being able to read the messages sent between subagents and due to being able to human-design the structure in which the agents are summoned and interact. (You say above that the agents would communicate in uninterpretable ways, which would make the first point moot, so fair enough. I guess I’d say it’s unclear but overall plausible.)
Hmm, I’d argue that multi-agent superorganisms might actually be easier to align, due to being able to read the messages sent between subagents and due to being able to human-design the structure in which the agents are summoned and interact. (You say above that the agents would communicate in uninterpretable ways, which would make the first point moot, so fair enough. I guess I’d say it’s unclear but overall plausible.)