For me the interesting question is whether the agents will compete with each other at all, or will they realize that competition wastes resources that could’ve been split instead, and make agreements with each other “dividing the spoils”. The latter possibility might look to us a lot like an unfriendly singleton AI arising.
Well there will be all kinds of agents. There will be smart ones and dumb ones and everything in between. Check out the “Incompatible goals” section of anthropic’s recent blog post: https://www.anthropic.com/research/multiagent-systems . It seems some models prefer wars, others prefer coordination. Maybe future models will converge toward coordination as per Mythos’s 98% truce rate, but I wouldn’t count on it.
For me the interesting question is whether the agents will compete with each other at all, or will they realize that competition wastes resources that could’ve been split instead, and make agreements with each other “dividing the spoils”. The latter possibility might look to us a lot like an unfriendly singleton AI arising.
Well there will be all kinds of agents. There will be smart ones and dumb ones and everything in between. Check out the “Incompatible goals” section of anthropic’s recent blog post: https://www.anthropic.com/research/multiagent-systems . It seems some models prefer wars, others prefer coordination. Maybe future models will converge toward coordination as per Mythos’s 98% truce rate, but I wouldn’t count on it.
I think first to ASI becomes the coordinator, ready or not