We already deal with entities with theoretically limitless capabilities. They’re called either corporations or states or organizations. Organizations potentially are ever growing.
Of course if AI ever obtained superhuman abilities, the first place these abilities would be deployed is in a corporation or state.
The great AI danger is a corporate danger. Wielding a corporation, the AI automatically obtains all the abilities of the individual humans making up a corporation, and AI can manipulate humanity the traditional way, through money. Any ability the AI lacks, well, AI can just hire the right people to fulfill that niche.
If AI obtains state power, it will manipulate humanity through the other tradition, war and violence.
Organizations can’t spawn copies for linear cost increases, can’t run at faster than human speeds, and generally suck at project management due to incentives. LLM agent systems seem poised to be insanely more powerful.
Organizations can’t even hire Elon Musk—and if they could, they would gain at most 16 of his hours every day—but if an AI obtains his skills, it will be able to simulate thousand Elon Musks.
We already deal with entities with theoretically limitless capabilities. They’re called either corporations or states or organizations. Organizations potentially are ever growing.
Of course if AI ever obtained superhuman abilities, the first place these abilities would be deployed is in a corporation or state.
The great AI danger is a corporate danger. Wielding a corporation, the AI automatically obtains all the abilities of the individual humans making up a corporation, and AI can manipulate humanity the traditional way, through money. Any ability the AI lacks, well, AI can just hire the right people to fulfill that niche.
If AI obtains state power, it will manipulate humanity through the other tradition, war and violence.
Organizations can’t spawn copies for linear cost increases, can’t run at faster than human speeds, and generally suck at project management due to incentives. LLM agent systems seem poised to be insanely more powerful.
Organizations can’t even hire Elon Musk—and if they could, they would gain at most 16 of his hours every day—but if an AI obtains his skills, it will be able to simulate thousand Elon Musks.