I mostly agree with one exception: I think robotics should be much higher up on the list.
The obvious reason why is because takeover is just easier if robotics is in the picture. There’s a reason why so many x-risk scenarios involve “nanobot swarms,” and it’s because AIs need to interact with the real world somehow. Absent robots, takeover scenarios usually involve the AI using humans to interact with the physical world via superpersuasion.
But even setting takeover aside, I expect sufficiently advanced robotics capabilities to automatically result in extreme power concentration due to human labor displacement and the subsequent loss of labor power—which has historically been extremely important for maintaining democracy. If a ruling class no longer relies on human labor for any physical tasks, there’s nothing preventing them from creating a walled garden or imposing martial law on the world directly. The latter risk is particularly concerning if the ruling class has robots with military capability.
And I expect military robots to be developed long before general-purpose robots, since armies are heavily incentivized to develop military robots (and also because it’s much easier to make non-human drones that kill people than it is to design human-like androids that can actually perform intricate manual labor tasks, which is why we’re already seeing drone usage in warfare).
Thanks, a lot of this makes sense to me. I mentioned similar things elsewhere. Maybe one reason it’s lower for me is that I think it’s a bit less likely to happen soon than some of the others.
Concretely, where would you put robotics on the list above? Right after the current #4 (human persuasion)? Or right before?
I’d say right before. It seems to me that mass LLM adoption hasn’t significantly degraded the epistemic commons and in some ways is set to improve it; I suspect it’s partially due to:
The old Aristotle reason (that it’s easier to convince people of true things than it is to convince them of false things: see also Guided By The Beauty Of Our Weapons)
The fact that removing true facts from their pre-training datasets would make them worse
The fact that it’s hard to post-train them into believing specific false things without causing other kinds of emergent misalignment (e.g. MechaHitler Grok).
I am worried about large-scale persuasion (for more on this, see @dynomight’s post on the topic), but I’d have to see a lot more LLM adoption / far greater persuasion capabilities before I begin to get worried, and I think there are some reasonable ways to avoid it. Also, the longer we wait, the more we’ll develop anti-LLM persuasion cultural antibodies, though this may not be reassuring to those with short timelines.
Conversely, the robotics seems like it will be immediately and obviously disruptive as soon as it happens. And the robotics concern is roughly symmetric across timelines, because people with short timelines think ASI will invent super robots immediately and people with longer timelines expert the world to keep spinning long enough for better robots to be developed.
(Unrelated to its ranking, it’s also very a great starting point when talking to people, because “machine replaces man in rote physical task” has been the story of the past couple hundred years—no one denies that it’s a thing.)
I think there’s an asymmetry between your beliefs here. The AIs aren’t that persuasive today, but they might get much better in the future. Similarly, they aren’t very good at robotics today, but they might get much better at that in the future.
However, a) we’re further along the persuasion tree today than we are along the robotics tree, and b) robotics is much more gated by deployment than persuasion is. Even if we develop super-dextrous robots there’d still be a gap before large fractions of the economy are robots.
Hm. I’m tempted to come up with some reasons and defend my argument, but if I’m being honest, I think I just have a strong intuition that mass persuasion (specifically scheming/deception-based mass persuasion) is going to prove harder than robotics. People, culture, and societies just feel intensely chaotic and non-deterministic in a way that robotics isn’t. But I could be wrong!
I mostly agree with one exception: I think robotics should be much higher up on the list.
The obvious reason why is because takeover is just easier if robotics is in the picture. There’s a reason why so many x-risk scenarios involve “nanobot swarms,” and it’s because AIs need to interact with the real world somehow. Absent robots, takeover scenarios usually involve the AI using humans to interact with the physical world via superpersuasion.
But even setting takeover aside, I expect sufficiently advanced robotics capabilities to automatically result in extreme power concentration due to human labor displacement and the subsequent loss of labor power—which has historically been extremely important for maintaining democracy. If a ruling class no longer relies on human labor for any physical tasks, there’s nothing preventing them from creating a walled garden or imposing martial law on the world directly. The latter risk is particularly concerning if the ruling class has robots with military capability.
And I expect military robots to be developed long before general-purpose robots, since armies are heavily incentivized to develop military robots (and also because it’s much easier to make non-human drones that kill people than it is to design human-like androids that can actually perform intricate manual labor tasks, which is why we’re already seeing drone usage in warfare).
Thanks, a lot of this makes sense to me. I mentioned similar things elsewhere. Maybe one reason it’s lower for me is that I think it’s a bit less likely to happen soon than some of the others.
Concretely, where would you put robotics on the list above? Right after the current #4 (human persuasion)? Or right before?
I’d say right before. It seems to me that mass LLM adoption hasn’t significantly degraded the epistemic commons and in some ways is set to improve it; I suspect it’s partially due to:
The old Aristotle reason (that it’s easier to convince people of true things than it is to convince them of false things: see also Guided By The Beauty Of Our Weapons)
The fact that removing true facts from their pre-training datasets would make them worse
The fact that it’s hard to post-train them into believing specific false things without causing other kinds of emergent misalignment (e.g. MechaHitler Grok).
I am worried about large-scale persuasion (for more on this, see @dynomight’s post on the topic), but I’d have to see a lot more LLM adoption / far greater persuasion capabilities before I begin to get worried, and I think there are some reasonable ways to avoid it. Also, the longer we wait, the more we’ll develop anti-LLM persuasion cultural antibodies, though this may not be reassuring to those with short timelines.
Conversely, the robotics seems like it will be immediately and obviously disruptive as soon as it happens. And the robotics concern is roughly symmetric across timelines, because people with short timelines think ASI will invent super robots immediately and people with longer timelines expert the world to keep spinning long enough for better robots to be developed.
(Unrelated to its ranking, it’s also very a great starting point when talking to people, because “machine replaces man in rote physical task” has been the story of the past couple hundred years—no one denies that it’s a thing.)
I think there’s an asymmetry between your beliefs here. The AIs aren’t that persuasive today, but they might get much better in the future. Similarly, they aren’t very good at robotics today, but they might get much better at that in the future.
However, a) we’re further along the persuasion tree today than we are along the robotics tree, and b) robotics is much more gated by deployment than persuasion is. Even if we develop super-dextrous robots there’d still be a gap before large fractions of the economy are robots.
Hm. I’m tempted to come up with some reasons and defend my argument, but if I’m being honest, I think I just have a strong intuition that mass persuasion (specifically scheming/deception-based mass persuasion) is going to prove harder than robotics. People, culture, and societies just feel intensely chaotic and non-deterministic in a way that robotics isn’t. But I could be wrong!