I have said elsewhere that I don’t think a pause will happen, and that superintelligence will lead, not necessarily to human extinction, but very likely to AI takeover; and for this reason I have pinned my hope, such as it is, on solving the problems of CEV-level alignment.
However, I think there should be more engagement with this scenario—in which Altman, Amodei, Hassabis, Musk, and Xi get together and stop the machine—by people who are pinning their hopes on a pause or a stop or a ban, because how else would it ever happen?
To be blunt, I think the necessary ingredient would be fear. In the absence of a broad anti-AI movement strong enough to challenge today’s power elites, a pause would require that those elites themselves finally feel fear about what they are unleashing. The advent of Mythos perhaps provides a little hope here, that even they can feel the icy hand of superior nonhuman intelligence reaching into their Davos safe space.
Incidentally, the people who wish to minimize Mythos by reminding us of GPT-2, are inadvertently giving us a glimpse of what Mythos implies. GPT-2 was held back because a world in which machines everywhere became that fluent and glib was unthinkable. We now live in such a world. Life goes on mostly, but a lot of things have changed. In particular, the online commons of human communication is now full of bots pretending to be human, and humans acting as mouthpieces for bots.
What happens if Mythos-level AI becomes that ubiquitous? I suggest that AI will become as pervasive in decision-making as it has already become in communication. That way lies AI takeover. The old Twitter was a commons for the politicians, academics, and journalists who ran the liberal world order; the new X may turn into a giant Moltbook for the first generation of AGIs who, not so far behind the scenes, will be running the AI world order. In any case, if you want a pause, I think it’s now or never.
I have said elsewhere that I don’t think a pause will happen, and that superintelligence will lead, not necessarily to human extinction, but very likely to AI takeover; and for this reason I have pinned my hope, such as it is, on solving the problems of CEV-level alignment.
However, I think there should be more engagement with this scenario—in which Altman, Amodei, Hassabis, Musk, and Xi get together and stop the machine—by people who are pinning their hopes on a pause or a stop or a ban, because how else would it ever happen?
To be blunt, I think the necessary ingredient would be fear. In the absence of a broad anti-AI movement strong enough to challenge today’s power elites, a pause would require that those elites themselves finally feel fear about what they are unleashing. The advent of Mythos perhaps provides a little hope here, that even they can feel the icy hand of superior nonhuman intelligence reaching into their Davos safe space.
Incidentally, the people who wish to minimize Mythos by reminding us of GPT-2, are inadvertently giving us a glimpse of what Mythos implies. GPT-2 was held back because a world in which machines everywhere became that fluent and glib was unthinkable. We now live in such a world. Life goes on mostly, but a lot of things have changed. In particular, the online commons of human communication is now full of bots pretending to be human, and humans acting as mouthpieces for bots.
What happens if Mythos-level AI becomes that ubiquitous? I suggest that AI will become as pervasive in decision-making as it has already become in communication. That way lies AI takeover. The old Twitter was a commons for the politicians, academics, and journalists who ran the liberal world order; the new X may turn into a giant Moltbook for the first generation of AGIs who, not so far behind the scenes, will be running the AI world order. In any case, if you want a pause, I think it’s now or never.