I think many people overrate how politically salient AI is.
I would say that much of my intuition that a pause (yes, driven by various national governments, and public desire) is plausibly doable comes from “the trajectory” of sentiments, rather than the total amount. I agree as a fraction of total political discourse it’s quite small.
Anti-AI sentiment is all over the place, but I think its a mile wide and an inch deep
This vaguely matches my impression for the most part, in the sense that if you look at discussion on AI in general, much of it may be negative but most of that isn’t people who deeply worry + care about x-risk. But to refine the picture, it’s mile wide, inch deep, but getting full of holes: it seems like there’s an ever growing number of people, including higher ups both in AI and also in government, who seem to take x-risk seriously, largely in words but also in more meaningful ways like drafting bills and stuff like that.
I think any politicians that did serious damage to the U.S. economy and potentially started wars to pause AI would be electorally punished.
Pardon my stupid question, but what goes wrong concretely? If you ban AGI, but let people keep running existing LLMs (say), does this really cause big and legible enough economic damage that voters would actually move? I mean, I’d think there’s lots of economically damaging things that don’t get credit-assigned into much actual vote shifts.
My concern is that a weak pause drives AI development underground, differentially hurts safety, and doesn’t allow people to update in the direction of a real pause. Like a think a world where AI development is nominally illegal, but the Chinese and U.S. Governments both had well funded secret programs is much worse than Evan’s proposal and likely worse than the status quo.
I may bow out, and feel free to take the maybe last, but I’ll just note that this still doesn’t make sense to me basically at all. Like, yes, there’s kinda-plausible scenarios where a global pause surprisingly ends up worse. But it would still be surprising, right? Like, there’s probably less human cloning right now, compared to the counterfactual where it wasn’t banned, right?? Someone could go underground with it, but that’s really hard and takes work!
I agree that slightly/somewhat unprecedentedly much access (official or espionage) might have to be somehow granted + enforced for treaty implementation. Maybe this is a strong defeater, I just don’t see it. Like, I agree it seems potentially kinda hard or quite hard in some scenarios. But this “global ban is actually worse than hugely resources companies going full tilt or even slightly less full tilt because of a “slowdown”″ seems galaxy-brained and false.
I’d also point out that a treaty isn’t a thing that happens, and then whatever’s written in the treaty determines how well the treaty helps the situation. A treaty is a step in a broader process that can continue to adapt and develop to avoid the dangerous stuff happening. Cf. https://www.lesswrong.com/posts/Sdrzo7z3STzdrnwKW/what-exactly-would-an-international-ai-treaty-say-is-a-bad
Kinda? It’s not really about stages. It’s just granularity of selection, period. For example, 1-stage chromosome selection, separately on two people’s gametes, is more powerful than 2- or 3-stage iterated embryo selection with realistic numbers, probably.