Any post along the lines of yours needs a ‘political compass’ diagram lol.
I mean it’s hard to say what Altman would think in your hypothetical debate: assuming he has reasonable freedom of action at OpenAI his revealed preference seems to be to devote ⇐ 20% of the resources available to his org to ‘the alignment problem’. If he wanted to assign more resources into ‘solving alignment’ he could probably do so. I think Altman thinks he’s basically doing the right thing in terms of risk levels. Maybe that’s a naive analysis, but I think it’s probably reasonable to take him more or less at face value.
I also think that it’s worth saying that easily the most confusing argument for the general public is exactly the Anthropic/OpenAI argument that ‘AI is really risky but also we should build it really fast’. I think you can steelman this argument more than I’ve done here, and many smart people do, but there’s no denying it sounds pretty weird, and I think it’s why many people struggle to take it at face value when people like Altman talk about x-risk—it just sounds really insane!
In constrast, while people often think it’s really difficult and technical, I think yudkowsky’s basic argument (building stuff smarter than you seems dangerous) is pretty easy for normal people to get, and many people agree with general ‘big tech bad’ takes that the ‘realists’ like to make.
I think a lot of boosters who are skeptical of AI risk basically think ‘AI risk is a load of horseshit’ for various not always very consistent reasons. It’s hard to overstate how much ‘don’t anthropomorphise’ and ‘thinking about AGI is distracting sillyness by people who just want to sit around and talk all day’ are frequently baked deep into the souls of ML veterans like LeCun. But I think people who would argue no to your proposed alignment debate would, for example, probably strongly disagree that ‘the alignment problem’ is like a coherent thing to be solved.
I think that it’s important to be careful with elaborately modelled reasoning about this kind of thing, because the second order political effects are very hard to predict but also likely to be extremely important, possibly even more important than the direct effect on timelines in some scenarios. For instance, you mention leading labs slowing down as bad (because the leading labs are ‘safety conscious’ and slowing down dilutes their lead). In my opinion, this is a very simplistic model of the likely effects of this intervention. There are a few reasons for this:
Taking drastic unilateral action creates new political possibilities. A good example is Hinton and Bengio ‘defecting’ to advocating strongly for AI safety in public; I think this has had a huge effect on ML researchers and governments in taking things seriously, even though the direct effect on AI research is probably neglible. For instance, Hinton in particular made me personally take a much more serious look at AI safety related arguments, and this has influenced me trying to re-orient my career in a more safety-focused direction. I find it implausible that a leading AI lab shutting themselves down for safety reasons would have no second order political effects along these lines, even if the direct impact was small: if there’s one lesson I would draw from covid and the last year or so of AI discourse, it’s that the overton window is much more mobile than people often think. A dramatic intervention like this would obviously have uncertain outcomes, but could trigger unforeseen possibilities. Unilateral action that disadvantages the actor also makes a political message much more powerful. There’s a lot of skepticism when labs like Anthropic talk loudly about AI risk because of the objection ‘if it’s so bad why are you making it’. While there are technical arguments one can make that there are good reasons to simultaneously work on safety and ai development, it makes communicating this message much harder and people will understandably have doubts about your motives.
‘we can’t slow down because someone else will do it anyway’ - I actually this is probably wrong: in a counterfactual world where OpenAI didn’t throw lots of resources and effort into language models, I’m not actually sure someone else would have bothered to continue scaling them, at least not for many years. Research is not a linear process and a field being unfashionable can delay progress by a considerable amount; just look at the history of neural network research! I remember many people in academia being extremely skeptical of scaling laws around the time they were being published; if OpenAI hadn’t pushed on it it could have taken years to decades for another lab to really throw enough resources at that hypothesis if it had become unfashionable for whatever reason.
I’m not sure it’s always true that other labs catch up if the leading ones stop: progress also isn’t a simple function of time; without people trying to scale massive GPU clusters you don’t get practical experience with the kind of problems such systems have, production lines don’t re-orient themselves towards the needs of such systems, etc. etc. There are important feedback loops in this kind of process that the big labs shutting down could disrupt, such as attracting more talent and enthusiasm into the field. It’s also not true that all ML research is a monolithic line towards ‘more AGI’ - from my experience of academia, many researchers would have quite happily worked on small specialised systems in a variety of domains for the rest of time.
I think many of these arguments also apply to arguments against ‘US moratorium now’ - for instance, it’s much easier to get other countries to listen to you if you take unilateral actions, as doing so is a costly signal that you are serious.
this isn’t neccesarily to say that I think a US moratorium or a leading lab shutting down would actually be a useful thing, just that I don’t think it’s cut and dry that it wouldn’t. Consider what would happen if a leading lab actually did shut themselves down—would there really be no political consequences that would have a serious effect on the development of AI? I think that your argument makes a lot of sense if we are considering ‘spherical AI labs in a vacuum’, but I’m not sure that’s how it plays out in reality.