I mean the counterfactual here is probably the labs doing completely half-arsed safety work, finding no issues with the models, and releasing them. Doesn’t seem any better. I think this argument made a lot of sense pre-2022 when people concerned about AI safety seemed to make up a high proportion of the people who wanted to work on AI, so them not working for the labs might have meaningfully impaired them, but at this point Anthropic and OpenAI are probably the two most desirable places to work in the world.
Also, even if you support an immediate pause (and so think that technical alignment work is not going to do anything useful) I think evals, and some of the stuff in the system cards, have played a significant role in driving concern about AI. The METR chart in particular seems to have alarmed people more than basically anything else and had an NYT story about it.
There are things people can do with their time besides “work at a lab”, “protest outside the lab’, and “bake cookies”. I think the ai world has not seriously tried to consider anything other than mad race or shutdown, or any way to use ai besides immediate attempts to build asi. Cf also my previous thoughts on trying to overcome molochian dynamics
I mean the counterfactual here is probably the labs doing completely half-arsed safety work, finding no issues with the models, and releasing them. Doesn’t seem any better. I think this argument made a lot of sense pre-2022 when people concerned about AI safety seemed to make up a high proportion of the people who wanted to work on AI, so them not working for the labs might have meaningfully impaired them, but at this point Anthropic and OpenAI are probably the two most desirable places to work in the world.
Also, even if you support an immediate pause (and so think that technical alignment work is not going to do anything useful) I think evals, and some of the stuff in the system cards, have played a significant role in driving concern about AI. The METR chart in particular seems to have alarmed people more than basically anything else and had an NYT story about it.
There are things people can do with their time besides “work at a lab”, “protest outside the lab’, and “bake cookies”. I think the ai world has not seriously tried to consider anything other than mad race or shutdown, or any way to use ai besides immediate attempts to build asi. Cf also my previous thoughts on trying to overcome molochian dynamics