I agree with sunwillrise but I think there is an even stronger argument for why it would be good for an AI company to drop out of the race. It is a strong jolt that has a good chance of waking up the world to AI risk. It sends a clear message:
We were paper billionaires and we were on track to be actual billionaires, but we gave that up because we were too concerned that the thing we were building could kill literally everyone. Other companies are still building it. They should stop too.
I don’t know exactly what effect that would have on public discourse, but the effect would be large.
A board firing a CEO is a pretty normal thing to happen, and it was very unclear that the firing had anything to do with safety concerns because the board communicated so little.
A big company voluntarily shutting down because its product is too dangerous is (1) a much clearer message and (2) completely unprecedented, as far as I know.
In my ideal world, the company would be very explicit that they are shutting down specifically because they are worried about AGI killing everyone.
I agree with sunwillrise but I think there is an even stronger argument for why it would be good for an AI company to drop out of the race. It is a strong jolt that has a good chance of waking up the world to AI risk. It sends a clear message:
I don’t know exactly what effect that would have on public discourse, but the effect would be large.
Larger than the OpenAI board fiasco? I doubt it.
A board firing a CEO is a pretty normal thing to happen, and it was very unclear that the firing had anything to do with safety concerns because the board communicated so little.
A big company voluntarily shutting down because its product is too dangerous is (1) a much clearer message and (2) completely unprecedented, as far as I know.
In my ideal world, the company would be very explicit that they are shutting down specifically because they are worried about AGI killing everyone.