AI risk is not common knowledge. There are many people who do not believe there’s any risk. I really wish people who make arguments of the following form:
From the inside view, this seems reasonable. From the outside view, however, you basically end up with this:
<meme of NPCs all thinking, EVEN THOUGH I’M IN AN ORGANIZATION THAT’S ADVANCING CAPABILITIES FASTER THAN SAFETY AT A 30:1 RATIO, I’M THE GOOD GUY ON THE INSIDE, IF IT WEREN’T FOR ME IT WOULD BE 40:1
would acknowledge this fact. It is simply not true that everyone in frontier labs thinks this way. You can ask them! They’ll tell you!
It would be nice if we were just in a prisoner’s dilemma type co-ordination problem. But when someone is publicly saying “hitting DEFECT has no downsides whatsoever, I plan on doing that as much as possible” you need to take this into consideration.
AI risk is not common knowledge. There are many people who do not believe there’s any risk. I really wish people who make arguments of the following form:
would acknowledge this fact. It is simply not true that everyone in frontier labs thinks this way. You can ask them! They’ll tell you!
It would be nice if we were just in a prisoner’s dilemma type co-ordination problem. But when someone is publicly saying “hitting DEFECT has no downsides whatsoever, I plan on doing that as much as possible” you need to take this into consideration.