One can imagine a scenario where there is little social impetus for safety (p117-8): alarmists will have been wrong for a long time, smarter AI will have been safer for a long time, large industries will be invested, an exciting new technique will be hard to set aside, useless safety rituals will be available, and the AI will look cooperative enough in its sandbox.
Where is my “long time”? Little more than 1 week is the long time? Because that is how much time the public got to test how aligned the smartest model is before said model got a substantial amount of compute and pretty much every tool it would ever need (ability to send arbitrary requests on the internet and with that to communicate between its separate instances)
Where is my “long time”? Little more than 1 week is the long time? Because that is how much time the public got to test how aligned the smartest model is before said model got a substantial amount of compute and pretty much every tool it would ever need (ability to send arbitrary requests on the internet and with that to communicate between its separate instances)