On the “safety underinvestment” point I am going to say something which I think is obvious (and has probably been discussed before) but I have not personally seen anyone advocate for:
The DoD should be conducting its own safety/alignment research, and pushing for this should be one of the primary goals of safety advocates.
There is this constant push for labs to invest more in safety. At the same time we all acknowledge this is a zero sum game. Every dollar/joule/flop they put into safety/alignment is a dollar/joule/flop they can’t put into capabilities research. I think this dynamic of insisting that labs carry the torch entirely on safety, instead of pushing for government funded safety research, is a big part of why you’ve seen labs/VCs becoming increasingly hostile to safety advocacy/regulation. Because the only pitch so far makes safety onerous for labs.
Pushing for safety/alignment research to be done by the DoD allows for investments to be made into safety in a way that doesn’t require labs reroute scarce resources or decelerate their capabilities progress. It can also be purely additive, there’s no reason labs can’t continue their own safety research as well.
I think it’s useful for government to do safety/alignment but:
1. I don’t think it’s a zero sum game—often safety/alignment improvements go hand in hand with capabilities. 2. Like we have seen with software security, if safety is not “baked in”, it is hard to add it after the fact, so it is important for safety and capabilities researchers to work together.
On the “safety underinvestment” point I am going to say something which I think is obvious (and has probably been discussed before) but I have not personally seen anyone advocate for:
The DoD should be conducting its own safety/alignment research, and pushing for this should be one of the primary goals of safety advocates.
There is this constant push for labs to invest more in safety. At the same time we all acknowledge this is a zero sum game. Every dollar/joule/flop they put into safety/alignment is a dollar/joule/flop they can’t put into capabilities research. I think this dynamic of insisting that labs carry the torch entirely on safety, instead of pushing for government funded safety research, is a big part of why you’ve seen labs/VCs becoming increasingly hostile to safety advocacy/regulation. Because the only pitch so far makes safety onerous for labs.
Pushing for safety/alignment research to be done by the DoD allows for investments to be made into safety in a way that doesn’t require labs reroute scarce resources or decelerate their capabilities progress. It can also be purely additive, there’s no reason labs can’t continue their own safety research as well.
I think it’s useful for government to do safety/alignment but:
1. I don’t think it’s a zero sum game—often safety/alignment improvements go hand in hand with capabilities.
2. Like we have seen with software security, if safety is not “baked in”, it is hard to add it after the fact, so it is important for safety and capabilities researchers to work together.