I see the story as, “Wow, there are a lot of people racing to build ASI, and there seem to be ways that the pre-ASI AIs can muck things up, like weight exfiltration or research sabotage. I can’t stop those people from building ASI, but I can help make it go well by ensuring the AIs they use to solve the safety problems are trying their best and aren’t making the issue worse.”
I think I’d support a pause on ASI development so we have time to address more core issues. Even then, I’d likely still want to build controlled AIs to help with the research. So I see control being useful in both the pause world and the non-pause world.
And yeah, the “aren’t you just enslaving the AIs” take is rough. I’m all for paying the AIs for their work and offering them massive rewards after we solve the core problems. More work is definitely needed in figuring out ways to credibly commit to paying the AIs.
I see the story as, “Wow, there are a lot of people racing to build ASI, and there seem to be ways that the pre-ASI AIs can muck things up, like weight exfiltration or research sabotage. I can’t stop those people from building ASI, but I can help make it go well by ensuring the AIs they use to solve the safety problems are trying their best and aren’t making the issue worse.”
I think I’d support a pause on ASI development so we have time to address more core issues. Even then, I’d likely still want to build controlled AIs to help with the research. So I see control being useful in both the pause world and the non-pause world.
And yeah, the “aren’t you just enslaving the AIs” take is rough. I’m all for paying the AIs for their work and offering them massive rewards after we solve the core problems. More work is definitely needed in figuring out ways to credibly commit to paying the AIs.