I’m glad we agree “they’d be one of the biggest wins in AI safety to date.”
“Implement shutdown ability” would not in fact be operationalized in a way which would ensure the ability to shutdown an actually-dangerous AI, because nobody knows how to do that
How so? It’s pretty straightforward if the model is still contained in the lab.
“Implement reasonable safeguards to prevent societal-scale catastrophes” would in fact be operationalized as checking a few boxes on a form and maybe writing some docs, without changing deployment practices at all
I think ticking boxes is good. This is how we went to the Moon, and it’s much better to do this than to not do it. It’s not trivial to tick all the boxes. Look at the number of boxes you need to tick if you want to follow the Code of Practice of the AI Act or this paper from DeepMind.
we simply do not have a way to reliably tell which models are and are not dangerous
How so? I think capabilities evaluations are much simpler than alignment evals, and at the very least we can run those. You might say: “A model might sandbag.” Sure, but you can fine-tune it and see if the capabilities are recovered. If even with some fine-tuning the model is not able to do the tasks at all, modulo the problem of gradient hacking that is, I think, very unlikely, we can be pretty sure that the model wouldn’t be capable of doing such feat. I think at the very least, following the same methodology as the one followed by Anthropic in their last system cards is pretty good and would be very helpful.
I’m glad we agree “they’d be one of the biggest wins in AI safety to date.”
How so? It’s pretty straightforward if the model is still contained in the lab.
I think ticking boxes is good. This is how we went to the Moon, and it’s much better to do this than to not do it. It’s not trivial to tick all the boxes. Look at the number of boxes you need to tick if you want to follow the Code of Practice of the AI Act or this paper from DeepMind.
How so? I think capabilities evaluations are much simpler than alignment evals, and at the very least we can run those. You might say: “A model might sandbag.” Sure, but you can fine-tune it and see if the capabilities are recovered. If even with some fine-tuning the model is not able to do the tasks at all, modulo the problem of gradient hacking that is, I think, very unlikely, we can be pretty sure that the model wouldn’t be capable of doing such feat. I think at the very least, following the same methodology as the one followed by Anthropic in their last system cards is pretty good and would be very helpful.