In my fake graphs I basically posited that we are in graph A, though it could be the case that we will be in B, and we need to constantly watch out for that. While our observability methods and evaluations are not perfect—monitoring in deployment can miss stuff and eval-awareness is a concern—they are not completely useless either. So there are some error bars wihch lead a gap between observed alignment and actual alignment, but it is not an unbounded gap.
I wouldn’t be confident OpenAI can claim they’d catch this with monitoring given that OpenAI doesn’t make the monitoring case in system cards (as in satisfying the preparedness framework for misalignment, where it’d have to presumably have substantial evidence). These graphs omit the hard part of the problem even in worlds where you get some signal via iterative deployment, which is that labs can and will continue to iterate against observable misalignment, so over time, your model looks aligned no matter what.
Is your claim with B that it’s some world where you detect bad things but the lab does not apply any mitigations? Realistically I’d be surprised if you don’t expect tremendous pressure in that exact situation to apply “whatever you can to get the model to be usable again” even if you’re not confident you’ve resolved the underlying misalignment.
I wouldn’t be confident OpenAI can claim they’d catch this with monitoring given that OpenAI doesn’t make the monitoring case in system cards (as in satisfying the preparedness framework for misalignment, where it’d have to presumably have substantial evidence). These graphs omit the hard part of the problem even in worlds where you get some signal via iterative deployment, which is that labs can and will continue to iterate against observable misalignment, so over time, your model looks aligned no matter what.
Is your claim with B that it’s some world where you detect bad things but the lab does not apply any mitigations? Realistically I’d be surprised if you don’t expect tremendous pressure in that exact situation to apply “whatever you can to get the model to be usable again” even if you’re not confident you’ve resolved the underlying misalignment.