most potentially dangerous capabilities should be highly correlated, such that measuring any of them should be okay. Thus, I think it should be fine to mostly focus on measuring the capabilities that are most salient to policymakers and most clearly demonstrate risks.
Once labs are trying to pass capability evaluations, they will spend effort trying to suppress the specific capabilities being evaluated*, so I think we’d expect them to stop being so highly correlated.
* If they try methods of more generally suppressing the kinds of capabilities that might be dangerous, I think they’re likely to test them most on the capabilities being evaluated by RSPs.
Once labs are trying to pass capability evaluations, they will spend effort trying to suppress the specific capabilities being evaluated*, so I think we’d expect them to stop being so highly correlated.
* If they try methods of more generally suppressing the kinds of capabilities that might be dangerous, I think they’re likely to test them most on the capabilities being evaluated by RSPs.