At first I thought this was making a Rice’s-theorem objection to alignment, to which the standard response is, we don’t need to be able to decide whether a program is aligned for all possible programs, just for the ones we build. But Claude suggests that the issue here is, how do you prove that a single specific AI is aligned across all possible states of the world. Is that correct?
At first I thought this was making a Rice’s-theorem objection to alignment, to which the standard response is, we don’t need to be able to decide whether a program is aligned for all possible programs, just for the ones we build. But Claude suggests that the issue here is, how do you prove that a single specific AI is aligned across all possible states of the world. Is that correct?