GR is almost certainly wrong, given how well it fails to fit with QM. I’m no expert, but QM seems to work better than GR does, so it’s more likely the latter will have to change—which is what you’d expect from reductionism, I suppose. GR is operating at entirely the wrong level of abstraction.
The point is if GR is wrong and the AI doesn’t output GR because it’s wrong, then your test will say that the AI isn’t that smart. And then you do something like letting it out of the box and everyone probably dies.
And if the AI is that smart it will lie anyway....
What if GR is wrong, and it does not output GR because it spots the flaw that we do not?
Well, good for it?
GR is almost certainly wrong, given how well it fails to fit with QM. I’m no expert, but QM seems to work better than GR does, so it’s more likely the latter will have to change—which is what you’d expect from reductionism, I suppose. GR is operating at entirely the wrong level of abstraction.
The point is if GR is wrong and the AI doesn’t output GR because it’s wrong, then your test will say that the AI isn’t that smart. And then you do something like letting it out of the box and everyone probably dies.
And if the AI is that smart it will lie anyway....