So assuming that the Löb’s theorem problem is actually solved, the next step is to define an ideal AI which can self modify without weakening its mathematical system? Is it clear how to do this once the Löb obstruction is dealt with, or is this another very difficult math problem?
Nope, it’s pretty much done—you can trust your future copies probabilistically, without weakening. However, that would require a P that’s both defined and computable—not a trivial task.
So assuming that the Löb’s theorem problem is actually solved, the next step is to define an ideal AI which can self modify without weakening its mathematical system? Is it clear how to do this once the Löb obstruction is dealt with, or is this another very difficult math problem?
Nope, it’s pretty much done—you can trust your future copies probabilistically, without weakening. However, that would require a P that’s both defined and computable—not a trivial task.
What does P mean here?