If it’s possible to self-improve in our universe, it’s possible to self-improve in the simulated universe. The only thing stopping us from putting together a reasonable simulation of the laws of physics, at this point, is raw computing power. Developing AGI is a problem of an entirely different sort: we simply don’t know how to do it yet, even in principle.
You’re right, but let me revise that slightly. In a simulated universe, some forms of self-improvement are possible, but others are cut off. Specifically, all forms of self-improvement which require more resources than you provide in the simulated universe are cut off. The problem is that that includes most of the interesting ones, and it’s entirely possible that it will self-modify into something bad but only when you give it more hardware.
If it’s possible to self-improve in our universe, it’s possible to self-improve in the simulated universe. The only thing stopping us from putting together a reasonable simulation of the laws of physics, at this point, is raw computing power. Developing AGI is a problem of an entirely different sort: we simply don’t know how to do it yet, even in principle.
You’re right, but let me revise that slightly. In a simulated universe, some forms of self-improvement are possible, but others are cut off. Specifically, all forms of self-improvement which require more resources than you provide in the simulated universe are cut off. The problem is that that includes most of the interesting ones, and it’s entirely possible that it will self-modify into something bad but only when you give it more hardware.