Again, there seems to be an assumption in your argument which I don’t understand. Namely, that a society/superintelligence which is intelligent enough to create a convincing simulation for an AGI would necessarily possess the tools (or be intelligent enough) to assess its alignment without running it. Superintelligence does not imply omniscience.
Maybe showing the alignment of an AI without running it is vastly more difficult than creating a good simulation. This feels unlikely, but I genuinely do not see any reason why this can’t be the case. If we create a simulation which is “correct” up to the nth digit of pi, beyond which the simpler explanation for the observed behavior becomes the simulation theory rather than a complex physics theory, then no matter how intelligent you are, you’d need to calculate n digits of pi to figure this out. And if n is huge, this will take a while.
Are you curious about this position mostly for its own sake or mostly because it might shed light on the question of how much hope there is for us in an SI’s being uncertain about whether it is in a simulation?
The latter, but I believe there are simply too many maybes for your or OP’s arguments to be made.
Again, there seems to be an assumption in your argument which I don’t understand. Namely, that a society/superintelligence which is intelligent enough to create a convincing simulation for an AGI would necessarily possess the tools (or be intelligent enough) to assess its alignment without running it. Superintelligence does not imply omniscience.
Maybe showing the alignment of an AI without running it is vastly more difficult than creating a good simulation. This feels unlikely, but I genuinely do not see any reason why this can’t be the case. If we create a simulation which is “correct” up to the nth digit of pi, beyond which the simpler explanation for the observed behavior becomes the simulation theory rather than a complex physics theory, then no matter how intelligent you are, you’d need to calculate n digits of pi to figure this out. And if n is huge, this will take a while.
The latter, but I believe there are simply too many maybes for your or OP’s arguments to be made.