Not at all—just look at the definition of solomonoff induction: the distribution over world models/theories is updated strictly on new historical observation bits, and never on future predicted observations. If you observe mental states inside the AGI, that is naturally valid observational evidence from your pov. But that is very different from you predicting that the AGI is going to kill you, and then updating your world model based on those internal predictions—those feedback loops rapidly diverge from reality (and may be related to schizophrenia).
Not at all—just look at the definition of solomonoff induction: the distribution over world models/theories is updated strictly on new historical observation bits, and never on future predicted observations. If you observe mental states inside the AGI, that is naturally valid observational evidence from your pov. But that is very different from you predicting that the AGI is going to kill you, and then updating your world model based on those internal predictions—those feedback loops rapidly diverge from reality (and may be related to schizophrenia).
I’m talking about observable evidence, like, transhumanists claiming they will drop their biological bodies on first possibility.