Wait, so it’s enough for the agents to just believe the observables are independent given the state of their latents? We only need the Xj to be independent conditional on λiundera particular modelMi?
I didn’t realise that. I thought the observables had to be ‘actually independent’ after conditioning in some sort of frequentist sense.
Getting a version of this that works under approximate Agreement on Observables sounds like it would be very powerful then. It’d mean that even if Alice is much smarter than Bob, with her model e.g. having more FLOP which she can use to squeeze more bits of information out of the data, there’d still need to be a mapping between the concepts Bob and Alice internally use in those domains where Bob doesn’t do very much worse than Alice on predictive accuracy.
So, if a superintelligence isn’t that much better than humanity at modelling some specific part of reality, there’d need to be an approximate mapping between humanity’s latents and (some of) the superintelligence’s latents for that part of reality. If the the theorems approximately hold under approximate agreement on observables.
If the the theorems approximately hold under approximate agreement on observables.
Yeah, there is still the issue that the theorems aren’t always robust to approximation on the Agreement on Observables condition, though the Solomonoff version is and there’s probably other ways to sidestep the issue.
Wait, so it’s enough for the agents to just believe the observables are independent given the state of their latents? We only need the Xj to be independent conditional on λi under a particular model Mi?
I didn’t realise that. I thought the observables had to be ‘actually independent’ after conditioning in some sort of frequentist sense.
Getting a version of this that works under approximate Agreement on Observables sounds like it would be very powerful then. It’d mean that even if Alice is much smarter than Bob, with her model e.g. having more FLOP which she can use to squeeze more bits of information out of the data, there’d still need to be a mapping between the concepts Bob and Alice internally use in those domains where Bob doesn’t do very much worse than Alice on predictive accuracy.
So, if a superintelligence isn’t that much better than humanity at modelling some specific part of reality, there’d need to be an approximate mapping between humanity’s latents and (some of) the superintelligence’s latents for that part of reality. If the the theorems approximately hold under approximate agreement on observables.
Yup, that is correct.
Yeah, there is still the issue that the theorems aren’t always robust to approximation on the Agreement on Observables condition, though the Solomonoff version is and there’s probably other ways to sidestep the issue.