Anyhow, regarding probability distributions, there’s some philosophical difficulty in my opinion about “grounding”. Specifically, what reason should I have to trust that the probability distribution is doing something sensible around my safety questions of interest? How did we construct things such that it was?
The best approach I’m aware of to building a computable (but not practical) distribution with some “grounding” results is logical induction / Garrabrant induction. They come with have a self-trust result of the form that logical inductors will, across time, converge to predicting their future selves’ probabilities agree with their current probabilities. If I understand correctly, this includes limiting toward predicting a conditional probability p for an event X if we are given that the future inductor assigns probability p.
...however, as I understand, there’s still scope for any probability distributions we try to base on logical inductors to be “ungrounded”, in that we only have a guarantee that ungrounded/adversarial perturbations must be “finite” across the limit to infinity.
Anyhow, regarding probability distributions, there’s some philosophical difficulty in my opinion about “grounding”. Specifically, what reason should I have to trust that the probability distribution is doing something sensible around my safety questions of interest? How did we construct things such that it was?
The best approach I’m aware of to building a computable (but not practical) distribution with some “grounding” results is logical induction / Garrabrant induction. They come with have a self-trust result of the form that logical inductors will, across time, converge to predicting their future selves’ probabilities agree with their current probabilities. If I understand correctly, this includes limiting toward predicting a conditional probability p for an event X if we are given that the future inductor assigns probability p.
...however, as I understand, there’s still scope for any probability distributions we try to base on logical inductors to be “ungrounded”, in that we only have a guarantee that ungrounded/adversarial perturbations must be “finite” across the limit to infinity.
Here is something more technical on the matter that I alas haven’t made the personal effort to read through: https://www.lesswrong.com/posts/5bd75cc58225bf067037556d/logical-inductor-tiling-and-why-it-s-hard