Mostly talking to you, talking to Abram, and reading Tom Sterkenburg’s thesis.
Briefly: I am now less confident that realizability assumptions are ever satisfied for embedded agents in our universe (Vanessa Kosoy / Diffractor argue this fairly convincingly). In fact this is probably similar to a standard observation about the scientific method (I read Alchin’s “theory of knowledge”, Hutter recommends avoiding editions 3rd and after). As an example intuition, with runtime restrictions it seems to be impossible to construct universal mixtures (Vladimir Vovk impressed this on me). In the unrealizable case, I now appreciate Bayesian learning as one specific expert advice aggregator (albeit an abnormally principled one equipped with now-standard analysis). I appreciate the advantages of other approaches with partial experts, with Garrabrant induction as an extreme case.
I still endorse the Bayesian approach in many cases, in particular when it is at least possible to formulate a reasonable hypothesis class that contains the truth.
Mostly talking to you, talking to Abram, and reading Tom Sterkenburg’s thesis.
Briefly: I am now less confident that realizability assumptions are ever satisfied for embedded agents in our universe (Vanessa Kosoy / Diffractor argue this fairly convincingly). In fact this is probably similar to a standard observation about the scientific method (I read Alchin’s “theory of knowledge”, Hutter recommends avoiding editions 3rd and after). As an example intuition, with runtime restrictions it seems to be impossible to construct universal mixtures (Vladimir Vovk impressed this on me). In the unrealizable case, I now appreciate Bayesian learning as one specific expert advice aggregator (albeit an abnormally principled one equipped with now-standard analysis). I appreciate the advantages of other approaches with partial experts, with Garrabrant induction as an extreme case.
I still endorse the Bayesian approach in many cases, in particular when it is at least possible to formulate a reasonable hypothesis class that contains the truth.