Buried somewhere in most contemporary bayesians’ I is the solomonoff prior (the prior that the most likely observations are those that have short generating machine encodings) Do we have standard symbol for the solomonoff prior? Claude suggests that M is the most common, but is more often used as a distribution function, or perhaps K for Komogorov? (which I like because it can also be thought to stand for “knowledgebase”, although really it doesn’t represent knowledge, it pretty much represents something prior to knowledge)
My current “background I” (maybe not the one from 2017, but one I would tend to deploy here in 2024) includes something like: “Kolmogorov complexity is a cool ideal, but it is formally uncomputable in theory unless you have a halting oracle laying around in your cardboard box in your garage labeled Time Travel Stuff, and Solomonoff Induction is not tractably approximably sampled by extant techniques that aren’t just highly skilled MCMC”.
Buried somewhere in most contemporary bayesians’ I is the solomonoff prior (the prior that the most likely observations are those that have short generating machine encodings) Do we have standard symbol for the solomonoff prior? Claude suggests that M is the most common, but is more often used as a distribution function, or perhaps K for Komogorov? (which I like because it can also be thought to stand for “knowledgebase”, although really it doesn’t represent knowledge, it pretty much represents something prior to knowledge)
My current “background I” (maybe not the one from 2017, but one I would tend to deploy here in 2024) includes something like: “Kolmogorov complexity is a cool ideal, but it is formally uncomputable in theory unless you have a halting oracle laying around in your cardboard box in your garage labeled Time Travel Stuff, and Solomonoff Induction is not tractably approximably sampled by extant techniques that aren’t just highly skilled MCMC”.
Yeah. But if you give up on reasoning about/approximating solomonoff, then where do you get your priors? Do you have a better approach?