Wei Dai’s argument was specifically against the Solomonoff prior, which assigns probability 0 to the existence of halting problem oracles. If you have an idea how to formulate another universal prior that would give such “impossible” things positive probability, but still sum to 1.0 over all hypotheses, then by all means let’s hear it.
Yeah well it is certainly a good argument against that. The title of the thread is “is induction unformalizable” which point I’m unconvinced of.
If I were to formalize some kind of prior, I would probably use a lot of epsilons (since zero is not a probability); including an epsilon for “things I haven’t thought up yet.” On the other hand I’m not really an expert on any of these things so I imagine Wei Dai would be able to poke holes in anything I came up with anyway.
There’s no general way to have a “none of the above” hypothesis as part of your prior, because it doesn’t make any specific prediction and thus you can’t update its likelihood as data comes in. See the discussion with Cyan and others about NOTA somewhere around here.
Well then I guess I would hypothesize that solving the problem of a universal prior is equivalent to solving the problem of NOTA. I don’t really know enough to get technical here. If your point is that it’s not a good idea to model humans as Bayesians, I agree. If your point is that it’s impossible, I’m unconvinced. Maybe after I finish reading Jaynes I’ll have a better idea of the formalisms involved.
Wei Dai’s argument was specifically against the Solomonoff prior, which assigns probability 0 to the existence of halting problem oracles. If you have an idea how to formulate another universal prior that would give such “impossible” things positive probability, but still sum to 1.0 over all hypotheses, then by all means let’s hear it.
Yeah well it is certainly a good argument against that. The title of the thread is “is induction unformalizable” which point I’m unconvinced of.
If I were to formalize some kind of prior, I would probably use a lot of epsilons (since zero is not a probability); including an epsilon for “things I haven’t thought up yet.” On the other hand I’m not really an expert on any of these things so I imagine Wei Dai would be able to poke holes in anything I came up with anyway.
There’s no general way to have a “none of the above” hypothesis as part of your prior, because it doesn’t make any specific prediction and thus you can’t update its likelihood as data comes in. See the discussion with Cyan and others about NOTA somewhere around here.
Well then I guess I would hypothesize that solving the problem of a universal prior is equivalent to solving the problem of NOTA. I don’t really know enough to get technical here. If your point is that it’s not a good idea to model humans as Bayesians, I agree. If your point is that it’s impossible, I’m unconvinced. Maybe after I finish reading Jaynes I’ll have a better idea of the formalisms involved.