That’s a failure to communicate, not a failure to update.
Okay, fair enough. I’ll give it a shot, and then I’m bowing out.
Let me explain the problem with
That’s why these algorithms exist—they spare you from having to choose a prior, if the data is strong enough that the choice makes no difference.
This is not why these algorithms exist. EM isn’t really an algorithm per se; it’s a recipe for building an optimization algorithm for an objective function with the form given in equation 1.1 of the seminal paper on the topic. Likewise, Gibbs sampling is a recipe for constructing a certain type of Markov chain Monte Carlo algorithm for a given target distribution.
If you read the source material I’ve linked, you’ll notice that the EM paper gives many examples in which nothing like what you call a prior (actually a proportion) is present, e.g., sections 4.1.3, 4.6. Something like what you call priors are present in the example of section 4.3, although those models don’t really match the problem you solved. (To see why I brought up empirical Bayes in the context of your problem, read section 4.5.)
You’ll also notice that the Wikipedia article on MCMC does not mention priors in either your sense or my sense at all. That is because such notions only arise in specific applications; a true grokking of MCMC in general and Gibbs sampling in particular does not require the notion of a prior in either sense.
You’ve understood how to use the Gibbs sampling technology to solve a problem; that does not mean you understand the key ideas underlying the technology. Your problem was in the space of problems addressed by the technology, but that space is much larger, and the key ideas much more general, than you have as yet appreciated.
Okay, fair enough. I’ll give it a shot, and then I’m bowing out.
Let me explain the problem with
This is not why these algorithms exist. EM isn’t really an algorithm per se; it’s a recipe for building an optimization algorithm for an objective function with the form given in equation 1.1 of the seminal paper on the topic. Likewise, Gibbs sampling is a recipe for constructing a certain type of Markov chain Monte Carlo algorithm for a given target distribution.
If you read the source material I’ve linked, you’ll notice that the EM paper gives many examples in which nothing like what you call a prior (actually a proportion) is present, e.g., sections 4.1.3, 4.6. Something like what you call priors are present in the example of section 4.3, although those models don’t really match the problem you solved. (To see why I brought up empirical Bayes in the context of your problem, read section 4.5.)
You’ll also notice that the Wikipedia article on MCMC does not mention priors in either your sense or my sense at all. That is because such notions only arise in specific applications; a true grokking of MCMC in general and Gibbs sampling in particular does not require the notion of a prior in either sense.
You’ve understood how to use the Gibbs sampling technology to solve a problem; that does not mean you understand the key ideas underlying the technology. Your problem was in the space of problems addressed by the technology, but that space is much larger, and the key ideas much more general, than you have as yet appreciated.