You are being a bad boy. In his earlier discussion Eliezer made it clear that he did not approve of this terminology of “updating priors.” One has posterior probability distributions. The prior is what one starts with. However, Eliezer has also been a bit confusing with his occasional use of such language as a “prior learning.” I repeat, agents learn, not priors, although in his view of the post-human computerized future, maybe it will be computerized priors that do the learning.
The only way one is going to get “wrong learning” at least somewhat asymptotically is if the dimensionality is high and the support is disconnected. Eliezer is right that if one starts off with a prior that is far enough off, one might well have “wrong learning,” at least for awhile. But, unless the conditions I just listed hold, eventually the learning will move in the right direction and head towards the correct answer, or probability distribution, at least that is what Bayes’ Theorem asserts.
OTOH, the reference to “deep Bayesianism” raises another issue, that of fundamental subjectivism. There is this deep divide among Bayesians between the ones that are ultimately classical frequentists but who argue that Bayesian methods are a superior way of getting to the true objective distribution, and the deep subjectivist Bayesians. For the latter, there are no ultimately “true” probability distributions. We are always estimating something derived out of our subjective priors as updated by more recent information, wherever those priors came from.
Also, saying a prior should the known probability distribution, say of cancer victims, assumes that this probability is somehow known. The prior is always subject to how much information the assumer of a prior has when they being their process of estimation.
Eliezer may not approve of it, but almost all of the literature uses the phrase “updating a prior” to mean exactly the type of sequential learning from evidence that Eliezer discusses. I prefer to think of it as ‘updating a prior’. Bayes’ theorem tells you that data is an operator on the space of probability distributions, converting prior information into posterior information. I think it’s helpful to think of that process as ‘updating’ so that my prior actually changes to something new before the next piece of information comes my way.
Hal,
You are being a bad boy. In his earlier discussion Eliezer made it clear that he did not approve of this terminology of “updating priors.” One has posterior probability distributions. The prior is what one starts with. However, Eliezer has also been a bit confusing with his occasional use of such language as a “prior learning.” I repeat, agents learn, not priors, although in his view of the post-human computerized future, maybe it will be computerized priors that do the learning.
The only way one is going to get “wrong learning” at least somewhat asymptotically is if the dimensionality is high and the support is disconnected. Eliezer is right that if one starts off with a prior that is far enough off, one might well have “wrong learning,” at least for awhile. But, unless the conditions I just listed hold, eventually the learning will move in the right direction and head towards the correct answer, or probability distribution, at least that is what Bayes’ Theorem asserts.
OTOH, the reference to “deep Bayesianism” raises another issue, that of fundamental subjectivism. There is this deep divide among Bayesians between the ones that are ultimately classical frequentists but who argue that Bayesian methods are a superior way of getting to the true objective distribution, and the deep subjectivist Bayesians. For the latter, there are no ultimately “true” probability distributions. We are always estimating something derived out of our subjective priors as updated by more recent information, wherever those priors came from.
Also, saying a prior should the known probability distribution, say of cancer victims, assumes that this probability is somehow known. The prior is always subject to how much information the assumer of a prior has when they being their process of estimation.
Eliezer may not approve of it, but almost all of the literature uses the phrase “updating a prior” to mean exactly the type of sequential learning from evidence that Eliezer discusses. I prefer to think of it as ‘updating a prior’. Bayes’ theorem tells you that data is an operator on the space of probability distributions, converting prior information into posterior information. I think it’s helpful to think of that process as ‘updating’ so that my prior actually changes to something new before the next piece of information comes my way.