[SEQ RERUN] Probability is Subjectively Objective
Today’s post, Probability is Subjectively Objective was originally published on 14 July 2008. A summary (taken from the LW wiki):
Is probability subjective or objective? Better question: what does “subjective” and “objective” mean?
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we’ll be going through Eliezer Yudkowsky’s old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Rebelling Within Nature, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day’s sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
My favourite post!
Can someone explain what “subjectively objective” is supposed to mean? I think I understand the difference between subjective probability (the map) and objective probability (the territory). Is the argument that there is not such thing as probability in the territory? I’m not sure I agree—unless one believes in determinism, there are contingent events and the language of probability is the most useful for articulating the relative likelihood, right?
I understand that the objective and subjective probabilities cannot be proved to be the same, for problem of induction reasons, but what is gained by labeling probability as “subjectively objective”?
Keep in mind that even if the universe is in some sense not deterministic, this is almost never the kind of uncertainty that is relevant.
I think I mostly agree with TheOtherDave, probability is a property of propositions relative to bodies of evidence. It’s subjective because people have different bodies of evidence and objective because the probabilities follow logically from the evidence.
Caveat: the following is just my thoughts, and does not necessarily reflect LW consensus, let alone authorial intent, in any way. Indeed, it’s quite likely that both the community and the author actively disagree with everything I say here.
With respect to probability being in the map rather than the territory, this is a big part of how probability is used on LW, and remembering that this is what Eliezer has in mind is important in making sense of what he’s on about in later posts in the sequence. This is also one thing the whole “Bayesianism vs. Frequentism” thing is all about.
Probability, as discussed in the LW context, is a property of propositions relative to bodies of evidence: if we both know I’m blindly pulling a ball out of a jar full of red and black balls, and I have evidence that suggests it’s a 70⁄30 R/B split and you have evidence that suggests it’s a 60⁄40 R/B split, then we have different probabilities of drawing a red ball—which does not in any way affect the frequency with which we will draw a red ball, supposing we pulled balls out repeatedly.
That is, the label “probability” is being remapped to refer to subjective probability, and the whole “subjectively objective” business is just one of a number of rhetorical techniques being used to reinforce that mapping.
As for what is gained… I suspect the ultimate goal is to lay the groundwork for later discussions of meta-ethics, in which “right” is similarly remapped to refer to subjective rightness.
“Subjectively objective” means this:
When you calculate a probability, the value that you get depends on your prior, and hence on the kind of mind that you have. A different kind of mind with a different prior would arrive at a different value.
But suppose that you made a probabilistic inference about some system that doesn’t include your mind, such as a lottery machine. Then the calculation that you performed did not refer to your mind. Which calculation you performed was determined by the mind that you had, but the calculation so determined did not include your mind as one of its inputs.
In other words, the calculation itself did not take into consideration which mind you had. The only facts taken into consideration were objective facts about how lottery machines work.
As a result, when your mind calculated the probability, this value was not tagged as “depending on which mind you have”. On the contrary, you conclude that someone with a different kind of mind that assigns a different probability-value (given the same information) is objectively wrong. You consider his wrongness to be objective in the sense that, were he to bet according to his probability-value, he would lose money on average, regardless of what kind of mind you or anyone else had.
Nonetheless, the fact that you consider him to be objectively wrong — that fact depends on your mind. You would have reached a different conclusion had you had a different mind. His own mind, for example, concludes that he is objectively right. Thus, the fact that you consider him to be objectively wrong is a subjective fact. This is the sense in which probability is subjectively objective.
I don’t understand your intended meaning for “kind of mind.”
Assume that I believe proposition P with probability x. Then I acquire new evidence E. Like a good rationalist, I update my belief to y.
Then I meet Bob. Bob claims: 1) His degree of belief in P is z. 2) He also just received new evidence E, and no other evidence 3) His prior probability was x. 4) He is a rationalist like me.
Assume I cannot come to agreement with Bob about the correct probability of P.
By Aumann’s agreement theory, something is wrong. One of Bob’s statements might be a lie or I might not be a rationalist.
Or E might not be objective evidence—or universal (that’s technically different from objective). In the category of not particularly interesting problems: our senses are not sufficiently reliable.
I’ve always taken the “subjectively objective” as addressing this problem somehow. Am I missing the point?
For the purposes of my comment, two minds are “of the same kind” if and only if they began with the same prior. But all the minds under consideration perform standard Bayesian updating on their prior when they encounter evidence.
If two Bayesian agents started with the same prior probability x for P, and both updated on the same evidence E, then you don’t need Aumann’s agreement theorem to know that they assign the same posterior probability y. That’s just an immediate consequence of both agents using Bayesian updating on the same prior and evidence.
Under the “common-knowledge” conditions of Aumann’s theorem, the agents agree even when they didn’t update on exactly the same evidence E. That is what makes Aumann’s theorem remarkable. Even though their evidence doesn’t exactly coincide, they still agree.
But Aumann’s theorem still requires that the agents started with the same prior. The “different kinds of minds” that I was talking about started with different priors, so Aumann’s theorem doesn’t apply to them. They might disagree even if they have seen exactly the same evidence.
My interpretation of “subjectively objective” is that probability is essentially a relation between a map and the territory.
The coin is not really “50% heads, 50% tails”. In any specific situation, it’s either head, or tails. Saying “50% heads, 50% tails” means that your map contains no information about the coin (besides the fact that it has a head and tails). Thus it is a fact about the coin and about your ignorance of the coin.
Since the map is a metaphor for the relationship between my mind and objective reality, I’m not sure what the “relationship between the map and objective reality” is actually supposed to reference.
My empirical beliefs are subjective because they are beliefs in my mind. They are supposed to be objective in that the beliefs are supposed to respond in certain kinds of ways to new sensory input (this is Aumann’s Agreement Theorem).
“Subjectively” is an adverb that attempts to modify “objective.” If all that it means is that empirical beliefs are subjective and objective at the same time, I think the grammar of the phrasing makes the point misleading.
Further, I’m unclear about the intended point of noticing that empirical beliefs are subjective AND objective. The map is not the territory, but Eliezer seems to think this post is saying more than that—but I don’t see what more there is to say.