Human beings are universal knowledge creators: they can create any knowledge that any other knowledge creator
can create.
In what sense do you mean this exactly,
Another way of saying it is that human beings can solve any problem that can be solved. Does that help?
and what evidence for it do you have?
Careful here—as I mentioned above, evidence never supports a theory, it just provides a ready stock of criticisms of rival theories. Let me give you an argument: If you hold that human beings are not universal knowledge creators, then you are saying that human knowledge creation processes are limited in some way, that there is some knowledge we cannot create. You are saying that humans can create a whole bunch of knowledge but whole realms of other knowledge are off limits to us. How does that work? Knowledge enables us to expand our abilities and that in turn enables us to create new knowledge and so on. Whatever this knowledge we can’t create is, it would have to be walled off from all this other expanding knowledge in a rather special way. How do you build a knowledge creation machine that only has the capability to create some knowledge? That would seem much much more difficult than creating a fully universal machine.
I’ve spoken to people like Elliot, but all they said was things like ‘humans > can function as a Turing Machine by
laboureously manipulating symbols’. > Which is nice, but not really relevant to anything in real-time.
I don’t know what point Elliot was answering here, but I guess he is saying that humans are universal Turing Machines and illustrating that. He is saying that humans are universal in the sense that they can compute anything that can be computed. That is a different notion of universality to the one under discussion here (though there is a connection between the two types of universality). Elliot agrees that humans are universal knowledge creators and has written a lot about it (see, for example, his posts on The Fabric of Reality list).
On a more general note, you should probably try to be a little clearer: ‘conjectures and refutations’ doesn’t really pick
out any particular strategy from strategy-space,
‘Conjectures and refutations’ is an evolutionary process. The general methodology (or strategy, if you prefer) is: When faced with a problem try to come up with conjectural explanations to solve the problem and then criticise them until you find one (and only one) that cannot be knocked down by any known criticism. Take that as your tentative solution. I guess what you are looking for is an explanation of how human conjecture engines work? That is an unsolved problem. We do know some things, eg: no induction is involved.
and neither does the phrase ‘explanation’ pick out anything in particular.
Explanations are valuable: they help you understand something. Are you looking for an explanation of how we generate “explanations”? Again, unsolved problem.
Additionally, ‘induction’ is sufficiently different from what people normally think of as myths that it could do with some
elaboration.
It’s not really different. It’s something that people believe is true that in fact isn’t. Hume was the first to realize that there was a “problem of induction” and philosophers have for years and years been trying to justify induction. It took Karl Popper to realize that induction isn’t actually how we create knowledge at all: induction is a myth.
Similarly, some of these issues we do take seriously; we know we’re fallible,
Yes, you are called “Less Wrong” after all! I was off-beam with that.
and it sounds like you don’t know what we mean by probability.
Actually, I am quite familiar with the Bayesian conception of probability. I just don’t think probability has a role in the realm of epistemology. Evidence does not make a theory more probable, not even from a subjective point of view. What evidence does, as I have said, is provide a stock of criticisms against rival theories. Also, evidence only goes so far: what really matters is how theories stand up to criticism as explanations. Evidence plays a role in that. I am quite happy to talk about the probability of events in the world, but events are different from explanatory theories. Apples and oranges.
Another way of saying it is that human beings can solve any problem that can be solved. Does that help?
What about the problem of building pyramids on alpha-centuri by 2012? We can’t, but aliens living there could.
More pressingly though, I don’t see why this is important. Have we been basing our arguments on an assumption that there are problems we can’t solve? Is there any evidence we can solve all problems without access to arbitrarily large amounts of computational power? Something like AIXI can solve pretty much anything, but not relevantly.
That would seem much much more difficult than creating a fully universal machine.
How about a neural network that can’t learn XOR?
When faced with a problem try to come up with conjectural explanations to solve the problem and then criticise them until you find one (and only one) that cannot be knocked down by any known criticism.
The manner in which explanations are knocked down seems under-specified, if you’re not doing Bayesian updating.
Are you looking for an explanation of how we generate “explanations”? Again, unsolved problem.
Nope, I just don’t know what in particular you mean by ‘explanation’. I know what the word means in general, but not your specific conception.
I just don’t think probability has a role in the realm of epistemology.
Well, that’s different from there being no such thing as a probability that a theory is true: your initial assertion implied that the concept wasn’t well defined, whereas now you just mean it’s irrelevant. Either way, you should probably produce some actual arguments against Jaynes’s conception of probability.
Meta: You want to reply directly to a post, not its descendants, or the other person won’t get a notification. I only saw your post via the Recent Posts list.
Also, it’s no good telling people that they can’t use evidence to support their position because it contradicts your theory when the other people haven’t been convinced of your theory.
The manner in which explanations are knocked down seems under-specified, if you’re not
doing Bayesian updating.
Criticism enables us to see flaws in explanations. What is under-specified about finding a flaw?
In your way, you need to come up with criticisms and also with probabilities associated with those criticisms. Criticisms of real world theories can be involved and complex. Isn’t it enough to expose a flaw in an explanatory theory? Must one also go to the trouble of calculating probabilities—a task that is surely fraught with difficulty for any realistic idea of criticism? You’re adding a huge amount of auxilliary theory and your evaluation is then also dependent on the truth of all this auxilliary theory.
I just don’t know what in particular you mean by ‘explanation’. I know what the word means
in general, but not your specific conception.
You don’t seem to be actually saying very much then; is LW really short on explanations, in the conventional sense? Explanation seems well evidenced by the last couple of top level posts. Similarly, do we really fail to criticise one another? A large number of the comments seem to be criticisms. If you’re essentially criticising us for not having learn rationality 101 - the sort of rationality you learn as a child of 12, arguing against god—then obviously it would be a problem if we didn’t bare in mind the stuff. But without providing evidence that we succumb to these faults, it’s hard to see what the problem is.
Your other points, however, are substantive. If humans could solve any problem, or it was impossible to design an agent which could learn some but not all things, or confirmation didn’t increase subjective plausibility, these would be important claims.
Actually, I am quite familiar with the Bayesian conception of probability.
I just don’t think probability has a role in the realm of epistemology. Evidence
does not make a theory more probable, not even from a subjective point of view.
Of course evidence makes theories more probable:
Imagine you have two large opaque bags full of beans, one 50% black beans and 50% white beans and the other full of white beans. The bags are well shaken, you are given one bag at random. You take out 20 beans—and they are all white.
That is clearly evidence that confirms the hypothesis that you have the bag full of white beans. If you had the “mixed” bag, that would only happen one time in a million.
Notice that the counterfactual event is possible (that you have the mixed bag). And even if you hold the bag full of white beans, the counterfactual event that you hold the mixed beans does occur elsewhere in the multiverse. This is what distinguishes events from theories. A false theory never obtains anywhere: it is simply false. So a theory being true or false is not at all like the situation with counterfactual events. You cannot assign anything objective to a false theory.
The actual theory you hold in your example is approximately the following: I have made a random selection from a bag and I know that I have been given one of two bags: one 50% black beans and 50% white beans and the other full of white beans and: I have been honestly informed about the setup, am not being tricked, no mistakes have been made etc. This theory predicts that if I take 20 white beans out of the bag, then the chance of that would be one in a million if I had the mixed bag. Do you understand? The real situation is that you have a theory that is making probabilistic predictions about events and, as I have said several times, I have no problem with probabilistic predictions of theories about events.
Firstly, this seems like a step forwards to me. You seem to agree that induction and confirmation are fine 90% of the time. You seem to agree that these ideas work in practice—and are useful—including in some realms of knowledge—such as knowledge relating to which bag is in front of you in the above example. This puts your anti-induction and anti-confirmation statements into a rather different light, IMO.
Confirmation theory has nothing to do with multiverses. It applies equally well for agents in single deterministic universes—such as can be modelled by cellular automata. So, reasoning that depends on the details of multiverse theories is broken from the outset. Imagine evidence for wavefunction collapse was found.
Not terribly likely—but it could happen—and you don’t want your whole theory of epistemology to break if it does!
Treating uncertainty about theories and uncertainty about events differently is a philosophical mistake. There is absolutely no reason to do it—and it gets people into all kinds of muddles.
We have a beautiful theory of subjective uncertainty that applies equally well to uncertainty about any belief—whether it refers to events, or scientific theories. You can’t really tease these categories apart anyway—since many events are contingent upon the truth of scientific theories—e.g. Higgs boson observations. Events are how physical law is known to us.
Instead of using one theory for hypotheses about events and another for hypotheses about universal laws you should—according to Occam’s razor—be treating them in the same way—and be using the same underlying general theory that covers all uncertain knowledge—namely the laws of subjective probability.
Tim—In the example we have been discussing, no confirmation of the actual theory (the one I gave in approximate outline) happens. The actual theory makes probabilistic predictions about events (it also makes non-probabilistic predictions) and tells you how to bet. Getting 20 white beans doesn’t make the actual theory any more probable—the probability was a prediction of the theory. Note also that a theory that you are being tricked might recommend that you choose the mixed bag when you get 20 white beans. Lots of theories are consistent with the evidence. What you need to look for is things to refute the possible theories. If you are concerned with confirmation, then the con man wins.
So I am not agreeing that induction and confirmation are fine any percentage of the time (how did you get that 90% figure?). When you consider the actual possible theories of the example, all that is happening is that you have explanatory theories that make predictions, some probabilistic, and that tell you how to bet. The theories are not being induced from evidence and no confirmation takes place.
You haven’t explained how we assign objective probabilities to theories that are false in all worlds.
What you need to look for is things to refute the possible theories. If you are concerned with confirmation, then the con man wins.
What you’re talking about here is a strategy for avoiding bias which Bayesians also use. It is not a fundamental feature of any particular epistemology.
You don’t seem to address the idea that multiverse theories are an irrelevance—and that in a single deterministic automaton, things work just the same way.
Indeed, scientists don’t even know which (If any) laws of physics are true everywhere, and which depend on the world you are in.
You don’t seem to address the idea that we have a nice general theory that covers all kinds of uncertainty, and that no extra theory to deal with uncertainty about scientific hypotheses is needed.
If you don’t class hypotheses about events as being “theories”, then I think you need to look at:
Also, your challenge doesn’t seem to make much sense. The things people assign probabilities to are things they are uncertain about. If you tell me a theory is wrong, it gets assigned a low probability. The interesting cases are ones where we don’t yet know the answer—like the clay theory of the origin of life, the orbital inclination theory of glacial cycles—and so on.
Distinguishing between scientific theories and events in the way that you do apparently makes little sense. Events depend on scientific theories. Scientific theories predict events. Every test of a scientific theory is an event. Observing the perihelion precession of Mercury was an event. The observation of the deflection of light by the Sun during an eclipse was an event. If you have probabilities about events which are tests of scientific theories, then you automatically wind up with probabilities about the theories that depend on their outcome.
Basically agents have probabilities about all their beliefs. That is Bayes 101. If an agent claims not to have a probability about some belief, you can usually set up a bet which reveals what they actually think about the subject. Matters of fundamental physics are not different from “what type of beans are in a bag”—in that respect.
Scientific theories predict events. Every test of a scientific theory is an event.
Observing the perihelion precession of Mercury was an event. The observation of
the deflection of light by the Sun during an eclipse was an event.
Yes, scientific theories predict events. So there is a distinction between events and theories right? If the event is observed to occur, all that happens is that rival theories that do not predict the event are refuted. The theory that predicted the event is not made truer (it already is either true or false). And there are always an infinite number of other theories that predict the same event. So observing the event doesn’t allow you to distinguish among those theories.
In the bean bag example you seem to think that the rival theories are “the bag I am holding is mixed” and “the bag I am holding is all white”. But what you actually have is a single theory that makes predictions about these two possible events. That theory says you have a one-in-a-million chance of holding the mixed bag.
Matters of fundamental physics are not different from “what type of beans are in a
bag”
No, General Relativity being true or false is not like holding a bag of white beans or holding a bag of mixed beans. The latter are events that can and do obtain: They happen. But GR is not true in some universes and false in others. It is either true or false. Everywhere. Furthermore, we accept GR not because it is judged most likely but because it is the best explanation we have.
Popperians claim that we don’t need any theory of uncertainty to explain how knowledge grows: uncertainty is irrelevant. That is an interesting claim don’t you think? And if you care about the future of humanity, it is a claim that you should take seriously and try to understand.
If you are still confused about my position, why don’t you try posting some questions on one of the following lists:
It might be useful for other Popperians to explain the position—perhaps I am being unclear in some way.
Edit: Just because people might be willing to place bets is no argument that the epistemological point I am making is wrong. What makes those people infallible authorities on epistemology? Also, if I accept a bet from someone that a universal theory is true, would I ever have to pay out?
In the bean bag example you seem to think that the rival theories are “the bag I am holding is mixed” and “the bag I am holding is all white”. But what you actually have is a single theory that makes predictions about these two possible events. That theory says you have a one-in-a-million chance of holding the mixed bag.
That’s a really powerful general argument against Bayesianism that I hadn’t considered before: any prior (edit: I should have said “prior information”) necessarily constitutes a hypothesis in which you have confidence 1.
I don’t think that statement makes sense; you seem to be mixing levels—the prior is a distribution over how the world could actually be, not over other distributions. It shouldn’t make sense to speak of your prior’s confidence in itself.
You have an explanatory theory that makes predictions about the events, but it is not the only possible explanatory theory. If someone offers to play the bean bag game with you on the street, then things might not be as they seem and your theory would be no good as an explanation of how to bet. Science is like that—what is actually going on might not be what you think, so you look for flaws and realize that one’s confidence is no guide to the truth.
If your confidence in your prior were 1, you would never be able to update it. But, it is true that if your prior distribution of probabilities over various hypotheses assigns 0 or 1 probability to a group of hypotheses, you will never be able to accrue enough evidence to change that. This is not a weakness of Bayesianism, because there is no other method of reasoning which will allow you to end up on a conclusion which you at no point considered as a possibility.
Did you read the quoted text? Inability to update is the whole point of my concern; but it in no way implies that my confidence in a particular outcome will never change.
Perhaps you’re confusing probabilities for priors. (edit: I was misusing my terms: I meant “prior probabilities” and “prior information” respectively.)
I think that the problem is that EY has introduced non-standard terminology here. Worse, he blames it on Jaynes, who makes no such mistake. I just looked it up.
There are two concepts here which must not be confused.
a priori information, aka prior information, aka background information
prior probabilities, aka priors (by everyone except EY. Jaynes dislikes this but acquiesces).
Prior information does indeed constitute a hypothesis in which you have complete confidence. I agree this is something of a weakness—a weakness which is recognized implicitly in such folklore as “Cromwell’s rule” Prior information cannot be updated.
Prior probabilities (frequently known simply as priors) can be updated. In a sense, being updated is their whole purpose in life.
You are welcome. Unfortunately, I was wrong. Or at least incomplete.
I misinterpreted what EY was saying in the posting you cited. He was not, as I mistakenly assumed, saying that prior probabilities should not be called priors. He was instead talking about a third kind of entity which should not be confused with either of the other two.
Prior distributions over hypotheses, which Eliezer wishes to call simply “priors”
But there is not a confusion with referring to both prior probabilities and prior distributions as simply priors because a prior probability is simply a special case of a prior distribution. A probability is simply a distribution over a set of two competing hypotheses—only one of which can be true.
Bayes theorem in its usual form applies only to simple prior probabilities. It tells you how to update the probability. In order to update a prior distribution, you effectively need to use Bayes’s theorem multiple times—once for each hypothesis in your set of hypotheses.
So what is that 1⁄2 number which Eliezer says is definitely not a prior? It is none of the above three things. It is something harder to describe. A statistic over a distribution. I am not even going to try to explain what that means.
Sorry for any confusion I may have created. And thx to Sniffnoy and timtyler for calling my attention to my mistake.
This can easily be “flattened” into a single, more complex, probability distribution:
25% draw white bean from mixed bag.
25% draw black bean from mixed bag.
50% draw white bean from unmixed bag.
If we wish to consider multiple draws, we can again flatten the total event into a single distribution:
1⁄8 mixed bag, black and black
1⁄8 mixed bag, black and white
1⁄8 mixed bag, white and black
1⁄8 mixed bag, white and white
1⁄2 unmixed bag, white and white
Translating the “what is that number” question into this situation, we can ask: what do we mean when we say that we are 5⁄8 sure that we will draw two white beans? I would say that it is a confidence; the “event” that has 5⁄8 probability is a partial event, a lossy description of the total event.
I’m not convinced that there’s a meaningful difference between prior distributions and prior probabilities.
There isn’t when you have only two competing hypotheses. Add a third hypothesis and
you really do have to work with distributions. Chapter 4 of Jaynes explains this wonderfully. It is a long chapter, but fully worth the effort.
But the issue is also nicely captured by your own analysis. As you show, any possible linear combination of the two hypotheses can be characterized by a single parameter, which is itself the probability that the next ball will be white. But when you have three hypotheses, you have two degrees of freedom. A single probability number no longer captures all there is to be said about what you know.
Popperians claim that we don’t need any theory of uncertainty to explain how
knowledge grows: uncertainty is irrelevant. That is an interesting claim don’t
you think? And if you care about the future of humanity, it is a claim that you
should take seriously and try to understand.
Popper’s views are out of date. I am somewhat curious about why anyone with access to the relevant information would fail to update their views—but that phenomenon is not that interesting. People fail to update all the time for a bunch of sociological reasons.
if I accept a bet from someone that a universal theory is true, would I ever have to pay out?
Check with the terms of the bet. Or...
Consider bets on when a bridge will fail. It might never fail—and if so, good for the bridge. However, if traders think it has a 50% chance of surviving to the end of the year, that tells you something. The market value of the bet gives us useful information about the expected lifespan of the bridge. It is just the same with scientific theories.
I claim that the distinction you make between events and theories is not nearly so clear-cut as you seem to think. You have already made the point that distinguishing between two or more apparent theories can readily be replaced by a parameterized theory. You restrict yourself to to the case where the parameterization is due to an “event”. I think most such cases can be tortured into such a view, particularly with your multiverse model. One of the earliest uses of probability theory was Laplace’s use in estimating orbital parameters for Jupiter and Saturn. If you take these parameters as themselves the theory, you would view it as illegitimate. If they are more akin to events, this seems fine. But your conception of events as “realizable” differently in the multiverse (i.e. all probabilities should be seen as indicial uncertainty) seems to be greatly underspecified. Given your example of GR as a theory rather than an event, why don’t you want to accept a multiverse model where GR really could hold in some universes, but not others? And of course, there’s a foundational issue that whatever multiverse model you take for events is itself a theory.
By multiverse I mean the everyday Everett/Deutsch one. I agree that the argument is a meta-theory about events and theories and that that meta-theory, like any theory, could have flaws.
Where ‘real-time’ can be taken literally to refer to time that is expected to exist in physics models of the universe.
Another way of saying it is that human beings can solve any problem that can be solved. Does that help?
Careful here—as I mentioned above, evidence never supports a theory, it just provides a ready stock of criticisms of rival theories. Let me give you an argument: If you hold that human beings are not universal knowledge creators, then you are saying that human knowledge creation processes are limited in some way, that there is some knowledge we cannot create. You are saying that humans can create a whole bunch of knowledge but whole realms of other knowledge are off limits to us. How does that work? Knowledge enables us to expand our abilities and that in turn enables us to create new knowledge and so on. Whatever this knowledge we can’t create is, it would have to be walled off from all this other expanding knowledge in a rather special way. How do you build a knowledge creation machine that only has the capability to create some knowledge? That would seem much much more difficult than creating a fully universal machine.
I don’t know what point Elliot was answering here, but I guess he is saying that humans are universal Turing Machines and illustrating that. He is saying that humans are universal in the sense that they can compute anything that can be computed. That is a different notion of universality to the one under discussion here (though there is a connection between the two types of universality). Elliot agrees that humans are universal knowledge creators and has written a lot about it (see, for example, his posts on The Fabric of Reality list).
‘Conjectures and refutations’ is an evolutionary process. The general methodology (or strategy, if you prefer) is: When faced with a problem try to come up with conjectural explanations to solve the problem and then criticise them until you find one (and only one) that cannot be knocked down by any known criticism. Take that as your tentative solution. I guess what you are looking for is an explanation of how human conjecture engines work? That is an unsolved problem. We do know some things, eg: no induction is involved.
Explanations are valuable: they help you understand something. Are you looking for an explanation of how we generate “explanations”? Again, unsolved problem.
It’s not really different. It’s something that people believe is true that in fact isn’t. Hume was the first to realize that there was a “problem of induction” and philosophers have for years and years been trying to justify induction. It took Karl Popper to realize that induction isn’t actually how we create knowledge at all: induction is a myth.
Yes, you are called “Less Wrong” after all! I was off-beam with that.
Actually, I am quite familiar with the Bayesian conception of probability. I just don’t think probability has a role in the realm of epistemology. Evidence does not make a theory more probable, not even from a subjective point of view. What evidence does, as I have said, is provide a stock of criticisms against rival theories. Also, evidence only goes so far: what really matters is how theories stand up to criticism as explanations. Evidence plays a role in that. I am quite happy to talk about the probability of events in the world, but events are different from explanatory theories. Apples and oranges.
What about the problem of building pyramids on alpha-centuri by 2012? We can’t, but aliens living there could.
More pressingly though, I don’t see why this is important. Have we been basing our arguments on an assumption that there are problems we can’t solve? Is there any evidence we can solve all problems without access to arbitrarily large amounts of computational power? Something like AIXI can solve pretty much anything, but not relevantly.
How about a neural network that can’t learn XOR?
The manner in which explanations are knocked down seems under-specified, if you’re not doing Bayesian updating.
Nope, I just don’t know what in particular you mean by ‘explanation’. I know what the word means in general, but not your specific conception.
Well, that’s different from there being no such thing as a probability that a theory is true: your initial assertion implied that the concept wasn’t well defined, whereas now you just mean it’s irrelevant. Either way, you should probably produce some actual arguments against Jaynes’s conception of probability.
Meta: You want to reply directly to a post, not its descendants, or the other person won’t get a notification. I only saw your post via the Recent Posts list.
Also, it’s no good telling people that they can’t use evidence to support their position because it contradicts your theory when the other people haven’t been convinced of your theory.
Criticism enables us to see flaws in explanations. What is under-specified about finding a flaw?
In your way, you need to come up with criticisms and also with probabilities associated with those criticisms. Criticisms of real world theories can be involved and complex. Isn’t it enough to expose a flaw in an explanatory theory? Must one also go to the trouble of calculating probabilities—a task that is surely fraught with difficulty for any realistic idea of criticism? You’re adding a huge amount of auxilliary theory and your evaluation is then also dependent on the truth of all this auxilliary theory.
My conception is the same as the general one.
You don’t seem to be actually saying very much then; is LW really short on explanations, in the conventional sense? Explanation seems well evidenced by the last couple of top level posts. Similarly, do we really fail to criticise one another? A large number of the comments seem to be criticisms. If you’re essentially criticising us for not having learn rationality 101 - the sort of rationality you learn as a child of 12, arguing against god—then obviously it would be a problem if we didn’t bare in mind the stuff. But without providing evidence that we succumb to these faults, it’s hard to see what the problem is.
Your other points, however, are substantive. If humans could solve any problem, or it was impossible to design an agent which could learn some but not all things, or confirmation didn’t increase subjective plausibility, these would be important claims.
Of course evidence makes theories more probable:
Imagine you have two large opaque bags full of beans, one 50% black beans and 50% white beans and the other full of white beans. The bags are well shaken, you are given one bag at random. You take out 20 beans—and they are all white.
That is clearly evidence that confirms the hypothesis that you have the bag full of white beans. If you had the “mixed” bag, that would only happen one time in a million.
Notice that the counterfactual event is possible (that you have the mixed bag). And even if you hold the bag full of white beans, the counterfactual event that you hold the mixed beans does occur elsewhere in the multiverse. This is what distinguishes events from theories. A false theory never obtains anywhere: it is simply false. So a theory being true or false is not at all like the situation with counterfactual events. You cannot assign anything objective to a false theory.
The actual theory you hold in your example is approximately the following: I have made a random selection from a bag and I know that I have been given one of two bags: one 50% black beans and 50% white beans and the other full of white beans and: I have been honestly informed about the setup, am not being tricked, no mistakes have been made etc. This theory predicts that if I take 20 white beans out of the bag, then the chance of that would be one in a million if I had the mixed bag. Do you understand? The real situation is that you have a theory that is making probabilistic predictions about events and, as I have said several times, I have no problem with probabilistic predictions of theories about events.
As briefly as possible:
Firstly, this seems like a step forwards to me. You seem to agree that induction and confirmation are fine 90% of the time. You seem to agree that these ideas work in practice—and are useful—including in some realms of knowledge—such as knowledge relating to which bag is in front of you in the above example. This puts your anti-induction and anti-confirmation statements into a rather different light, IMO.
Confirmation theory has nothing to do with multiverses. It applies equally well for agents in single deterministic universes—such as can be modelled by cellular automata. So, reasoning that depends on the details of multiverse theories is broken from the outset. Imagine evidence for wavefunction collapse was found. Not terribly likely—but it could happen—and you don’t want your whole theory of epistemology to break if it does!
Treating uncertainty about theories and uncertainty about events differently is a philosophical mistake. There is absolutely no reason to do it—and it gets people into all kinds of muddles.
We have a beautiful theory of subjective uncertainty that applies equally well to uncertainty about any belief—whether it refers to events, or scientific theories. You can’t really tease these categories apart anyway—since many events are contingent upon the truth of scientific theories—e.g. Higgs boson observations. Events are how physical law is known to us.
Instead of using one theory for hypotheses about events and another for hypotheses about universal laws you should—according to Occam’s razor—be treating them in the same way—and be using the same underlying general theory that covers all uncertain knowledge—namely the laws of subjective probability.
“Bayesian Confirmation Theory”
http://plato.stanford.edu/entries/epistemology-bayesian/#BayTheBayConThe
Tim—In the example we have been discussing, no confirmation of the actual theory (the one I gave in approximate outline) happens. The actual theory makes probabilistic predictions about events (it also makes non-probabilistic predictions) and tells you how to bet. Getting 20 white beans doesn’t make the actual theory any more probable—the probability was a prediction of the theory. Note also that a theory that you are being tricked might recommend that you choose the mixed bag when you get 20 white beans. Lots of theories are consistent with the evidence. What you need to look for is things to refute the possible theories. If you are concerned with confirmation, then the con man wins.
So I am not agreeing that induction and confirmation are fine any percentage of the time (how did you get that 90% figure?). When you consider the actual possible theories of the example, all that is happening is that you have explanatory theories that make predictions, some probabilistic, and that tell you how to bet. The theories are not being induced from evidence and no confirmation takes place.
You haven’t explained how we assign objective probabilities to theories that are false in all worlds.
We don’t assign objective probabilities, full stop.
What you’re talking about here is a strategy for avoiding bias which Bayesians also use. It is not a fundamental feature of any particular epistemology.
I think you are too lost for me :-(
You don’t seem to address the idea that multiverse theories are an irrelevance—and that in a single deterministic automaton, things work just the same way.
Indeed, scientists don’t even know which (If any) laws of physics are true everywhere, and which depend on the world you are in.
You don’t seem to address the idea that we have a nice general theory that covers all kinds of uncertainty, and that no extra theory to deal with uncertainty about scientific hypotheses is needed.
If you don’t class hypotheses about events as being “theories”, then I think you need to look at:
http://en.wikipedia.org/wiki/Scientific_theory
Also, your challenge doesn’t seem to make much sense. The things people assign probabilities to are things they are uncertain about. If you tell me a theory is wrong, it gets assigned a low probability. The interesting cases are ones where we don’t yet know the answer—like the clay theory of the origin of life, the orbital inclination theory of glacial cycles—and so on.
Distinguishing between scientific theories and events in the way that you do apparently makes little sense. Events depend on scientific theories. Scientific theories predict events. Every test of a scientific theory is an event. Observing the perihelion precession of Mercury was an event. The observation of the deflection of light by the Sun during an eclipse was an event. If you have probabilities about events which are tests of scientific theories, then you automatically wind up with probabilities about the theories that depend on their outcome.
Basically agents have probabilities about all their beliefs. That is Bayes 101. If an agent claims not to have a probability about some belief, you can usually set up a bet which reveals what they actually think about the subject. Matters of fundamental physics are not different from “what type of beans are in a bag”—in that respect.
Yes, scientific theories predict events. So there is a distinction between events and theories right? If the event is observed to occur, all that happens is that rival theories that do not predict the event are refuted. The theory that predicted the event is not made truer (it already is either true or false). And there are always an infinite number of other theories that predict the same event. So observing the event doesn’t allow you to distinguish among those theories.
In the bean bag example you seem to think that the rival theories are “the bag I am holding is mixed” and “the bag I am holding is all white”. But what you actually have is a single theory that makes predictions about these two possible events. That theory says you have a one-in-a-million chance of holding the mixed bag.
No, General Relativity being true or false is not like holding a bag of white beans or holding a bag of mixed beans. The latter are events that can and do obtain: They happen. But GR is not true in some universes and false in others. It is either true or false. Everywhere. Furthermore, we accept GR not because it is judged most likely but because it is the best explanation we have.
Popperians claim that we don’t need any theory of uncertainty to explain how knowledge grows: uncertainty is irrelevant. That is an interesting claim don’t you think? And if you care about the future of humanity, it is a claim that you should take seriously and try to understand.
If you are still confused about my position, why don’t you try posting some questions on one of the following lists:
http://groups.yahoo.com/group/Fabric-of-Reality/
http://groups.yahoo.com/group/criticalrationalism/
It might be useful for other Popperians to explain the position—perhaps I am being unclear in some way.
Edit: Just because people might be willing to place bets is no argument that the epistemological point I am making is wrong. What makes those people infallible authorities on epistemology? Also, if I accept a bet from someone that a universal theory is true, would I ever have to pay out?
That’s a really powerful general argument against Bayesianism that I hadn’t considered before: any prior (edit: I should have said “prior information”) necessarily constitutes a hypothesis in which you have confidence 1.
I don’t think that statement makes sense; you seem to be mixing levels—the prior is a distribution over how the world could actually be, not over other distributions. It shouldn’t make sense to speak of your prior’s confidence in itself.
You have an explanatory theory that makes predictions about the events, but it is not the only possible explanatory theory. If someone offers to play the bean bag game with you on the street, then things might not be as they seem and your theory would be no good as an explanation of how to bet. Science is like that—what is actually going on might not be what you think, so you look for flaws and realize that one’s confidence is no guide to the truth.
If your confidence in your prior were 1, you would never be able to update it. But, it is true that if your prior distribution of probabilities over various hypotheses assigns 0 or 1 probability to a group of hypotheses, you will never be able to accrue enough evidence to change that. This is not a weakness of Bayesianism, because there is no other method of reasoning which will allow you to end up on a conclusion which you at no point considered as a possibility.
Did you read the quoted text? Inability to update is the whole point of my concern; but it in no way implies that my confidence in a particular outcome will never change.
Perhaps you’re confusing probabilities for priors. (edit: I was misusing my terms: I meant “prior probabilities” and “prior information” respectively.)
I think that the problem is that EY has introduced non-standard terminology here. Worse, he blames it on Jaynes, who makes no such mistake. I just looked it up.
There are two concepts here which must not be confused.
a priori information, aka prior information, aka background information
prior probabilities, aka priors (by everyone except EY. Jaynes dislikes this but acquiesces).
Prior information does indeed constitute a hypothesis in which you have complete confidence. I agree this is something of a weakness—a weakness which is recognized implicitly in such folklore as “Cromwell’s rule” Prior information cannot be updated.
Prior probabilities (frequently known simply as priors) can be updated. In a sense, being updated is their whole purpose in life.
This is exactly what’s going on. Thank you.
I apologize for my confused terminology.
You are welcome. Unfortunately, I was wrong. Or at least incomplete.
I misinterpreted what EY was saying in the posting you cited. He was not, as I mistakenly assumed, saying that prior probabilities should not be called priors. He was instead talking about a third kind of entity which should not be confused with either of the other two.
Prior distributions over hypotheses, which Eliezer wishes to call simply “priors”
But there is not a confusion with referring to both prior probabilities and prior distributions as simply priors because a prior probability is simply a special case of a prior distribution. A probability is simply a distribution over a set of two competing hypotheses—only one of which can be true.
Bayes theorem in its usual form applies only to simple prior probabilities. It tells you how to update the probability. In order to update a prior distribution, you effectively need to use Bayes’s theorem multiple times—once for each hypothesis in your set of hypotheses.
So what is that 1⁄2 number which Eliezer says is definitely not a prior? It is none of the above three things. It is something harder to describe. A statistic over a distribution. I am not even going to try to explain what that means. Sorry for any confusion I may have created. And thx to Sniffnoy and timtyler for calling my attention to my mistake.
I’m not convinced that there’s a meaningful difference between prior distributions and prior probabilities.
Going back to the beans problem, we have this:
This can easily be “flattened” into a single, more complex, probability distribution:
If we wish to consider multiple draws, we can again flatten the total event into a single distribution:
Translating the “what is that number” question into this situation, we can ask: what do we mean when we say that we are 5⁄8 sure that we will draw two white beans? I would say that it is a confidence; the “event” that has 5⁄8 probability is a partial event, a lossy description of the total event.
There isn’t when you have only two competing hypotheses. Add a third hypothesis and you really do have to work with distributions. Chapter 4 of Jaynes explains this wonderfully. It is a long chapter, but fully worth the effort.
But the issue is also nicely captured by your own analysis. As you show, any possible linear combination of the two hypotheses can be characterized by a single parameter, which is itself the probability that the next ball will be white. But when you have three hypotheses, you have two degrees of freedom. A single probability number no longer captures all there is to be said about what you know.
In retrospect, it’s obvious that “probability” should refer to a real scalar on the interval [0,1].
Everyone calls prior probabilities “priors”—including: http://yudkowsky.net/rational/bayes
Uh, what? No it doesn’t. If your confidence in your priors was that high, they would never shift.
Popper’s views are out of date. I am somewhat curious about why anyone with access to the relevant information would fail to update their views—but that phenomenon is not that interesting. People fail to update all the time for a bunch of sociological reasons.
Check with the terms of the bet. Or...
Consider bets on when a bridge will fail. It might never fail—and if so, good for the bridge. However, if traders think it has a 50% chance of surviving to the end of the year, that tells you something. The market value of the bet gives us useful information about the expected lifespan of the bridge. It is just the same with scientific theories.
I claim that the distinction you make between events and theories is not nearly so clear-cut as you seem to think. You have already made the point that distinguishing between two or more apparent theories can readily be replaced by a parameterized theory. You restrict yourself to to the case where the parameterization is due to an “event”. I think most such cases can be tortured into such a view, particularly with your multiverse model. One of the earliest uses of probability theory was Laplace’s use in estimating orbital parameters for Jupiter and Saturn. If you take these parameters as themselves the theory, you would view it as illegitimate. If they are more akin to events, this seems fine. But your conception of events as “realizable” differently in the multiverse (i.e. all probabilities should be seen as indicial uncertainty) seems to be greatly underspecified. Given your example of GR as a theory rather than an event, why don’t you want to accept a multiverse model where GR really could hold in some universes, but not others? And of course, there’s a foundational issue that whatever multiverse model you take for events is itself a theory.
By multiverse I mean the everyday Everett/Deutsch one. I agree that the argument is a meta-theory about events and theories and that that meta-theory, like any theory, could have flaws.