The most notable problem with Pascal’s Goldpan is that when you calculate the utility of believing a particular hypothesis, you’ll find that there is a term in that equation for “is this hypothesis actually true?”
That is, suppose you are considering whether or not to believe that you can fly by leaping off a cliff and flapping your arms. What is the expected utility of holding this belief?
Well, if the belief is correct, there’s a large utility to be gained: you can fly, and you’re a scientific marvel. But if it’s false, you may experience a tremendous disutility in the form of a gruesome death.
The point is that deciding you’re just going to believe whatever is most useful doesn’t even solve the problem of deciding what to believe. You still need a way of evaluating what is true. It may be that there are situations where one can expect a higher utility for believing something falsely, but as EY has touched on before, if you know you believe falsely, then you don’t actually believe. Human psychology does seem to contain an element of Pascal’s Goldpan, but that doesn’t make it rational (at least not in the sense of “optimal”; it does seem to imply that at some point in our evolution such a system tended to win in some sense).
At present the best we can do seems to be keeping our truth-determining and our utility-maximizing systems mostly separate (though there may be room for improvement on this), and Occam’s Razor is one of our tried-and-true principles for the truth-determining part.
That is, suppose you are considering whether or not to believe that you can fly by leaping off a cliff and flapping your arms. What is the expected utility of holding this belief?
I completely grant that this scheme can have disastrous consequences for a utility function that discounts consistency with past evidence, has short time horizons, considers only direct consequences, fails to consider alternatives, or is in any other way poorly chosen. Part of the point in naming it Pascal’s Goldpan was as a reminder of how naive utility functions using it will be excessively susceptible to wagers, muggings, and so on. Although I expect that highly weighting consistency with past evidence, long time horizons, considering direct and indirect consequences, considering all alternative hypotheses, and so on would prevent the obvious failure modes, it may nevertheless be that there exists no satisfactory utility function that would be safe using the Goldpan. That would certainly be compelling reason to abandon it.
That’s a little too vaguely stated for me to interpret. Can you give an illustration? For comparison, here’s one of how I assumed it would work:
A paperclip-making AI is given a piece of black-box machinery and given specifications for two possible control schemes for it. It calculates that if scheme A is true, it can make 700 paperclips per second, and if scheme B is true, only 300 per second. As a Bayesian AI using Pascal’s Goldpan formalized as a utilitarian prior, it assigns a prior probability of 0.7 for A and 0.3 for B. Then it either acts based on a weighted sum of models (0.7A+0.3B) or runs some experiments until it reaches a satisfactory posterior probability.
Occam’s razor is the basis for believing that those experiments tell us anything whatsoever about the future. Without it, there is no way to assign the probabilities you mention.
Clearly people who don’t know about Occam’s Razor, and people who explicitly reject it, still believe in the future. Just as clearly, we can use Occam’s Razor or other principles in evaluating theories about what happened in the past. Your claim appears wholly unjustified. Was it just a vague hifalutin’ metaphysical claim, or are there some underlying points that you’re not bringing out?
People who don’t know about Newtonian mechanics still believe that rocks fall downwards, but people who reject it explicitly will have a harder time reconciling their beliefs with the continued falling of rocks. It would be a mistake to reject Newtonian mechanics, then say “people who reject Newtonian mechanics clearly still believe that rocks fall”, then to conclude that there is no problem in rejecting Newtonian mechanics. Similarly, if you reject Occam’s razor then you need to replace it with something that actually fills the explanatory gap—it’s not good enough to say “well people who reject Occam’s razor clearly still believe Occam’s razor”, and then just carry right on.
The most notable problem with Pascal’s Goldpan is that when you calculate the utility of believing a particular hypothesis, you’ll find that there is a term in that equation for “is this hypothesis actually true?”
That is, suppose you are considering whether or not to believe that you can fly by leaping off a cliff and flapping your arms. What is the expected utility of holding this belief?
Well, if the belief is correct, there’s a large utility to be gained: you can fly, and you’re a scientific marvel. But if it’s false, you may experience a tremendous disutility in the form of a gruesome death.
The point is that deciding you’re just going to believe whatever is most useful doesn’t even solve the problem of deciding what to believe. You still need a way of evaluating what is true. It may be that there are situations where one can expect a higher utility for believing something falsely, but as EY has touched on before, if you know you believe falsely, then you don’t actually believe. Human psychology does seem to contain an element of Pascal’s Goldpan, but that doesn’t make it rational (at least not in the sense of “optimal”; it does seem to imply that at some point in our evolution such a system tended to win in some sense).
At present the best we can do seems to be keeping our truth-determining and our utility-maximizing systems mostly separate (though there may be room for improvement on this), and Occam’s Razor is one of our tried-and-true principles for the truth-determining part.
I completely grant that this scheme can have disastrous consequences for a utility function that discounts consistency with past evidence, has short time horizons, considers only direct consequences, fails to consider alternatives, or is in any other way poorly chosen. Part of the point in naming it Pascal’s Goldpan was as a reminder of how naive utility functions using it will be excessively susceptible to wagers, muggings, and so on. Although I expect that highly weighting consistency with past evidence, long time horizons, considering direct and indirect consequences, considering all alternative hypotheses, and so on would prevent the obvious failure modes, it may nevertheless be that there exists no satisfactory utility function that would be safe using the Goldpan. That would certainly be compelling reason to abandon it.
The point is that to evaluate the utility of holding a belief, you need to have already decided upon a scheme to set your beliefs.
That’s a little too vaguely stated for me to interpret. Can you give an illustration? For comparison, here’s one of how I assumed it would work:
A paperclip-making AI is given a piece of black-box machinery and given specifications for two possible control schemes for it. It calculates that if scheme A is true, it can make 700 paperclips per second, and if scheme B is true, only 300 per second. As a Bayesian AI using Pascal’s Goldpan formalized as a utilitarian prior, it assigns a prior probability of 0.7 for A and 0.3 for B. Then it either acts based on a weighted sum of models (0.7A+0.3B) or runs some experiments until it reaches a satisfactory posterior probability.
That doesn’t seem intractably circular.
Occam’s razor is the basis for believing that those experiments tell us anything whatsoever about the future. Without it, there is no way to assign the probabilities you mention.
Clearly people who don’t know about Occam’s Razor, and people who explicitly reject it, still believe in the future. Just as clearly, we can use Occam’s Razor or other principles in evaluating theories about what happened in the past. Your claim appears wholly unjustified. Was it just a vague hifalutin’ metaphysical claim, or are there some underlying points that you’re not bringing out?
People who don’t know about Newtonian mechanics still believe that rocks fall downwards, but people who reject it explicitly will have a harder time reconciling their beliefs with the continued falling of rocks. It would be a mistake to reject Newtonian mechanics, then say “people who reject Newtonian mechanics clearly still believe that rocks fall”, then to conclude that there is no problem in rejecting Newtonian mechanics. Similarly, if you reject Occam’s razor then you need to replace it with something that actually fills the explanatory gap—it’s not good enough to say “well people who reject Occam’s razor clearly still believe Occam’s razor”, and then just carry right on.