Is there some level of evidence more convincing than study published in reputable, peer-reviewed journal with lots of replication?
I would say so. For instance, the claim that letting go of my pen in mid-air will cause it to fall onto the floor and not the ceiling. The above is backed up by so much evidence and in so many different ways* that it is even if a replicated study were to contradict it, I would continue to believe it (it’s one of the few claims that really does deserve a subjective probability in the region of 99.9%).
I don’t know about Yudkowsky, but the threshold for ’what would be enough evidence to convince me of the existence of ESP” lies somewhere between those two.
I initially took ‘some claims are just too extraordinary’ to be talking about the first level, while ‘how to convince me that 2+2=3’ talks about the second level, but maybe I’m being too charitable.
For evidence: * thousands of trials from my own experience, millions from other people, universal agreement from everyone I know or have heard, coherent scientific theory explaining how and why it happens etc...
(it’s [that letting go of a pen will cause it to fall to the floor and not the ceiling] one of the few claims that really does deserve a subjective probability in the region of 99.9%).
Only three nines? This seems very underconfident. I would assign it more like twelve-nines confidence.
I was debating between that and 99.95% or even 99.99%, but 12 9s! Do you understand what that means! Do you really think that you cold make 1 trillion similar claims and only be wrong once? One trillion possible worlds, and it only goes up in one of them?
To put it differently, suppose Warren Buffet comes up to me, and suggests a game. I drop the pen, if it falls, I give him ten cents, otherwise, he gives me everything he owns (about $50 billion). By your estimate, he’s ripping me off to the tune of about 5 cents, but I think I would accept the bet, which suggests my estimate is not as high.
Y’see, this sort of thing is why I distrust the betting model for extreme probabilities like this.
I mean, I would take that bet as well, but I’m pretty sure that my estimate under normal circumstances is closer to jimrandomh’s… it’s just that the offer itself is so outrageous that the circumstances stop being normal. I think my reasoning would be more along the lines of “what the hell, I’m willing to throw away ten cents just to get to tell this story,” or possibly even “I no longer have any idea what’s going on, but losing ten cents is no big deal and I’m curious to see what happens next.”
Put another way: I very much doubt that my willingness to take that bet is the result of an EV calculation using my real prior probability of a randomly selected pen falling to the ground; therefore, I am disinclined to treat that willingness as significant evidence of that prior.
Would you take the offer if it was only $10? I wouldn’t (I get to tell the story either way), which suggests my decision really does depend upon my prior.
Another interefering factor is the fact that my utility function is nowhere near monotonic in dollars, especially for large amount. I’d take a certainty of $1 million over a 5% chance of $1 billion, which suggests that $50 billion is worth a lot less than a trillion times as much as 5 cents to me, so my prior must be some way below 99.9999999999%
Bear in mind overconfidence bias, its very easy to get trigger happy with 9s, and forget that even 99.9% is very impressive in a world with as many unknowns and interfering factors as ours.
If it were Warren Buffet? Probably. “Warren Buffet offered me $50 billion if my pen falls to the ceiling” is a much cooler story than “Warren Buffet offered me $10 if my pen falls to the ceiling,” but the latter is still easily worth ten cents.
OTOH, “Some guy on the Internet offered me $10 if my pen falls to the ceiling” is not so cool a story. I probably would turn that down.
Agreed that utility is radically nonmonotonic in dollars.
Agreed that it’s easy to get overconfident with 9s. It’s also easy to anchor on the integers.
All that said: roughly how many objects have you seen or otherwise had compelling experience of having been dropped in your lifetime, would you estimate? How many of those have fallen to the ground, and how many have floated to the ceiling? What happens to those numbers if we eliminating ones more theoretically likely to float than pens (e.g., helium balloons)?
At a guess, I’d say I’ve personally seen on the order of five thousand non-helium-balloon-like objects dropped in my life, and they’ve all fallen down. So just starting from there, I’d estimate a > 99.9998 chance that the next one will, too.
If I start additionally factoring in the additional evidentiary value of theoretical considerations, and the absence of other people’s reports of such objects floating, and of stories people tell involving objects falling to the ground… man, the evidence starts to add up.
(We’ve established what I am, now we’re haggling over the price...)
All that said: roughly how many objects have you seen or otherwise had compelling experience of having been dropped in your lifetime, would you estimate? How many of those have fallen to the ground, and how many have floated to the ceiling? What happens to those numbers if we eliminating ones more theoretically likely to float than pens (e.g., helium balloons)?
At a guess, I’d say I’ve personally seen on the order of five thousand non-helium-balloon-like objects dropped in my life, and they’ve all fallen down. So just starting from there, I’d estimate a > 99.9998 chance that the next one will, too.
This a good point, 99.9% probably is too low, although I’d be a little worried to go as high as 99.9998%. I’m not sure how you got that figure anyway, if you’ve seen five-thousand that means Laplace’s law suggests about 99.98% doesn’t it? I’ve probably seen a similar number (maybe a bit less), and I can remember one that failed to fall downward (although it was in very high winds so perhaps I should have been able to predict it).
The other evidence probably doesn’t count for a Bayes factor of as much as 100:1, since by that point a good part of the remaining probability mass is concentrated in hypotheses like “pens always fall downwards except under this one specific rare circumstance” and “I am completely insane”.
I distrust the betting model for extreme probabilities like this.
Can any of us really comprehend that much money? Frankly, I bet even Warren Buffet would have trouble; he has said after he gives away 99% of his wealth, he and his family will have all the money they will ever need. Does 100 times as much money as you’ll ever need really feel different from 10x or 1000x?
The betting heuristic is useful because our intuitive sense for money has a much larger dynamic range than our intuitive sense of probability—it’s easier to conceptualize “a subway ticket against a nice house” than “million to one odds”, but “0.05 subway tickets against 25000 nice houses” doesn’t really fit the way we think.
Do you really think that you cold make 1 trillion similar claims and only be wrong once?
No, I would run out of statements I was that confident in long before I reached a trillion.
To put it differently, suppose Warren Buffet comes up to me, and suggests a game. I drop the pen, if it falls, I give him ten cents, otherwise, he gives me everything he owns (about $50 billion). By your estimate, he’s ripping me off to the tune of about 5 cents, but I think I would accept the bet, which suggests my estimate is not as high.
There are two problems with converting probability estimates like that into bets. First, there is more than a 1/10^12 chance of cheating in that game, by putting a strong magnet in the ceiling for example. That issue does not apply in non-game contexts. And second, utility is not linear in money over that interval; Warren Buffet would value a ten cent gain less than 1/10^12 as much as avoiding a $10^11 loss.
No, I would run out of statements I was that confident in long before I reached a trillion.
Nitpicking.
First, there is more than a 1/10^12 chance of cheating in that game, by putting a strong magnet in the ceiling for example.
You know that you’re not cheating, and it doesn’t seem likely that Buffet would cheat when doing so would make him less likely to win. Of course, maybe there’s a 10^-10 chance that Buffet would go insane and cheat anyway, but can we just assume a least convenient possible world where we ignore those interfering issues.
Or come up with your own hypothetical if you don’t like mine (you could use Omega instead of Buffet to eliminate cheating).
And second, utility is not linear in money over that interval; Warren Buffet would value a ten cent gain less than 1/10^12 as much as avoiding a $10^11 loss.
I don’t care what Buffet values, the important thing is what I value, and think I actually value avoiding a ten cent loss a lot more than 10^-12 as much as achieving a $50billion gain.
No, I would run out of statements I was that confident in long before I reached a trillion.
Nitpicking.
Not at all. The feeling of impossibility of making trillion statements like that and never being wrong partly stems from our inability to conceive trillion distinct statements supported so much by evidence as the validity of gravitational laws. (The second reason for the intuition is imagining making mistake because of being tired of making one prediction after another.) Certainly there is far less than trillion independent statements of comparable trustworthiness that people can utter, which makes the general calibration approach to defining the subjective probability hard to use here.
You know that you’re not cheating, and it doesn’t seem likely that Buffet would cheat when doing so would make him less likely to win.
Someone else may put the magnet in the pen; this is the sort of concerns you cannot rule out in real life. Or perhaps Buffet is fed up with being rich and wants to award his possessions to some random person and do it in an unusual way. But I find most probable that your willingness to accept this very bet is caused by the same bias which makes people buy lottery tickets; except here you are able to rationalise the probabilities afterwards to justify your decision.
As for your estimate for pen falling down instead of up (no cheating assumed) being 99.99%: seriously? Does it mean that you expect the pen fall up once in every ten thousand trials? If 99.99% is your upper bound for probability in general (this may be a reasonable interpretation because the pen example is certainly one of the most certain predictions one can state), do you play any lottery where the jackpot is more than 10000 more worth the ticket?
Someone else may put the magnet in the pen; this is the sort of concerns you cannot rule out in real life. Or perhaps Buffet is fed up with being rich and wants to award his possessions to some random person and do it in an unusual way.
As I said, least convenient possible world.
As for your estimate for pen falling down instead of up (no cheating assumed) being 99.99%: seriously?
Firstly, as of my recent discussion with TOD my actual estimate is now more like 99.999% than 99.99%, but still way below 99.9999999999%. I do not assume no cheating in this estimate, I merely assume the amount of cheating you’d expect when idly tossing a pen in my room rather than the amount I’d expect when playing billion-dollar games with Warren Buffet.
Does it mean that you expect the pen fall up once in every ten thousand trials?
No. Most of that probability is concentrated in hypotheses where the pen never falls up. This is the difference between a Bayesian probability and a frequency.
I will point out though, that I have probably dropped fewer than 10000 pens in my life and one of them did go sideways (to the best I could tell there was no statistically significant vertical component to its movement), though I suppose I should have predicted that given the weather conditions at the time.
If 99.99% is your upper bound for probability in general (this may be a reasonable interpretation because the pen example is certainly one of the most certain predictions one can state), do you play any lottery where the jackpot is more than 10000 more worth the ticket?
Actually, this is one kind of claim I will happily assign probabilities like 99.9999999999% and higher to, which is the negation of a very specific claim (this is equivalent to the union of a huge number of other claims), for example:
“There is not currently a gang of 17643529 elephants, each painted with the national flag of Moldova, all riding the same giant unicycle around a 57km wide crater on the innermost planet of the 59th closest star to earth in the Andromeda galaxy”.
Happy to go well above 99.9999999999% on that one, as there are easily 10^12 mutually exclusive alternatives, all at least as plausible (in addition, by allowing claims like this, but slightly more plausible, I bet I could find a trillion claims each about as plausible as the pen).
What is impressive about the pen, as well as about most scientific theories, is that it is a single, very specific, hypothesis which earned its high probability through accuracy rather than bulk.
Judging by the existence of this later post, it appears Eliezer eventually changed his mind about this.
Is there some level of evidence more convincing than study published in reputable, peer-reviewed journal with lots of replication?
I would say so. For instance, the claim that letting go of my pen in mid-air will cause it to fall onto the floor and not the ceiling. The above is backed up by so much evidence and in so many different ways* that it is even if a replicated study were to contradict it, I would continue to believe it (it’s one of the few claims that really does deserve a subjective probability in the region of 99.9%).
I don’t know about Yudkowsky, but the threshold for ’what would be enough evidence to convince me of the existence of ESP” lies somewhere between those two.
I initially took ‘some claims are just too extraordinary’ to be talking about the first level, while ‘how to convince me that 2+2=3’ talks about the second level, but maybe I’m being too charitable.
For evidence: * thousands of trials from my own experience, millions from other people, universal agreement from everyone I know or have heard, coherent scientific theory explaining how and why it happens etc...
Only three nines? This seems very underconfident. I would assign it more like twelve-nines confidence.
I was debating between that and 99.95% or even 99.99%, but 12 9s! Do you understand what that means! Do you really think that you cold make 1 trillion similar claims and only be wrong once? One trillion possible worlds, and it only goes up in one of them?
To put it differently, suppose Warren Buffet comes up to me, and suggests a game. I drop the pen, if it falls, I give him ten cents, otherwise, he gives me everything he owns (about $50 billion). By your estimate, he’s ripping me off to the tune of about 5 cents, but I think I would accept the bet, which suggests my estimate is not as high.
Y’see, this sort of thing is why I distrust the betting model for extreme probabilities like this.
I mean, I would take that bet as well, but I’m pretty sure that my estimate under normal circumstances is closer to jimrandomh’s… it’s just that the offer itself is so outrageous that the circumstances stop being normal. I think my reasoning would be more along the lines of “what the hell, I’m willing to throw away ten cents just to get to tell this story,” or possibly even “I no longer have any idea what’s going on, but losing ten cents is no big deal and I’m curious to see what happens next.”
Put another way: I very much doubt that my willingness to take that bet is the result of an EV calculation using my real prior probability of a randomly selected pen falling to the ground; therefore, I am disinclined to treat that willingness as significant evidence of that prior.
Would you take the offer if it was only $10? I wouldn’t (I get to tell the story either way), which suggests my decision really does depend upon my prior.
Another interefering factor is the fact that my utility function is nowhere near monotonic in dollars, especially for large amount. I’d take a certainty of $1 million over a 5% chance of $1 billion, which suggests that $50 billion is worth a lot less than a trillion times as much as 5 cents to me, so my prior must be some way below 99.9999999999%
Bear in mind overconfidence bias, its very easy to get trigger happy with 9s, and forget that even 99.9% is very impressive in a world with as many unknowns and interfering factors as ours.
If it were Warren Buffet? Probably. “Warren Buffet offered me $50 billion if my pen falls to the ceiling” is a much cooler story than “Warren Buffet offered me $10 if my pen falls to the ceiling,” but the latter is still easily worth ten cents.
OTOH, “Some guy on the Internet offered me $10 if my pen falls to the ceiling” is not so cool a story. I probably would turn that down.
Agreed that utility is radically nonmonotonic in dollars.
Agreed that it’s easy to get overconfident with 9s. It’s also easy to anchor on the integers.
All that said: roughly how many objects have you seen or otherwise had compelling experience of having been dropped in your lifetime, would you estimate? How many of those have fallen to the ground, and how many have floated to the ceiling? What happens to those numbers if we eliminating ones more theoretically likely to float than pens (e.g., helium balloons)?
At a guess, I’d say I’ve personally seen on the order of five thousand non-helium-balloon-like objects dropped in my life, and they’ve all fallen down. So just starting from there, I’d estimate a > 99.9998 chance that the next one will, too.
If I start additionally factoring in the additional evidentiary value of theoretical considerations, and the absence of other people’s reports of such objects floating, and of stories people tell involving objects falling to the ground… man, the evidence starts to add up.
(We’ve established what I am, now we’re haggling over the price...)
This a good point, 99.9% probably is too low, although I’d be a little worried to go as high as 99.9998%. I’m not sure how you got that figure anyway, if you’ve seen five-thousand that means Laplace’s law suggests about 99.98% doesn’t it? I’ve probably seen a similar number (maybe a bit less), and I can remember one that failed to fall downward (although it was in very high winds so perhaps I should have been able to predict it).
The other evidence probably doesn’t count for a Bayes factor of as much as 100:1, since by that point a good part of the remaining probability mass is concentrated in hypotheses like “pens always fall downwards except under this one specific rare circumstance” and “I am completely insane”.
I got that figure stupidly (1-1/5000), and I am chagrined. hides under desk
Ah, the old decimal turns into a percentage error. You reduce all of us to gibbering heaps.
Can any of us really comprehend that much money? Frankly, I bet even Warren Buffet would have trouble; he has said after he gives away 99% of his wealth, he and his family will have all the money they will ever need. Does 100 times as much money as you’ll ever need really feel different from 10x or 1000x?
The betting heuristic is useful because our intuitive sense for money has a much larger dynamic range than our intuitive sense of probability—it’s easier to conceptualize “a subway ticket against a nice house” than “million to one odds”, but “0.05 subway tickets against 25000 nice houses” doesn’t really fit the way we think.
No, I would run out of statements I was that confident in long before I reached a trillion.
There are two problems with converting probability estimates like that into bets. First, there is more than a 1/10^12 chance of cheating in that game, by putting a strong magnet in the ceiling for example. That issue does not apply in non-game contexts. And second, utility is not linear in money over that interval; Warren Buffet would value a ten cent gain less than 1/10^12 as much as avoiding a $10^11 loss.
Nitpicking.
You know that you’re not cheating, and it doesn’t seem likely that Buffet would cheat when doing so would make him less likely to win. Of course, maybe there’s a 10^-10 chance that Buffet would go insane and cheat anyway, but can we just assume a least convenient possible world where we ignore those interfering issues.
Or come up with your own hypothetical if you don’t like mine (you could use Omega instead of Buffet to eliminate cheating).
I don’t care what Buffet values, the important thing is what I value, and think I actually value avoiding a ten cent loss a lot more than 10^-12 as much as achieving a $50billion gain.
Not at all. The feeling of impossibility of making trillion statements like that and never being wrong partly stems from our inability to conceive trillion distinct statements supported so much by evidence as the validity of gravitational laws. (The second reason for the intuition is imagining making mistake because of being tired of making one prediction after another.) Certainly there is far less than trillion independent statements of comparable trustworthiness that people can utter, which makes the general calibration approach to defining the subjective probability hard to use here.
Someone else may put the magnet in the pen; this is the sort of concerns you cannot rule out in real life. Or perhaps Buffet is fed up with being rich and wants to award his possessions to some random person and do it in an unusual way. But I find most probable that your willingness to accept this very bet is caused by the same bias which makes people buy lottery tickets; except here you are able to rationalise the probabilities afterwards to justify your decision.
As for your estimate for pen falling down instead of up (no cheating assumed) being 99.99%: seriously? Does it mean that you expect the pen fall up once in every ten thousand trials? If 99.99% is your upper bound for probability in general (this may be a reasonable interpretation because the pen example is certainly one of the most certain predictions one can state), do you play any lottery where the jackpot is more than 10000 more worth the ticket?
As I said, least convenient possible world.
Firstly, as of my recent discussion with TOD my actual estimate is now more like 99.999% than 99.99%, but still way below 99.9999999999%. I do not assume no cheating in this estimate, I merely assume the amount of cheating you’d expect when idly tossing a pen in my room rather than the amount I’d expect when playing billion-dollar games with Warren Buffet.
No. Most of that probability is concentrated in hypotheses where the pen never falls up. This is the difference between a Bayesian probability and a frequency.
I will point out though, that I have probably dropped fewer than 10000 pens in my life and one of them did go sideways (to the best I could tell there was no statistically significant vertical component to its movement), though I suppose I should have predicted that given the weather conditions at the time.
Actually, this is one kind of claim I will happily assign probabilities like 99.9999999999% and higher to, which is the negation of a very specific claim (this is equivalent to the union of a huge number of other claims), for example:
“There is not currently a gang of 17643529 elephants, each painted with the national flag of Moldova, all riding the same giant unicycle around a 57km wide crater on the innermost planet of the 59th closest star to earth in the Andromeda galaxy”.
Happy to go well above 99.9999999999% on that one, as there are easily 10^12 mutually exclusive alternatives, all at least as plausible (in addition, by allowing claims like this, but slightly more plausible, I bet I could find a trillion claims each about as plausible as the pen).
What is impressive about the pen, as well as about most scientific theories, is that it is a single, very specific, hypothesis which earned its high probability through accuracy rather than bulk.
I think you’re definitely right about that; that would also explain why this post isn’t in the Sequences.