Of course, seeing no piece-meal sabotage is also evidence for there being no fifth column at all.
That makes no sense at all. How can a fact be both for and against the same thing? You can’t split the evidence. You can’t say 20% of the time it means there isn’t a 5th Column and 80% of the time it means there is.
It is possible to have two mutually exclusive propositions, Q and R, such that some observation E is both evidence for Q and evidence for R. That is, it is possible to have both p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R.
Do you disagree?
In my argument above,
Q is “there is a competent fifth column”,
R is “there is no fifth column”, and
E is “there has been no piece-meal sabotage”.
Before moving on to the other issues, does this part make sense now?
ETA: It’s important to note that Q and R, while mutually exclusive, are not exhaustive. A third possibility is that there is fifth column, but it’s incompetent. Q is not equivalent to not-R.
Warren did not say anything about piece-meal sabotage. He called it a “lack of subversive activity”, and he didn’t put limits on what type of subversion.
The way you described E is also counter-intuitive and confusing, and is not the way Warren described it. E must be that there has been subversive activity, that’s what Warren described. He said the lack of E (~E) was evidence of Q (p(Q|~E)). That’s fine, as long as E must be evidence that Q is not true (p(~Q|E)).
Since ~E is weak evidence for Q, E is strong evidence for ~Q. The next subversive activity of any kind will be the first instance of E, and it will invalidate his entire reasoning for detaining the Japanese-American citizens.
Warren did not say anything about piece-meal sabotage. He called it a “lack of subversive activity”, and he didn’t put limits on what type of subversion.
The way you described E is also counter-intuitive and confusing, and is not the way Warren described it.
Okay, replace my earlier definition of E with
E is “we have seen no subversive activity”.
Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?
Set aside the question of whether these are reasonable priors. My point was only this; Warren didn’t make the simple mistake with the probability calculus that Eliezer thought Warren made. He wasn’t simultaneously asserting p(H|E) > p(H) and p(H|~E) > p(H). That would be wrong under any prior, no matter how bizarre. But it’s not what Warren was doing.
What Warren said is consistent with coherent Bayesian updating, even if he was updating on a bizarre prior. It might have been wrong to put a high prior probability on subversive activity, but the probability calculus doesn’t tell you how to pick your prior. All I am saying is that the Warren quote, in and of itself, does not constitute a violation of the rules of the probability calculus.
Maybe Warren committed such a violation earlier on. Maybe that’s how he arrived at such a high prior for the existence of subversive activity. But those earlier steps in his reasoning aren’t laid out before us here, so we can’t point to any specific misapplication of Bayes’s rule, as Eliezer tried to do.
I don’t like the way you describe that. It is confusing. The evidence is subversive activity. You cannot go out and look for no subversive activity, that makes no sense. You have to look for subversive activity. I’m not sure why you’re fighting so hard for this point, since not finding something suggests just as much as finding something does. The only reason I suggest a change is for clarity. I don’t want to think about no subversive activity and not no subversive activity, I want to think about subversive activity or no subversive activity. There is no difference, the second is simply less confusing.
E is subversive activity, and Warren’s position is p(Q|~E).
Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?
Absolutely. I said so two posts up. The question is not about Q and R, though, it’s about Q and ~Q.
Set aside the question of whether these are reasonable priors.
But the whole argument is about the priors. The reason Warren’s position is nonsensical is not because he believes a lack of subversion suggests some fact, it’s that he argues that a lack of subversion suggests a fact, and then behaves in a manner counter to his argument. I’ve been arguing the fact that Warren argues p(Q|~E), But the reason he is locking up the Japanese-Americans is because he expects p(E|Q). The only way p(E|Q) makes any sense is if p(Q|E) is also true.
Warren’s fundamental fear is based on p(E|Q) - that is, the 5th Column is plotting and scheming, and this will lead to subversion. The argument he uses to support this, however, is that p(Q|~E). The two positions are inversely related. If p(E|Q) is strong, then p(Q|~E) must be weak.
In other words, if p(E|Q) is strong, and p(Q) is high, then p(E) should be very high (because Q implies E), and p(~E) should be very small. Yet a very high ~E is used as evidence of p(Q). That makes no sense. If p(E|Q) is high, then ~E can exist in spite of Q, but it cannot exist because of Q.
The only way this is at all tenable is if p(E|Q) and p(~E|Q) are both true. In which case, neither E nor ~E is evidence of Q.
That’s the whole point.
probability calculus doesn’t tell you how to pick your prior.
The whole point of this discussion is that his reasoning does not coincide with his actions. Thus one or the other is wrong.
Doesn’t his position make sense if he believes that:
if there’s no organized fifth column, we should see some intermittent, disorganized sabotage, and
if there is an organized fifth column, we should see NO sabotage before some date, at which there is a devastating attack
?
Of course, I agree that it’s likely he would have made a different argument if he had seen evidence of sabotage—but as presented it seems his position is at least potentially coherent.
But what is the date? Is it 2 months? 6 months? A year? 5 years? What if it never happens? If nothing happens in 200 years, does that mean we must be absolutely certain that a fifth column is planning an attack?
It’s not evidence, it’s a lack of evidence. That’s the point, and that’s the problem.
Warren states it is his most ominous evidence that they are planning something. What evidence? It’s not there, it doesn’t exist. His whole position is based on the idea that the lack of evidence indicates they are planning something, yet he has nothing to suggest that such a lack of evidence indicates anything. The only thing the fact that they haven’t attacked yet is evidence for is that they haven’t attacked yet. Nothing more, nothing less, unless you have a pattern of behavior to base that on. There was no such pattern for the fifth column.
he has nothing to suggest that such a lack of evidence indicates anything.
That’s a different discussion. As you said,
The whole point of this discussion is that his reasoning does not coincide with his actions.
I was simply arguing that your characterization of his argument as inherently self-contradictory was incorrect. Yeah, his supposed priors are probably wrong, but that’s a different issue.
But what is the date? Is it 2 months? 6 months? A year? 5 years?
Okay, say it’s 6 months. Does that make his argument non-contradictory?
If I predict it’s going to rain soon because of a long dry spell, when it rains that doesn’t prove me wrong.
If I predict it’s going to rain soon because of a long dry spell, when it rains that doesn’t prove me wrong.
Of course not, you have a pattern of weather to base that on, in which dry spells were consistently followed by rain.
Where is the basis for a lack of subversion? Historically, a lack of subversion has meant no subversion was ever planned, on what basis is this different for the 5th Column?
Okay, say it’s 6 months. Does that make his argument non-contradictory?
Yes, because now your evidence is that, if there is a 5th column, major subversion occurs every 6 months. This is testable.
His classification of a lack of subversion as evidence that the 5th Column is planning a major strike flies in the face of history—he has a small handful of anomalies to rely on. That’s all.
I’ll point to Eliezer’s example of mammograms in his “Intuitive Explanation of Bayes Theorem” to help describe what I mean, particularly since it’s pretty easy to find a very in-depth beysian analysis of this particular problem by Eliezer himself. In the example, 1% of women get breast cancer. 80% of the time a mammogram will test positive if a woman has breast cancer, 20% of the time it will test negative. 10% of the time a mammogram will test positive for someone who doesn’t have breast cancer. This works out to a 7.8% likelihood that a woman has cancer if she gets a positive result on a mammogram. Conversely, getting a negative result on a mammogram results in a 0.22% likelihood that a woman has cancer.
In the Warren scenario, the 5th Column planning an attack is like the 1% breast cancer rate, and finding evidence of subversion is the mammogram. Not finding any evidence of subversion is the exact same as getting a negative on a mammogram in the breast cancer scenario. It has happened, sure, but it is extremely rare and in the vast majority of cases no subversion means no planned subversion. The problem is you don’t have a history of major subversion without evidence of subversion. Throughout history it has been the exact opposite, therefore a lack of subversion must have a very low probability for preceding a major subversive attack.
Warren’s position is like saying he believes there is a high risk of breast cancer because the mammogram came up negative. The only reasonable response to that is WTF? Yes, it’s possible that the fifth column is planning something, but you cannot assume that because the evidence says otherwise, that’s not reasonable at all. You can come to the conclusion through other evidence, but not with that evidence.
What Warren managed to do is take evidence that did not support his fear and claim that it did. It doesn’t make any sense, it is an unreasonable position to take.
Now, if Warren had said “There is a very low likelihood that the 5th Column is planning a surprise attack, but I am not willing to take that risk” then it’s an entirely different situation, and that is a completely reasonable response. If breast cancer means being forced to fight through Dante’s 9 levels of hell, then it might be worth a double-mastectomy in spite of the 1 in 500 chance that it would happen.
I was wrong when I said that a single case of subversion falsifies his position. Obviously surprise attacks exist, so that was clearly incorrect, and I think it led to a lot of the disagreement in the discussion. I was looking at the problem too narrowly. However the reason surprise attacks are a surprise is because they are very rare, so the fact that nothing has happened must still overwhelmingly support the idea that nothing will happen. In other words, it is overwhelming evidence against an attack, not for it. That’s the only reason surprise attacks work at all, because you you have no evidence to suggest they are coming (and that they haven’t attacked is not such evidence).
I don’t like the way you describe that. It is confusing. The evidence is subversive activity. You cannot go out and look for no subversive activity, that makes no sense.
Our evidence is always only what we have observed. Maybe it is strange to say that you “looked for no subversive activity”. But you certainly can look for subversive activity and fail to find it. Not seeing subversive activity when you looked for it is Bayesian evidence. But it would be an error to condition on there being no subversive activity at all, even hidden activity. That would be going beyond your observations. You can only condition on what you saw or didn’t see when you looked.
Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?
Absolutely. I said so two posts up. The question is not about Q and R, though, it’s about Q and ~Q.
Okay, I think that we’re homing in on the nub of the disagreement.
The propositions in question are
“There is a fifth column that will coordinate a Pearl-Harbor type attack”, and
“There is no fifth column”.
These are Q and R, respectively. They are not negations of each other. Do you agree?
Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
These are Q and R, respectively. They are not negations of each other. Do you agree?
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
With Q there was no evidence that the fifth column was coordinating a timed attack, yet Warren’s strongest evidence for it was that there was no evidence for it.
Pearl harbor types of evidence are black swans. You can’t just pull them out of the air and add them to your reasoning when you have no solid justification for it. There are a billion other black swans he could have used—what if the Japanese are actually all vampires and had designs on draining the Americans dry? You’ve got no evidence they aren’t, so clearly they are just biding their time!
The former is slightly more reasonable, since something similar had happened recently (though in an entirely different context), but it is no more justified as evidence than the evidence in the vampire scenario.
You must look for other evidence that suggests the 5th column was planning an attack, the fact that you have not been attacked yet is not in any way evidence that they are planning an attack. It is only really evidence that, if they were planning something, they hadn’t done it yet. That’s all you can get from that—just a guess.
To that end, Warren had no evidence that an American chapter of the 5th Column even existed. There was secret evidence to that effect, but he was not privy to it. He was making the whole thing up because he was afraid.
It was completely unjustified.
Besides, it doesn’t make sense. Timed attacks are designed to catch you off guard. After Pearl Harbor, people were always on guard. It wouldn’t have had the same effect; a much more effective strategy would have been smaller, guerrilla-type sabotages from within, which they also had zero evidence of.
Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
Think of ~E as meaning “We observed something, but that ‘something’ was something other than subversive activity. That is, what we observed was a member of the class of all things that aren’t subversive activity.”
Is this still “silly”?
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
This wouldn’t change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean “There is no fifth column”. So Warren’s claim, on my reading, would be
p(Q|E) > P(Q) and p(~R|E) > p(~R).
That is, I would just replace “R” everywhere with “~R”. Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.
Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.
My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.
Also, I’m not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.
Warren also never argued their existence, only their activity, so I don’t see why you have a Q and an R at all. Re-read the statement, he took the 5th column’s existence as a given.
What I’m arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.
To that end, I went ahead and made some calculations.
These are my assumptions, and I feel they are historically reasonable (I didn’t cite studies, so I can’t exactly call them accurate):
1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).
Evidence for such plots I call p(evidence).
90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).
This is the critical part of Warren’s statement—he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I’ve frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.
Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn’t have a problem adjusting this number down as low as 1%.
Everything else is calculated based on these three assumptions.
So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.
Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.
It looks like our views have converged. What you wrote above seems to be in agreement with what I wrote here:
What Warren said is consistent with coherent Bayesian updating, even if he was updating on a bizarre prior. It might have been wrong to put a high prior probability on subversive activity, but the probability calculus doesn’t tell you how to pick your prior. All I am saying is that the Warren quote, in and of itself, does not constitute a violation of the rules of the probability calculus.
Maybe Warren committed such a violation earlier on. Maybe that’s how he arrived at such a high prior for the existence of subversive activity. But those earlier steps in his reasoning aren’t laid out before us here, so we can’t point to any specific misapplication of Bayes’s rule, as Eliezer tried to do.
The priors that you use in your calculations look approximately right to me. Warren evidently arrived at different numbers prior to the reasoning that Eliezer quoted, so I agree that he probably made some kind of Bayesian error to get to that point. But I would be hard pressed to say exactly why your numbers seem right to me, so I can’t point to exactly where Warren made his mistake. Whatever his mistake was, it was made prior to the reasoning that Eliezer quoted.
The upshot is that we do not have this nice real-life single-paragraph encapsulation of mathematically fallacious Bayesian reasoning.
Doesn’t his position make sense if he believes that:
if there’s no organized fifth column, we should see some intermittent, disorganized sabotage, and
if there is an organized fifth column, we should see NO sabotage before some date, at which there is a devastating attack
?
Of course, I agree that it’s likely he would have made a different argument if he had seen evidence of sabotage—but as presented it seems his position is at least potentially coherent.
It is possible to have two mutually exclusive propositions, Q and R, such that some observation E is both evidence for Q and evidence for R. That is, it is possible to have both p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R.
Do you disagree?
In my argument above,
Q is “there is a competent fifth column”,
R is “there is no fifth column”, and
E is “there has been no piece-meal sabotage”.
Before moving on to the other issues, does this part make sense now?
ETA: It’s important to note that Q and R, while mutually exclusive, are not exhaustive. A third possibility is that there is fifth column, but it’s incompetent. Q is not equivalent to not-R.
Warren did not say anything about piece-meal sabotage. He called it a “lack of subversive activity”, and he didn’t put limits on what type of subversion.
The way you described E is also counter-intuitive and confusing, and is not the way Warren described it. E must be that there has been subversive activity, that’s what Warren described. He said the lack of E (~E) was evidence of Q (p(Q|~E)). That’s fine, as long as E must be evidence that Q is not true (p(~Q|E)).
Since ~E is weak evidence for Q, E is strong evidence for ~Q. The next subversive activity of any kind will be the first instance of E, and it will invalidate his entire reasoning for detaining the Japanese-American citizens.
Okay, replace my earlier definition of E with
E is “we have seen no subversive activity”.
Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?
Set aside the question of whether these are reasonable priors. My point was only this; Warren didn’t make the simple mistake with the probability calculus that Eliezer thought Warren made. He wasn’t simultaneously asserting p(H|E) > p(H) and p(H|~E) > p(H). That would be wrong under any prior, no matter how bizarre. But it’s not what Warren was doing.
What Warren said is consistent with coherent Bayesian updating, even if he was updating on a bizarre prior. It might have been wrong to put a high prior probability on subversive activity, but the probability calculus doesn’t tell you how to pick your prior. All I am saying is that the Warren quote, in and of itself, does not constitute a violation of the rules of the probability calculus.
Maybe Warren committed such a violation earlier on. Maybe that’s how he arrived at such a high prior for the existence of subversive activity. But those earlier steps in his reasoning aren’t laid out before us here, so we can’t point to any specific misapplication of Bayes’s rule, as Eliezer tried to do.
I don’t like the way you describe that. It is confusing. The evidence is subversive activity. You cannot go out and look for no subversive activity, that makes no sense. You have to look for subversive activity. I’m not sure why you’re fighting so hard for this point, since not finding something suggests just as much as finding something does. The only reason I suggest a change is for clarity. I don’t want to think about no subversive activity and not no subversive activity, I want to think about subversive activity or no subversive activity. There is no difference, the second is simply less confusing.
E is subversive activity, and Warren’s position is p(Q|~E).
Absolutely. I said so two posts up. The question is not about Q and R, though, it’s about Q and ~Q.
But the whole argument is about the priors. The reason Warren’s position is nonsensical is not because he believes a lack of subversion suggests some fact, it’s that he argues that a lack of subversion suggests a fact, and then behaves in a manner counter to his argument. I’ve been arguing the fact that Warren argues p(Q|~E), But the reason he is locking up the Japanese-Americans is because he expects p(E|Q). The only way p(E|Q) makes any sense is if p(Q|E) is also true.
Warren’s fundamental fear is based on p(E|Q) - that is, the 5th Column is plotting and scheming, and this will lead to subversion. The argument he uses to support this, however, is that p(Q|~E). The two positions are inversely related. If p(E|Q) is strong, then p(Q|~E) must be weak.
In other words, if p(E|Q) is strong, and p(Q) is high, then p(E) should be very high (because Q implies E), and p(~E) should be very small. Yet a very high ~E is used as evidence of p(Q). That makes no sense. If p(E|Q) is high, then ~E can exist in spite of Q, but it cannot exist because of Q.
The only way this is at all tenable is if p(E|Q) and p(~E|Q) are both true. In which case, neither E nor ~E is evidence of Q.
That’s the whole point.
The whole point of this discussion is that his reasoning does not coincide with his actions. Thus one or the other is wrong.
Doesn’t his position make sense if he believes that:
if there’s no organized fifth column, we should see some intermittent, disorganized sabotage, and
if there is an organized fifth column, we should see NO sabotage before some date, at which there is a devastating attack
?
Of course, I agree that it’s likely he would have made a different argument if he had seen evidence of sabotage—but as presented it seems his position is at least potentially coherent.
But what is the date? Is it 2 months? 6 months? A year? 5 years? What if it never happens? If nothing happens in 200 years, does that mean we must be absolutely certain that a fifth column is planning an attack?
It’s not evidence, it’s a lack of evidence. That’s the point, and that’s the problem.
Warren states it is his most ominous evidence that they are planning something. What evidence? It’s not there, it doesn’t exist. His whole position is based on the idea that the lack of evidence indicates they are planning something, yet he has nothing to suggest that such a lack of evidence indicates anything. The only thing the fact that they haven’t attacked yet is evidence for is that they haven’t attacked yet. Nothing more, nothing less, unless you have a pattern of behavior to base that on. There was no such pattern for the fifth column.
That’s a different discussion. As you said,
I was simply arguing that your characterization of his argument as inherently self-contradictory was incorrect. Yeah, his supposed priors are probably wrong, but that’s a different issue.
Okay, say it’s 6 months. Does that make his argument non-contradictory?
If I predict it’s going to rain soon because of a long dry spell, when it rains that doesn’t prove me wrong.
Of course not, you have a pattern of weather to base that on, in which dry spells were consistently followed by rain.
Where is the basis for a lack of subversion? Historically, a lack of subversion has meant no subversion was ever planned, on what basis is this different for the 5th Column?
Yes, because now your evidence is that, if there is a 5th column, major subversion occurs every 6 months. This is testable.
His classification of a lack of subversion as evidence that the 5th Column is planning a major strike flies in the face of history—he has a small handful of anomalies to rely on. That’s all.
I’ll point to Eliezer’s example of mammograms in his “Intuitive Explanation of Bayes Theorem” to help describe what I mean, particularly since it’s pretty easy to find a very in-depth beysian analysis of this particular problem by Eliezer himself. In the example, 1% of women get breast cancer. 80% of the time a mammogram will test positive if a woman has breast cancer, 20% of the time it will test negative. 10% of the time a mammogram will test positive for someone who doesn’t have breast cancer. This works out to a 7.8% likelihood that a woman has cancer if she gets a positive result on a mammogram. Conversely, getting a negative result on a mammogram results in a 0.22% likelihood that a woman has cancer.
In the Warren scenario, the 5th Column planning an attack is like the 1% breast cancer rate, and finding evidence of subversion is the mammogram. Not finding any evidence of subversion is the exact same as getting a negative on a mammogram in the breast cancer scenario. It has happened, sure, but it is extremely rare and in the vast majority of cases no subversion means no planned subversion. The problem is you don’t have a history of major subversion without evidence of subversion. Throughout history it has been the exact opposite, therefore a lack of subversion must have a very low probability for preceding a major subversive attack.
Warren’s position is like saying he believes there is a high risk of breast cancer because the mammogram came up negative. The only reasonable response to that is WTF? Yes, it’s possible that the fifth column is planning something, but you cannot assume that because the evidence says otherwise, that’s not reasonable at all. You can come to the conclusion through other evidence, but not with that evidence.
What Warren managed to do is take evidence that did not support his fear and claim that it did. It doesn’t make any sense, it is an unreasonable position to take.
Now, if Warren had said “There is a very low likelihood that the 5th Column is planning a surprise attack, but I am not willing to take that risk” then it’s an entirely different situation, and that is a completely reasonable response. If breast cancer means being forced to fight through Dante’s 9 levels of hell, then it might be worth a double-mastectomy in spite of the 1 in 500 chance that it would happen.
I was wrong when I said that a single case of subversion falsifies his position. Obviously surprise attacks exist, so that was clearly incorrect, and I think it led to a lot of the disagreement in the discussion. I was looking at the problem too narrowly. However the reason surprise attacks are a surprise is because they are very rare, so the fact that nothing has happened must still overwhelmingly support the idea that nothing will happen. In other words, it is overwhelming evidence against an attack, not for it. That’s the only reason surprise attacks work at all, because you you have no evidence to suggest they are coming (and that they haven’t attacked is not such evidence).
Hopefully I’ve explained myself adequately now.
Our evidence is always only what we have observed. Maybe it is strange to say that you “looked for no subversive activity”. But you certainly can look for subversive activity and fail to find it. Not seeing subversive activity when you looked for it is Bayesian evidence. But it would be an error to condition on there being no subversive activity at all, even hidden activity. That would be going beyond your observations. You can only condition on what you saw or didn’t see when you looked.
Okay, I think that we’re homing in on the nub of the disagreement.
The propositions in question are
“There is a fifth column that will coordinate a Pearl-Harbor type attack”, and
“There is no fifth column”.
These are Q and R, respectively. They are not negations of each other. Do you agree?
Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
With Q there was no evidence that the fifth column was coordinating a timed attack, yet Warren’s strongest evidence for it was that there was no evidence for it.
Pearl harbor types of evidence are black swans. You can’t just pull them out of the air and add them to your reasoning when you have no solid justification for it. There are a billion other black swans he could have used—what if the Japanese are actually all vampires and had designs on draining the Americans dry? You’ve got no evidence they aren’t, so clearly they are just biding their time!
The former is slightly more reasonable, since something similar had happened recently (though in an entirely different context), but it is no more justified as evidence than the evidence in the vampire scenario.
You must look for other evidence that suggests the 5th column was planning an attack, the fact that you have not been attacked yet is not in any way evidence that they are planning an attack. It is only really evidence that, if they were planning something, they hadn’t done it yet. That’s all you can get from that—just a guess.
To that end, Warren had no evidence that an American chapter of the 5th Column even existed. There was secret evidence to that effect, but he was not privy to it. He was making the whole thing up because he was afraid.
It was completely unjustified.
Besides, it doesn’t make sense. Timed attacks are designed to catch you off guard. After Pearl Harbor, people were always on guard. It wouldn’t have had the same effect; a much more effective strategy would have been smaller, guerrilla-type sabotages from within, which they also had zero evidence of.
He didn’t observe “nothing.” He observed factories and shipyards and so forth, continuing to operate without apparent sabotage.
Think of ~E as meaning “We observed something, but that ‘something’ was something other than subversive activity. That is, what we observed was a member of the class of all things that aren’t subversive activity.”
Is this still “silly”?
This wouldn’t change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean “There is no fifth column”. So Warren’s claim, on my reading, would be
p(Q|E) > P(Q) and p(~R|E) > p(~R).
That is, I would just replace “R” everywhere with “~R”. Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.
Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.
My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.
Also, I’m not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.
Warren also never argued their existence, only their activity, so I don’t see why you have a Q and an R at all. Re-read the statement, he took the 5th column’s existence as a given.
What I’m arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.
To that end, I went ahead and made some calculations.
These are my assumptions, and I feel they are historically reasonable (I didn’t cite studies, so I can’t exactly call them accurate):
1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).
Evidence for such plots I call p(evidence).
90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).
This is the critical part of Warren’s statement—he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I’ve frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.
Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn’t have a problem adjusting this number down as low as 1%.
Everything else is calculated based on these three assumptions.
p(subversion) = 1% p(~subversion) = 99%
p(evidence|subversion) = 90% p(~evidence|subversion) = 10% p(evidence|~subversion) = 4.95% p(~evidence|~subversion) = 94.05%
p(evidence) = 5.85% p(~evidence) = 94.15%
p(subversion|evidence) = 15.52% p(~subversion|evidence) = 35.35% p(subversion|~evidence) = 0.11% p(~subversion|~evidence) = 99.89%
So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.
Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.
It looks like our views have converged. What you wrote above seems to be in agreement with what I wrote here:
The priors that you use in your calculations look approximately right to me. Warren evidently arrived at different numbers prior to the reasoning that Eliezer quoted, so I agree that he probably made some kind of Bayesian error to get to that point. But I would be hard pressed to say exactly why your numbers seem right to me, so I can’t point to exactly where Warren made his mistake. Whatever his mistake was, it was made prior to the reasoning that Eliezer quoted.
The upshot is that we do not have this nice real-life single-paragraph encapsulation of mathematically fallacious Bayesian reasoning.
Doesn’t his position make sense if he believes that:
if there’s no organized fifth column, we should see some intermittent, disorganized sabotage, and
if there is an organized fifth column, we should see NO sabotage before some date, at which there is a devastating attack
?
Of course, I agree that it’s likely he would have made a different argument if he had seen evidence of sabotage—but as presented it seems his position is at least potentially coherent.