Would it help if I wrote them out more explicitly rather than tagged them into sentences?
Yes, the edit definitely makes it better.
Yes I have to choose a distribution but if I’m forced to predict an unknown int32 with no additional information the uniform distribution seems like a reasonable choice. Ad-hoc, not explicitly defined probability distributions are common in this discussion.
Well it’s clearly not a reasonable choice given that it results in a fallacy. I think if the actual error is using the wrong prior distribution then this should be reflected in what the diagnostic dilution fallacy is defined as. It’s not putting the element into the larger set because that isn’t false. I’d suggest a phrasing but I don’t have a good enough grasp on the concept to do this, and I’m still not even sure that the argument you’re criticizing is in fact an example of this fallacy if were phrased more precisely. (Also nitpicky, but “condition” in the definition is odd as well if they’re general hypotheses.)
The form of the fallacy is “forgetting” the first proposition X in favour of a weaker one Y. In the example, this means we put ourselves in the position of answering “you will be given a nonnegative int32 tomorrow, no further clues, predict what it is”. How would you choose a distribution in this situation?
Empirically, most uses of values of 32-bit integers (e.g. in programming in general) have magnitudes very much smaller than 2^30. A significant proportion of their empirical values are between 0 and 10. So my prior would be very much non-uniform.
I would expect something like a power law, possibly on the order of P(X >= x) ~ x^-0.2 or so except with some smaller chance of negative values as well, maybe 10-20% of that of positive ones (which the statement “nonnegative” eliminates), and a spike at zero of about 20-40% of total probability. It should also have some smaller spikes at powers of two and ten, and also one less than those.
This would still be a bad estimate for the number of children, once I found out that you were actually talking about children, but not by nearly as much.
Now you’re sort of asking me to do the work for you, but I did get interested enough to start thinking about it, so here’s my more invested take.
So first of all, I don’t see how this is a form of the fallacy of the undistributed middle. The article you linked to says that we’re taking A⟹C and B⟹C and conclude A⟹B. I don’t see how your fallacy is doing (a probabilistic version of) that. Your fallacy is taking X"⟹"Y as given (with "⟹" meaning “makes more likely”), and Y"⟹"A and concluding X"⟹"A
Second
We’ve substituted a sharp condition for a vague one, hence the name diagnostic dilution.
I think we’ve substituted a vague condition for a sharp one, not vice-versa? The 32 bit integer seems a lot more vague than the number being about kids?
Third, your leading example isn’t an example of this fallacy, and I think you only got away with pretending it’s one by being vague about the distribution. Because if we tried to fix it, it would have to be like this
A: the number is probably > 100000
X: the number is a typical prior for having kids
Y: the number is a roughly uniformly distributed 32 bit integer
And X"⟹"Y is not actually true here. Whereas in the example you’re criticizing
A: the AI will have seek to eliminate threatening agents
X: the AI builds football stadiums
Y: the AI has goal-directed behavior
here X"⟹"Y does seem to be true.
(And also I believe the fallacy isn’t even a fallacy because if X"⟹"Y and Y"⟹"A together do in fact imply X"⟹"A, at least if both "⟹" are sufficiently strong?)
So my conclusion here is that the argument actually just doesn’t work, or I still just don’t get what you’re asserting,[1] but the example you make does not seem to have the same problems as the example you’re criticizing, and neither of them seems to have the same structure as the example of the fallacy you’re linking. (For transparency, I initially weak-downvoted because the post seemed confusing and I wasn’t sure if it’s valid, then removed the downvote because you improved the presentation, now strong-downvoted because the central argument seems just broken to me now.)
like maybe the fallacy isn’t about propositions implying each other but instead about something more specific to an element being in a set, but at this point the point is just not argued clearly.
You’ve just substituted a different proposition and then claimed that the implication doesn’t hold because it doesn’t hold for your alternative proposition. “We’re counting kids” absolutely implies “the count can be represented by a nonnegative int32”. If I want to show that an argument is unsound I am allowed to choose the propositions that demonstrate it’s unsoundness.
The X⟹Y implication is valid in your formulation, but then Y doesn’t imply anything because it says nothing about the distribution. I’m saying that if you change Y to actually support your Y⟹A conclusion, then X⟹Y fails. Either way the entire argument doesn’t seem to work.
Sorry but this is nonsense. JBlack’s comment shows the argument works fine even if you take a lot of trouble to construct P(count|Y) to give a better answer.
But this isn’t even particularly important, because for your objection to stand, it must be impossible to find any situation where P(A|Y) would give you a silly answer, which is completely false.
Yes, the edit definitely makes it better.
Well it’s clearly not a reasonable choice given that it results in a fallacy. I think if the actual error is using the wrong prior distribution then this should be reflected in what the diagnostic dilution fallacy is defined as. It’s not putting the element into the larger set because that isn’t false. I’d suggest a phrasing but I don’t have a good enough grasp on the concept to do this, and I’m still not even sure that the argument you’re criticizing is in fact an example of this fallacy if were phrased more precisely. (Also nitpicky, but “condition” in the definition is odd as well if they’re general hypotheses.)
The form of the fallacy is “forgetting” the first proposition X in favour of a weaker one Y. In the example, this means we put ourselves in the position of answering “you will be given a nonnegative int32 tomorrow, no further clues, predict what it is”. How would you choose a distribution in this situation?
Empirically, most uses of values of 32-bit integers (e.g. in programming in general) have magnitudes very much smaller than 2^30. A significant proportion of their empirical values are between 0 and 10. So my prior would be very much non-uniform.
I would expect something like a power law, possibly on the order of P(X >= x) ~ x^-0.2 or so except with some smaller chance of negative values as well, maybe 10-20% of that of positive ones (which the statement “nonnegative” eliminates), and a spike at zero of about 20-40% of total probability. It should also have some smaller spikes at powers of two and ten, and also one less than those.
This would still be a bad estimate for the number of children, once I found out that you were actually talking about children, but not by nearly as much.
Now you’re sort of asking me to do the work for you, but I did get interested enough to start thinking about it, so here’s my more invested take.
So first of all, I don’t see how this is a form of the fallacy of the undistributed middle. The article you linked to says that we’re taking A⟹C and B⟹C and conclude A⟹B. I don’t see how your fallacy is doing (a probabilistic version of) that. Your fallacy is taking X"⟹"Y as given (with "⟹" meaning “makes more likely”), and Y"⟹"A and concluding X"⟹"A
Second
I think we’ve substituted a vague condition for a sharp one, not vice-versa? The 32 bit integer seems a lot more vague than the number being about kids?
Third, your leading example isn’t an example of this fallacy, and I think you only got away with pretending it’s one by being vague about the distribution. Because if we tried to fix it, it would have to be like this
A: the number is probably > 100000
X: the number is a typical prior for having kids
Y: the number is a roughly uniformly distributed 32 bit integer
And X"⟹"Y is not actually true here. Whereas in the example you’re criticizing
A: the AI will have seek to eliminate threatening agents
X: the AI builds football stadiums
Y: the AI has goal-directed behavior
here X"⟹"Y does seem to be true.
(And also I believe the fallacy isn’t even a fallacy because if X"⟹"Y and Y"⟹"A together do in fact imply X"⟹"A, at least if both "⟹" are sufficiently strong?)
So my conclusion here is that the argument actually just doesn’t work, or I still just don’t get what you’re asserting,[1] but the example you make does not seem to have the same problems as the example you’re criticizing, and neither of them seems to have the same structure as the example of the fallacy you’re linking. (For transparency, I initially weak-downvoted because the post seemed confusing and I wasn’t sure if it’s valid, then removed the downvote because you improved the presentation, now strong-downvoted because the central argument seems just broken to me now.)
like maybe the fallacy isn’t about propositions implying each other but instead about something more specific to an element being in a set, but at this point the point is just not argued clearly.
You’ve just substituted a different proposition and then claimed that the implication doesn’t hold because it doesn’t hold for your alternative proposition. “We’re counting kids” absolutely implies “the count can be represented by a nonnegative int32”. If I want to show that an argument is unsound I am allowed to choose the propositions that demonstrate it’s unsoundness.
The X⟹Y implication is valid in your formulation, but then Y doesn’t imply anything because it says nothing about the distribution. I’m saying that if you change Y to actually support your Y⟹A conclusion, then X⟹Y fails. Either way the entire argument doesn’t seem to work.
Sorry but this is nonsense. JBlack’s comment shows the argument works fine even if you take a lot of trouble to construct P(count|Y) to give a better answer.
But this isn’t even particularly important, because for your objection to stand, it must be impossible to find any situation where P(A|Y) would give you a silly answer, which is completely false.