The form of the fallacy is “forgetting” the first proposition X in favour of a weaker one Y. In the example, this means we put ourselves in the position of answering “you will be given a nonnegative int32 tomorrow, no further clues, predict what it is”. How would you choose a distribution in this situation?
Empirically, most uses of values of 32-bit integers (e.g. in programming in general) have magnitudes very much smaller than 2^30. A significant proportion of their empirical values are between 0 and 10. So my prior would be very much non-uniform.
I would expect something like a power law, possibly on the order of P(X >= x) ~ x^-0.2 or so except with some smaller chance of negative values as well, maybe 10-20% of that of positive ones (which the statement “nonnegative” eliminates), and a spike at zero of about 20-40% of total probability. It should also have some smaller spikes at powers of two and ten, and also one less than those.
This would still be a bad estimate for the number of children, once I found out that you were actually talking about children, but not by nearly as much.
Now you’re sort of asking me to do the work for you, but I did get interested enough to start thinking about it, so here’s my more invested take.
So first of all, I don’t see how this is a form of the fallacy of the undistributed middle. The article you linked to says that we’re taking A⟹C and B⟹C and conclude A⟹B. I don’t see how your fallacy is doing (a probabilistic version of) that. Your fallacy is taking X"⟹"Y as given (with "⟹" meaning “makes more likely”), and Y"⟹"A and concluding X"⟹"A
Second
We’ve substituted a sharp condition for a vague one, hence the name diagnostic dilution.
I think we’ve substituted a vague condition for a sharp one, not vice-versa? The 32 bit integer seems a lot more vague than the number being about kids?
Third, your leading example isn’t an example of this fallacy, and I think you only got away with pretending it’s one by being vague about the distribution. Because if we tried to fix it, it would have to be like this
A: the number is probably > 100000
X: the number is a typical prior for having kids
Y: the number is a roughly uniformly distributed 32 bit integer
And X"⟹"Y is not actually true here. Whereas in the example you’re criticizing
A: the AI will have seek to eliminate threatening agents
X: the AI builds football stadiums
Y: the AI has goal-directed behavior
here X"⟹"Y does seem to be true.
(And also I believe the fallacy isn’t even a fallacy because if X"⟹"Y and Y"⟹"A together do in fact imply X"⟹"A, at least if both "⟹" are sufficiently strong?)
So my conclusion here is that the argument actually just doesn’t work, or I still just don’t get what you’re asserting,[1] but the example you make does not seem to have the same problems as the example you’re criticizing, and neither of them seems to have the same structure as the example of the fallacy you’re linking. (For transparency, I initially weak-downvoted because the post seemed confusing and I wasn’t sure if it’s valid, then removed the downvote because you improved the presentation, now strong-downvoted because the central argument seems just broken to me now.)
like maybe the fallacy isn’t about propositions implying each other but instead about something more specific to an element being in a set, but at this point the point is just not argued clearly.
You’ve just substituted a different proposition and then claimed that the implication doesn’t hold because it doesn’t hold for your alternative proposition. “We’re counting kids” absolutely implies “the count can be represented by a nonnegative int32”. If I want to show that an argument is unsound I am allowed to choose the propositions that demonstrate it’s unsoundness.
The X⟹Y implication is valid in your formulation, but then Y doesn’t imply anything because it says nothing about the distribution. I’m saying that if you change Y to actually support your Y⟹A conclusion, then X⟹Y fails. Either way the entire argument doesn’t seem to work.
Sorry but this is nonsense. JBlack’s comment shows the argument works fine even if you take a lot of trouble to construct P(count|Y) to give a better answer.
But this isn’t even particularly important, because for your objection to stand, it must be impossible to find any situation where P(A|Y) would give you a silly answer, which is completely false.
The form of the fallacy is “forgetting” the first proposition X in favour of a weaker one Y. In the example, this means we put ourselves in the position of answering “you will be given a nonnegative int32 tomorrow, no further clues, predict what it is”. How would you choose a distribution in this situation?
Empirically, most uses of values of 32-bit integers (e.g. in programming in general) have magnitudes very much smaller than 2^30. A significant proportion of their empirical values are between 0 and 10. So my prior would be very much non-uniform.
I would expect something like a power law, possibly on the order of P(X >= x) ~ x^-0.2 or so except with some smaller chance of negative values as well, maybe 10-20% of that of positive ones (which the statement “nonnegative” eliminates), and a spike at zero of about 20-40% of total probability. It should also have some smaller spikes at powers of two and ten, and also one less than those.
This would still be a bad estimate for the number of children, once I found out that you were actually talking about children, but not by nearly as much.
Now you’re sort of asking me to do the work for you, but I did get interested enough to start thinking about it, so here’s my more invested take.
So first of all, I don’t see how this is a form of the fallacy of the undistributed middle. The article you linked to says that we’re taking A⟹C and B⟹C and conclude A⟹B. I don’t see how your fallacy is doing (a probabilistic version of) that. Your fallacy is taking X"⟹"Y as given (with "⟹" meaning “makes more likely”), and Y"⟹"A and concluding X"⟹"A
Second
I think we’ve substituted a vague condition for a sharp one, not vice-versa? The 32 bit integer seems a lot more vague than the number being about kids?
Third, your leading example isn’t an example of this fallacy, and I think you only got away with pretending it’s one by being vague about the distribution. Because if we tried to fix it, it would have to be like this
A: the number is probably > 100000
X: the number is a typical prior for having kids
Y: the number is a roughly uniformly distributed 32 bit integer
And X"⟹"Y is not actually true here. Whereas in the example you’re criticizing
A: the AI will have seek to eliminate threatening agents
X: the AI builds football stadiums
Y: the AI has goal-directed behavior
here X"⟹"Y does seem to be true.
(And also I believe the fallacy isn’t even a fallacy because if X"⟹"Y and Y"⟹"A together do in fact imply X"⟹"A, at least if both "⟹" are sufficiently strong?)
So my conclusion here is that the argument actually just doesn’t work, or I still just don’t get what you’re asserting,[1] but the example you make does not seem to have the same problems as the example you’re criticizing, and neither of them seems to have the same structure as the example of the fallacy you’re linking. (For transparency, I initially weak-downvoted because the post seemed confusing and I wasn’t sure if it’s valid, then removed the downvote because you improved the presentation, now strong-downvoted because the central argument seems just broken to me now.)
like maybe the fallacy isn’t about propositions implying each other but instead about something more specific to an element being in a set, but at this point the point is just not argued clearly.
You’ve just substituted a different proposition and then claimed that the implication doesn’t hold because it doesn’t hold for your alternative proposition. “We’re counting kids” absolutely implies “the count can be represented by a nonnegative int32”. If I want to show that an argument is unsound I am allowed to choose the propositions that demonstrate it’s unsoundness.
The X⟹Y implication is valid in your formulation, but then Y doesn’t imply anything because it says nothing about the distribution. I’m saying that if you change Y to actually support your Y⟹A conclusion, then X⟹Y fails. Either way the entire argument doesn’t seem to work.
Sorry but this is nonsense. JBlack’s comment shows the argument works fine even if you take a lot of trouble to construct P(count|Y) to give a better answer.
But this isn’t even particularly important, because for your objection to stand, it must be impossible to find any situation where P(A|Y) would give you a silly answer, which is completely false.