Thanks for the post. I love it.
My comments:
First sidenote that dont assume that if something is a heuristic it is automatically a wrong way of thinking.(sorry if i misinterpret this, because you dont explicitly say this at all :) In some situations simple heuristics will outperform regression analysis for example.
But about your mainpoint. If I understood right this is actually a problem of violating so called “ratio rule”.
(1) The degree to which c is representative of S is indicated by the conditional propability p (c | S)- that is, the propability of members of S have characterestic c.
(2) The propability that the characteristic c implies membership S is given by p (S | c). (Like you write)
(3) p (c | S) / p (S | c) = p(c) / p(S)
This is the Ratio Rule= Ratio of inverse propabilities equals the ratio of simple propabilities. So to equate these two propabilities p(c|S) and p(S|c) in the absence of equating ALSO the simple propabilitis is just wrong and bad thinking.
Representative thinking does not reflect these differences between p(c|S) and p(S|c) and introduces a symmetry in the map (thought) that does not exist in the world.
For example: “Home is the most dangerous place in the world because most accidents happen in home. So stay away from home!!!” --> This is confusion about the propability of accident given being home with propability being home given accident.
Thanks for the link! Very nice publication!