This could possibly be explained if the heuristic also embodied some background information that allows us to correct for overconfidence if P(f|e) that isn’t explicitly mentioned in the data
Exactly. If there’s some structure to the full dataset that’s unrecoverable from the part that’s kept, you can code that structure into a heuristic which will outperform Naive Bayes on the remaining data- but an ideal Bayesian reasoner with access to the full dataset would have picked up that structure as well, and you wouldn’t be outperforming xer.
So the post is evidence of interesting structure in word frequencies, not a deficiency of Bayes’ Rule.
Exactly. If there’s some structure to the full dataset that’s unrecoverable from the part that’s kept, you can code that structure into a heuristic which will outperform Naive Bayes on the remaining data- but an ideal Bayesian reasoner with access to the full dataset would have picked up that structure as well, and you wouldn’t be outperforming xer.
So the post is evidence of interesting structure in word frequencies, not a deficiency of Bayes’ Rule.