A related mistake I made was to be impressed by the cleverness of the aphorism “The plural of ‘anecdote’ is not ‘data’.” There may be a helpful distinction between scientific evidence and Bayesian evidence. But anecdotal evidence is evidence, and it ought to sway my beliefs.
The selection biases in anecdotes make them nearly useless for updating. A more correct version would be that you can update on the first anecdote, less on a similar second one, even less on a third, and so on. Once you have ten or so anecdotes pointing in the same direction, then extra anecdotes should have essentially no impact.
So yes, the plural of anecdote is not data. Their value does not scale in the same way.
Essentially the difference between the two is how systematic the gathering of information is. A juicy urban legend gets passed around and repeated with small variations all other place: hearing it ten times is uninformative. TripAdivsor gathers reviews in a more systematic way, so is better. If people started sending each other the snarkiest reviews they saw on TripAdvisor, this would degenerate more into anecdotes again.
The question is always, are you getting a reasonable sample of the anecdotes out there.
The selection biases in anecdotes make them nearly useless for updating. A more correct version would be that you can update on the first anecdote, less on a similar second one, even less on a third, and so on. Once you have ten or so anecdotes pointing in the same direction, then extra anecdotes should have essentially no impact.
So yes, the plural of anecdote is not data. Their value does not scale in the same way.
It depends on how independent the anecdotes are.
I wouldn’t agree that 100 reviews on TripAdvisor for a hotel should weigh little more than 10 reviews.
That approaches data more.
Essentially the difference between the two is how systematic the gathering of information is. A juicy urban legend gets passed around and repeated with small variations all other place: hearing it ten times is uninformative. TripAdivsor gathers reviews in a more systematic way, so is better. If people started sending each other the snarkiest reviews they saw on TripAdvisor, this would degenerate more into anecdotes again.
The question is always, are you getting a reasonable sample of the anecdotes out there.
The plural of anecdote is qualitative study.