What is unpersuasive about the responses to Mother Jones?
They’re exactly what I thought when I read it. Actually, I had a more specific thought: what changed in 2011 is that they started collecting data live, rather than through archives. Of course, rejecting a data set because it was produced by hand in an ad hoc manner does not give you a replacement data set and thus does not produce an actual analysis. But the Reason link suggests Duwe’s data as a replacement. Since he starts with official data and only uses media coverage to fill in details, he isn’t subject to temporal bias.
What is unpersuasive about the responses to Mother Jones?
I agree that Duwe’s point is the closest thing there to a decent argument against MJ’s data. But I think the accusation that there data is “cherrypicked” is not reasonably supported. The entire paragraph in Siegel’s piece where he argues for this is essentially ignoring that what they are using is what fits closely with the common intuition of what is a mass shooting. The only one which one could plausibly take out of that set is Fort Hood but it doesn’t alter the data very much.
Most of Siegel’s points are correct but not relevant to the question of increases of shootings. For example, he’s correct that there’s a serious measuring issue with whether shootings are stopped by others with weapons, and he’s also correct that even if the trend identified by MJ is accurate it is still a tiny fraction of total crimes and will remain so, but that’s not actually relevant to evaluating the central claim.
What about the methodology of starting with news reports? These have strong biases that probably change by time. And how did they locate decades old news reports? A recency bias is to be expected. (Added: Duwe has a paper about this!)
Their definition is pretty reasonable, except for the part where they make tons of exceptions. The examples they gave of the exceptions that they made sound intuitive, but what about all the exceptions that they didn’t talk about? Why didn’t they include the Ridgewood Postal murders? Since they don’t actually have a consistent rule, it’s impossible to decide if they applied it to this case, or if the actual criterion was that it was too old.
Added: I tried spot-checking a few examples from Duwe’s book against the MJ list. On pages 115-116 he lists 12 high profile workplace mass shootings. In addition to Ridgewood, they omit Alan Winterbourne and Willie Woods, both very straight-forward examples. If they miss these high profile examples, why would you expect them to reliably find others?
What is unpersuasive about the responses to Mother Jones?
They’re exactly what I thought when I read it. Actually, I had a more specific thought: what changed in 2011 is that they started collecting data live, rather than through archives. Of course, rejecting a data set because it was produced by hand in an ad hoc manner does not give you a replacement data set and thus does not produce an actual analysis. But the Reason link suggests Duwe’s data as a replacement. Since he starts with official data and only uses media coverage to fill in details, he isn’t subject to temporal bias.
I agree that Duwe’s point is the closest thing there to a decent argument against MJ’s data. But I think the accusation that there data is “cherrypicked” is not reasonably supported. The entire paragraph in Siegel’s piece where he argues for this is essentially ignoring that what they are using is what fits closely with the common intuition of what is a mass shooting. The only one which one could plausibly take out of that set is Fort Hood but it doesn’t alter the data very much.
Most of Siegel’s points are correct but not relevant to the question of increases of shootings. For example, he’s correct that there’s a serious measuring issue with whether shootings are stopped by others with weapons, and he’s also correct that even if the trend identified by MJ is accurate it is still a tiny fraction of total crimes and will remain so, but that’s not actually relevant to evaluating the central claim.
What about the methodology of starting with news reports? These have strong biases that probably change by time. And how did they locate decades old news reports? A recency bias is to be expected. (Added: Duwe has a paper about this!)
Their definition is pretty reasonable, except for the part where they make tons of exceptions. The examples they gave of the exceptions that they made sound intuitive, but what about all the exceptions that they didn’t talk about? Why didn’t they include the Ridgewood Postal murders? Since they don’t actually have a consistent rule, it’s impossible to decide if they applied it to this case, or if the actual criterion was that it was too old.
Added: I tried spot-checking a few examples from Duwe’s book against the MJ list. On pages 115-116 he lists 12 high profile workplace mass shootings. In addition to Ridgewood, they omit Alan Winterbourne and Willie Woods, both very straight-forward examples. If they miss these high profile examples, why would you expect them to reliably find others?
Thinking about this more, I think you are correct. The data is much too spotty to make a strong conclusion.