I should note, considering the original graph, that it’s designed in such a way that it would look rather similar even if the point it were trying to make were universally considered the exact opposite of the truth by all study participants.
Consider the current U.S. population pyramid. Suppose participants were fairly sampled from its adults, maybe with some trailing off among people on the threshold who don’t have landlines and live in dorms where they don’t check their mail. Suppose further that everyone in America agreed that the 1990′s were when America had the most reliable news reporting. You’d end up with a bimodal distribution with a small bump around the 30 year mark, and a slightly higher bump around zero—exactly what we observe. This would be read as “These silly people! Most of them are just telling us the time when they were kids and couldn’t understand the news, with others saying their 20′s and 30′s, when they were first forming their lifelong political allegiances.”. Despite the fact that the entire sample agreed that a single point in time was objectively the best, and every member responded accordingly.
This, of course, doesn’t falsify the point that Yglesias is trying to make, but it does show that the way he presented it could make the same point that it does even if the data was clearly opposed to it. Moreover, by altering the ages sampled, you could get any individual graph to say anything you want—sample younger people for a lower “YEARS OLD” on any issue, older people for a higher one. A much more informative way of reporting the data would be reporting the distributions of responses for each 10 year age bracket, which would show a smoothly X-shifting distribution between groups if the poll’s thesis is correct, and something else if it is not.
Agreed, but when one sees a graph like that one assumes that the person making it has ensured their sample draws equally from every age group (not proportionally from every age group).
I should note, considering the original graph, that it’s designed in such a way that it would look rather similar even if the point it were trying to make were universally considered the exact opposite of the truth by all study participants.
Consider the current U.S. population pyramid. Suppose participants were fairly sampled from its adults, maybe with some trailing off among people on the threshold who don’t have landlines and live in dorms where they don’t check their mail. Suppose further that everyone in America agreed that the 1990′s were when America had the most reliable news reporting. You’d end up with a bimodal distribution with a small bump around the 30 year mark, and a slightly higher bump around zero—exactly what we observe. This would be read as “These silly people! Most of them are just telling us the time when they were kids and couldn’t understand the news, with others saying their 20′s and 30′s, when they were first forming their lifelong political allegiances.”. Despite the fact that the entire sample agreed that a single point in time was objectively the best, and every member responded accordingly.
This, of course, doesn’t falsify the point that Yglesias is trying to make, but it does show that the way he presented it could make the same point that it does even if the data was clearly opposed to it. Moreover, by altering the ages sampled, you could get any individual graph to say anything you want—sample younger people for a lower “YEARS OLD” on any issue, older people for a higher one. A much more informative way of reporting the data would be reporting the distributions of responses for each 10 year age bracket, which would show a smoothly X-shifting distribution between groups if the poll’s thesis is correct, and something else if it is not.
Agreed, but when one sees a graph like that one assumes that the person making it has ensured their sample draws equally from every age group (not proportionally from every age group).