Another proof that survey design is hard: should I answer “yay male/male sex, I strongly support same-sex ” or “boo male/male sex, I am not interested?” Or, taking a page from Alicorn’s book, what about those who say “yay male/male sex, I’d like to be interested in men?” (I’d expect this to be a statistically detectable portion of test-takers.)
Also, making people write essays just to throw them away is not a terribly productive use of anyone’s time.
In the meantime, I suppose individuals can approximate the same behavior by writing such things in a file on their hard drive. It won’t affect processing of the survey, of course, but then it wouldn’t really do so anyway.
Longer-term, presumably the goals we want to achieve with a question should drive the options we provide for answers. If we want to correlate demographic category with other answers, then we really don’t care about demographic categories that cover fewer than 5% or so of the population, since such correlations would be even less useful than baseline, but we do care about standardizing answers. If we want to know how LessWrong readers identify themselves because we’re curious, we don’t really care about standardizing answers, but we do want to let respondents use their own terms to describe themselves. Etc.
Perhaps future surveys should have exhaust valves channeling people’s need to express themselves:
1) In any number of words, what is your theory of gender? (essay section)
2) On unsophisticated government forms that only have the options “male” and “female”, which do you select? (multiple choice, two options)
3) Sex with people who gave the same answer to 2), yay or boo? (multiple choice, two options)
4) Sex with people who gave a different answer to 2), yay or boo? (multiple choice, two options)
5) In any number of words, what are your political views? (essay section)
6) Which nine of the following ten political terms most poorly describe that position (multiple choice, ten options).
etc.
Another proof that survey design is hard: should I answer “yay male/male sex, I strongly support same-sex ” or “boo male/male sex, I am not interested?” Or, taking a page from Alicorn’s book, what about those who say “yay male/male sex, I’d like to be interested in men?” (I’d expect this to be a statistically detectable portion of test-takers.)
Also, making people write essays just to throw them away is not a terribly productive use of anyone’s time.
In the meantime, I suppose individuals can approximate the same behavior by writing such things in a file on their hard drive. It won’t affect processing of the survey, of course, but then it wouldn’t really do so anyway.
Longer-term, presumably the goals we want to achieve with a question should drive the options we provide for answers. If we want to correlate demographic category with other answers, then we really don’t care about demographic categories that cover fewer than 5% or so of the population, since such correlations would be even less useful than baseline, but we do care about standardizing answers. If we want to know how LessWrong readers identify themselves because we’re curious, we don’t really care about standardizing answers, but we do want to let respondents use their own terms to describe themselves. Etc.