Aella didn’t even mention Lizardmen’s Constant once, but she made a bunch of twitter polls, which shows a different percentage of liars depending on the seriousness of the question and the survey.
I think probably the immediate visibility and informality of the twitter poll gives people a jolt of delight to fuck with the results, as long as fucking with the results would be funny. 1+1 = 3 is hilarious when it’s in front of a crowd and everybody sees the stupid percentages being stupid, but it’s boring when it’s inside the solemn walls of a researcher’s guidedtrack url.
(3³ − 23) ÷ 2 = 3, however, is deadly serious. Nobody finds that funny, crowd or not. It’s the one question where the twitter poll and survey question collapsed to almost exactly the same % of correct answer.
In a new study my collaborators and I focus on this hypothesis. We ask participants if this statement is true or false: “The Canadian Armed Forces have been secretly developing an elite army of genetically engineered, super intelligent, giant raccoons to invade nearby countries”.
We found that 10% of participants endorsed this statement (i.e., they selected “Definitely true” or “Probably true”) and that this was a strong predictor of endorsing six pre-existing conspiracy theories, including two that that directly contradicted each other.
Thank you for the references! Aella’s case is interesting a very good example ofhow design matters to how you should think about the problem, particularly her statement (emphasis added):
in general whenever I’ve asked variations on this set of questions, the number of correct votes roughly increases with the difficulty of the problem, before hitting a ceiling when the difficulty is too high, and slowly decreases with that difficulty.
In a lot of survey design the opposite is a much larger problem, particularly if you incentivize responses (i.e. paying people for completing the survey). The more complex you make a question, the more often you will have lazy respondents just pick the first question instead of engaging with what is being asked. Because she is just posing a twitter poll with little incentive to answer, I imagine most lazy respondents simply do not read the question.
Related: How Your Survey Responder Lies
Aella didn’t even mention Lizardmen’s Constant once, but she made a bunch of twitter polls, which shows a different percentage of liars depending on the seriousness of the question and the survey.
There is also this showing Lizardmen’s Constant = 10%, showing how much the “constant” can vary:
Thank you for the references! Aella’s case is interesting a very good example of how design matters to how you should think about the problem, particularly her statement (emphasis added):
In a lot of survey design the opposite is a much larger problem, particularly if you incentivize responses (i.e. paying people for completing the survey). The more complex you make a question, the more often you will have lazy respondents just pick the first question instead of engaging with what is being asked. Because she is just posing a twitter poll with little incentive to answer, I imagine most lazy respondents simply do not read the question.