If there’s a reason to be interested in average IQ beyond mutual ego-massage, I guess the best way would be to have an IQ test where you logged on as ‘Less Wrong member X’ and then it reported all the results, not just the ones that people chose to share.
Would still suffer from selection effects. People that thought they might not do so well would be disinclined to do it, and people who knew they were hot shit would be extra inclined to do it. The phrase “anonymous survey” doesn’t really penetrate into our status-aware hindbrains.
Better: randomly select a group of users (within some minimal activity criteria) and offer the test directly to that group. Publicly state the names of those selected (make it a short list, so that people actually read it, maybe 10-20) and then after a certain amount of time give another public list of those who did or didn’t take it, along with the results (although don’t associate results with names). That will get you better participation, and the fact that you have taken a group of known size makes it much easier to give outer bounds on the size of the selection effect caused by people not participating.
You can also improve participation by giving those users an easily accessible icon on Less Wrong itself which takes them directly to the test, and maybe a popup reminder once a day or so when they log on to the site if they’ve been selected but haven’t done it yet. Requires moderate coding.
I would find such a feature to be extraordinarily obnoxious, to the point that I’d be inclined to refused such a test purely out of anger (and my scores are not at all embarrassing). I can’t think of any other examples of a website threatening to publicly shame you for non-compliance.
Would still suffer from selection effects. People that thought they might not do so well would be disinclined to do it, and people who knew they were hot shit would be extra inclined to do it. The phrase “anonymous survey” doesn’t really penetrate into our status-aware hindbrains.
Yep! But it’s the best way I can imagine that someone could plausibly create on the forum.
Better: randomly select a group of users (within some minimal activity criteria) and offer the test directly to that group. Publicly state the names of those selected (make it a short list, so that people actually read it, maybe 10-20) and then after a certain amount of time give another public list of those who did or didn’t take it, along with the results (although don’t associate results with names). That will get you better participation, and the fact that you have taken a group of known size makes it much easier to give outer bounds on the size of the selection effect caused by people not participating.
You can also improve participation by giving those users an easily accessible icon on Less Wrong itself which takes them directly to the test, and maybe a popup reminder once a day or so when they log on to the site if they’ve been selected but haven’t done it yet. Requires moderate coding.
I would find such a feature to be extraordinarily obnoxious, to the point that I’d be inclined to refused such a test purely out of anger (and my scores are not at all embarrassing). I can’t think of any other examples of a website threatening to publicly shame you for non-compliance.