One way to think of it is that there were three filters:
only some LW visitors became aware of the survey when you first posted it (by reading about it in the last paragraph of your article)
of those who made it through filter 1, only some took the survey right then (in round 1)
of those who made it through filter 1 and did not make it through filter 2, only some took the survey later in round 2
With all three filters, it seemed like high AQ people would be more likely to make it through, but it was hard to estimate how strongly each filter would select for high AQ. I expected filter 1 to be the strongest (at selecting for high AQ), and filter 2 to be a bit stronger than filter 3. The data suggest that filter 3 was (if anything) very slightly stronger than filter 2, which requires some updating. But since one group went through filters 1 & 2 and the other went through filters 1 & 3, the data don’t speak directly to the strength of filter 1. You’re inferring that all three filters are probably relatively weak, but I don’t see a good reason to conclude that about filter 1.
Yeah, you actually predicted that the second responders would be slightly less AS-ish than the first responders, but actually they are slightly more.
To be honest, this result surprised me too, I expected that the AQ scores would go down, so I’m updating towards the “no large net bias” hypothesis.
One way to think of it is that there were three filters:
only some LW visitors became aware of the survey when you first posted it (by reading about it in the last paragraph of your article)
of those who made it through filter 1, only some took the survey right then (in round 1)
of those who made it through filter 1 and did not make it through filter 2, only some took the survey later in round 2
With all three filters, it seemed like high AQ people would be more likely to make it through, but it was hard to estimate how strongly each filter would select for high AQ. I expected filter 1 to be the strongest (at selecting for high AQ), and filter 2 to be a bit stronger than filter 3. The data suggest that filter 3 was (if anything) very slightly stronger than filter 2, which requires some updating. But since one group went through filters 1 & 2 and the other went through filters 1 & 3, the data don’t speak directly to the strength of filter 1. You’re inferring that all three filters are probably relatively weak, but I don’t see a good reason to conclude that about filter 1.