when framed in terms of social interactions, people’s performance dramatically improves
From the Wikipedia article, after invoking evolutionary psychology and social interaction to explain the improvement:
Alternatively, it could just mean that there are some linguistic contexts in which people tend to interpret “if” as a material conditional, and other linguistic contexts in which its most common vernacular meaning is different.
It shouldn’t be hard to present the test as a real world example that doesn’t involve social interaction (e.g. “If lights are on, there is electricity in the house”).
/me goes off to test this on a couple of linguistics students
Result: One correct and one incorrect answer.
You can append ”?context=100″ to the comment permalink.