[Question] If I showed the EQ-SQ theory’s findings to be due to measurement bias, would anyone change their minds about it?

[tailcalled]

Background on EQ-SQ

The EQ-SQ theory is a theory by Simon Baron-Cohen which states that there’s a biological sex difference in tendency to empathize with people (Empathizing) versus try to understand deterministic systems (Systematizing), and that autism represents the extreme Systematizing end of this spectrum.

Measurement bias

Since writing this subthread where I found ambiguous and partial evidence that the EQ-SQ theory’s findings are due to measurement bias, I’ve been considering collecting direct evidence with SBC’s EQ and SQ-R scales, and write up a detailed post investigating measurement bias.

Measurement bias, if it exists (as it very much seems to do on e.g. the EQ scale), would most likely involve observing that the sex difference on the scales is limited to one subset of the items, the autistic-allistic difference is limited to another subset of the items, and maybe also the main thing the scale measures exists on a yet third subset of the items.

That is, it would involve observing that the scales mix different things together, and that autism and sex have distinct relationships with the things that get mixed together. As an analogy, imagine showing that autism is extreme masculinity by creating a “maleness test” which mixes together physical strength and poor social skills. Yes, this would have a large sex difference and a large autistic-allistic difference, but that’d transparently be measurement bias; the reason for the large sex difference (it measures physical strength) would be distinct from the reason for the large autistic-allistic difference (it measures social skills).

My concern

My concern is that people didn’t start believing in the EQ-SQ theory based on statistical correlations found with Simon Baron-Cohen’s scales. They presumably started believing in it based on fuzzy intuitions arrived at through social experience.

So I could imagine that if I did show SBC’s findings to be due to measurement error, people would sort of dismiss it as poorly designed scales and still continue believing in the theory. Like basically maybe SBC’s studies provide a faux-objective criterion they can use to throw at feminists, but if the criterion doesn’t validate sex differences then EQ-SQ enthusiasts would discard it and pick a new one.

Thoughts?

[Ninety-Three]

If someone updated towards the “autism is extreme maleness” theory after reading an abstract based on your hypothetical maleness test, you could probably argue them out of that belief by explaining the specific methodology of the test, because it’s obviously dumb. If you instead had to do a bunch of math to show why it was flawed, then it would be much harder to convince people because some wouldn’t be interested in reading a bunch of math, some wouldn’t be able to follow it, and some would have complicated technical nitpicks about how if you run these numbers slightly differently you get a different result.

Separate from the “Is that your true rejection?” question, I think the value of making this argument depends heavily on how simple you can make the explanation. No matter how bulletproof it is, a counterargument that takes 10000 words to make will convince fewer people than one that can be made in 100 words.

[tailcalled]

Maybe it would help if the explanation also had a simplified story and then an in-depth description of how one arrived at the simplified story?

Like the simplified story for how the EQ is wrong is “The EQ conflates two different things, ‘not caring about people’ and ‘not knowing how to interact with people’. The former is male while the latter is autistic.”

I don’t know for sure what the issue with the SQ is, but I suspect it’s going to be something like “The SQ conflates five different things, ‘being interested in technology’, ‘being interested in politics’, ‘being interested in nature’, ‘orderliness’ and ‘artistic creativity’. The former two are male while ?some unknown subset? are autistic.”

The noteworthy bit is that one can detect these sorts of conflations from the statistics of the scales.

[Steven Byrnes]

I’m already not sympathetic to extreme-male-brain-theory-of-autism, so I guess I’m not the target of this question. I suppose that if you found the opposite results of what you expect, I would be mildly interested.

Unrelatedly, I strong-vote for you to spend more time thinking about cluster B personality disorders. If paying you money would help make that happen, we should talk! (Your recent post on narcissism was already helpful!)

[tailcalled]

I’m already not sympathetic to extreme-male-brain-theory-of-autism, so I guess I’m not the target of this question. I suppose that if you found the opposite results of what you expect, I would be mildly interested.

Intense World does seem like it could be a contributing factor.

Ironically, despite being autistic, I feel like I don’t really understand how autism is recognized or how it works. So while I do have some theories on it, they feel uncertain/​not very crisp.

One of my current main theories (which might be overupdating on my own case, but I guess I might as well put it out here so I can either get corrected or confirmed) is that a core contributor is “falling off” the general “track” of socialization/​enculturation. This is kind of abstract, so let me try to explain:

I think as people interact with each other, they get exposed to certain norms and cultural ideas. If they have the appropriate context, they can notice these norms/​ideas and parse them in a similar way that others do, “keeping up” with everyone else about what is going on. This probably starts all the way back in childhood, with games, movies, play and other things leading to social experience. If one doesn’t engage in this in similar ways that others do, one might miss out on a lot.

In my case, the missing out seems to be polycausal. I’m temperamentally naturally introverted; my family moved to a new place when I was 3; I have difficulties with sports (a primary activity) for reasons that seem connected to neurological autism symptoms; I have strong unusual interests (mainly computers and science, which of course fits into some variants of the EQ-SQ theory); my family didn’t have many cartoons or shows for children and didn’t keep up well with movies (people keep being surprised that I didn’t watch classics such as The Lion King); I tended to get involved with weird political ideas; and I didn’t make many friends.

There’s a lot of basic norms and purposes about social interactions that I didn’t really learn until relatively recently. And there’s a lot of basic signs that I still don’t understand—people often seem to mention stuff like car brands, cities, clothing brands or other things and have an idea about the relative status differences (and other things?) implied by which brands you use or which cities you come from/​live in, but I usually don’t know what people are referring to.

And I think this also contributed to a feedback loop? Like once you’re already behind, you’re gonna be the weird kid who has few friends engaged in unusual activities and who doesn’t “get it” when stuff is explained.

So basically, my lead theory—which I don’t at all feel confident in but which seems reasonable to me—is that autism is due to a variety of factors (neurological, temperamental, environmental, …) which interfere with the enculturation process and creates a feedback loop of being off track.

Unrelatedly, I strong-vote for you to spend more time thinking about cluster B personality disorders. If paying you money would help make that happen, we should talk! (Your recent post on narcissism was already helpful!)

Neat that you like my post! I’d be curious if there’s anything specific you can mention as being helpful? (Maybe in DMs if you prefer?)

As for spending more time thinking about Cluster B, maybe? I think I’m currently evidence-bottlenecked rather than theory-bottlenecked. But this constraint can of course also be loosened through the use of money and time. I’ve also been reading a few case studies, though the case studies typically don’t have the sorts of details I need, so it is kind of of limited value.

(In a way, it’s kind of weird for me to lack so much evidence, since I grew up with a BPD mother. But I think my mother’s conflicts were mostly with my brother, my father, and various people outside the home, not so much with me until later. At which point I think my dabbling in conservative online spaces had made me adopt memes which present a stoic, resistant face to BPD while ignoring the conflicts. So I don’t think I got a clear view of the conflicts. Have been meaning to discuss them more with my brother.)

[Steven Byrnes]

I currently think there’s kinda a “neurotypical way of relating to people”, which involves having certain involuntary innate reactions in certain social circumstances.

I disagree with the popular narrative that the “neurotypical way of relating” is equal to “social intelligence”, or to “good theory of mind”. In this comment I offer an example where the “neurotypical way of relating” leads to transparently awful theory-of-mind. The “autistic people have less social intelligence” claims I’ve seen are very unconvincing, and seem to be a mix of “autistic people have to work harder to predict/​model neurotypical people, and vice-versa (!!), for obvious reasons” [like Ann mentioned in a different comment], and “autistic people tend to have less social motivation, and a great many sloppy scientists will mix up social motivation with social intelligence /​ theory-of-mind”.

Anyway, “autism” /​ ASD is a big tent (and getting bigger each year—I have personal experience here as a parent, see 1,2), complicating any discussion or literature analysis. But I think “classic autism” (i.e. historical diagnostic standards, see here) more-or-less corresponds to not engaging in the “neurotypical way of relating”.

For example, there’s an interesting report here (I haven’t read the book yet, just bought it!) which I interpret as: a lifelong autistic person was shoved into “the neurotypical way of relating” via brain stimulation. They found this way of relating to be often overwhelming, which incidentally (I presume) is why they were avoiding that way of relating thus far in life. (Presumably this started with a deliberate flinching-away from that kind of relating during very early childhood, which eventually into an unconscious deep-rooted lifelong habit.)

Does the “neurotypical way of relating” develop by practicing a lot as a child? I think some practice is important—completely isolated children have severe issues, for example. But leaving aside extreme cases, I think I’m less inclined to emphasize practice than you do. My impression is that it’s possible for kids to be awfully introverted, antisocial, and oblivious to pop-culture, while being clearly not autistic. Also, the guy from the previous paragraph also seems to be a point against “lack of practice” being central. Another example: when adults immigrate to a different culture, the social norms and conversational norms and cultural references are all unknown-to-them, and they certainly have issues getting by for a while, but I don’t think those transient enculturation issues look anything like autism. I’m open to changing my mind though. And sorry if I’m misunderstanding.

Neat that you like my post! I’d be curious if there’s anything specific you can mention as being helpful?

I’ll reply at that post. :)

[Ann]

Another example: when adults immigrate to a different culture, the social norms and conversational norms and cultural references are all unknown-to-them, and they certainly have issues getting by for a while, but I don’t think those transient enculturation issues look anything like autism.

Interesting looked at in reverse—from at least anecdotal data, autistic folk often report being much more comfortable traveling in another culture, because the social norms, conversational norms and cultural references are expected to be unknown to them, and people we interact with therefore tend to be much more charitable about them.

[tailcalled]

Does this include socially conservative autistic people? I have the impression that the autistic people who are more prominent or coordinated tend to be socially progressive, and that socially progressive people have greater enjoyment of foreign cultures.

(Incidentally, I also have the impression that a lot of the EQ-SQ debate is really about this? Some conservative male autist saying “sex/​race differences are real!”, puritanical progressives going “how could you say such a horrible thing?!”, the conservative autist going “I don’t understand what I did wrong, maybe it is because of my male brain being very logical rather than obsessed about social harmony?”. And then a big part of why EQ-SQ theory is so marginalized is because progressive autists don’t want to be associated with sexism/​racism, so they go “no, that’s not autism, he’s just a horrible person!”. In a way, this connects to my point in the post; I could respond on an object level to the arguments forwarded by SBC, but if EQ-SQ stuff is really motivated by this sort of drama, then maybe people wouldn’t be convinced by anything other than a response to that drama?)

[Ann]

I don’t know as many probably-socially-conservative probably-autistic people, but from who I do know they seem to enjoy spending time in foreign cultures still? Not very firm data there, even anecdotally, though.

[tailcalled]

I disagree with the popular narrative that the “neurotypical way of relating” is equal to “social intelligence”, or to “good theory of mind”.

This comment inspired me to collect data on autistic social functioning, and I was surprised at how basically similarly good it appeared to be to allistic social functioning.^[1] Probably worth investigating in greater detailed, but yes very good to take into account.

My theory is that in autism the “response feelings” in story (2) are so strong that they’re aversive, and people with classic autism adopt the coping strategy of avoiding invoking them altogether, by avoiding the specific type of mental operation that I call “empathetic simulation”.

This is interesting because it does square with my experience if I understand correctly. Like as an example, as a child my mother explained how she had gotten a teacher’s education because she had wanted to be a psychologist and the government had some thing where you could get a brief psychologist education on top of the teacher’s education, but then they eliminated that and now she had to be a teacher. And I got really sad about that. I guess one difference from your story is that I less suppressed it because of direct unpleasantness and more because of social conformity.

Does the “neurotypical way of relating” develop by practicing a lot as a child? I think some practice is important—completely isolated children have severe issues, for example. But leaving aside extreme cases, I think I’m less inclined to emphasize practice than you do. My impression is that it’s possible for kids to be awfully introverted, antisocial, and oblivious to pop-culture, while being clearly not autistic. Also, the guy from the previous paragraph also seems to be a point against “lack of practice” being central. Another example: when adults immigrate to a different culture, the social norms and conversational norms and cultural references are all unknown-to-them, and they certainly have issues getting by for a while, but I don’t think those transient enculturation issues look anything like autism. I’m open to changing my mind though. And sorry if I’m misunderstanding.

Good arguments, I’m convinced.

1. ^

   Admittedly feels kind of hard to square with Against Against Autism Cures, e.g. “One study investigated how many autistics have at least one friend and found it was just under 50%.”. Not sure what makes the difference here; I’d be inclined to say that maybe a lot of autistic dysfunction is for nonsocial reasons (e.g. “locked in a sensory hell without the ability to explain their problems verbally, and maybe having seizures all the time to boot”?) and my method filters away the cases that are this severe, and autistic people can basically function fine socially if neither oppressed (which I’d conjecture goes together with being an adult in practice?) nor disabled by sensory sensitivities? Possibly it’s also just that my measurement is too bad.

[tailcalled]

Oh also, another thing:

The concept of “Social Intelligence” seems to come up a bunch, both in the context of autism and in the context of a bunch of socioeconomic research. But I think existing social intelligence scales are kind of bad. To understand how, let’s contrast them to the notion of general intelligence.

IQ tests try to measure the g factor, cognitive abilities that are useful across a wide variety of tasks. To do so, they can just grab a bunch of cognitive tasks and average the performance across them together, and this tends to give a reasonable measurement of the g factor, precisely because it is so general.

One might want to do the same with social intelligence, grabbing a bunch of social tasks and evaluating performance over them. But one can’t really do this because social tasks are “big”; they involve interacting with other people, but it is expensive to have another person around, and it takes a long time to build enough relationships with them to perform any sort of meaningful task.

My view of existing social intelligence tests is that they address this by postulating some “core capabilities” which are used in social tasks, and then they aim to measure those core capabilities rather than social performance. For instance, “Mind In The Eyes test” assumes that a core capability of social intelligence is being able to read someone’s mind from the expression they give with their eyes.

At its core, there are basically two kinds of social intelligence tasks, differing in the ways they measure their “core capabilities”. “Trait-based” social intelligence tasks measure them in basically the same way that personality tests do, and so they basically end up being personality tests that have been relabeled to be “social intelligence”. Meanwhile, “ability-based” social intelligence tests measure them in basically the same way that IQ tests do, and so they basically end up being IQ tests that have been relabeled to be “social intelligence”.

Ultimately, because social tasks are “big”, I think a social intelligence measure must be “formative” like the above. However, without a fairly accurate objective outcome criterion, it is hard to know what exactly to measure (what is redundant, important, missing, etc.). Lately I’ve been thinking that it must be possible to do way better with respect to outcome criteria. I feel like it should surely be possible to measure social performance by considering relationship history. That is, people may have relationships with their family, friends, coworkers, romantic partners and others, and it seems like it should be easy to assess how well those relationships are going, and use this as a ground truth criterion for a sort of “general social functioning”. And then it seems like it should be possible to study why some people do better or worse in relationships, and come up with a more specific measure of social intelligence which centers on the factors relevant to that.

Constructing a measure of social intelligence seems relevant to autism, as autism is often conceptualized as being closely related to poor social skills. It might also be relevant to personality disorders, as poor social skills might also contribute to them? Idk. I guess if this was to be done specifically for autism, one should probably first check to what extent autistic social relationship performance is worse than allistic social relationship performance. The effect size might be too small to be informative.

[Ann]

Also needs to account for any manifestation of the “double empathy problem”—if us autistic folk have some degree of ‘social intelligence’ that works perfectly well with autistic folk but falters with allistic folk, and vice versa, then what are we measuring?

An example might be, one allistic social intelligence test is to determine emotional state from the expression of the eyes, and …
… here I realize that there’s not exactly a standardized way to correctly determine recognition of states like inanimate object feelings, and not everyone is lexythmic enough to score perception of their emotional state overall …

… well, it needs some workshopping. But given the potential extent it’s just tricky for minds that don’t think alike to connect socially, we want to be explicit about what we’re measuring; if that’s social relationship performance independent of allistic/​autistic state, specifically our ability to perform social relationships with allistic folk, the difference, or what.

[tailcalled]

I think there’s two ways this would show up in the research process.

First, autistic people’s social outcomes would have the potential to become better than what one would naively expect, as their relationships with other autists would drag up their social outcomes scores. This would lead to such factors being weighted less when one studies factors which influence social outcomes.

Secondly, once some concrete social abilities are discovered, one can directly investigate whether they exhibit double empathy dynamics.

[Muireall]

I hope you don’t mind my piling on. This is one of those things (along with moral foundations theory) where it really frustrates me that people seem to believe it has much better scientific backing than it does.

My concern is that people didn’t start believing in the EQ-SQ theory based on statistical correlations found with Simon Baron-Cohen’s scales. They presumably started believing in it based on fuzzy intuitions arrived at through social experience.

It’s hard to gloss empathizing-systemizing/​extreme male brain as a theory that Baron-Cohen himself arrived at in a principled or scientific way.

As presented in The Extreme Male Brain Theory of Autism (2002), researchers do find apparent sex differences in behavior and cognition. It’s possible to frame many of these as female-empathizing and male-systemizing. Then there are a bunch of other, mostly unrelated studies (and a few citation-free stereotypes) of behavior and cognition in autistic people, which can also be framed as low-empathizing and high-systemizing. The authors draw this figure, classifying brain types by the difference between empathizing and systemizing dimensions in “standard deviations from the mean”:

But the first, key quantitative paper I’m aware of is Empathizing and Systemizing in Males, Females, and Autism (2005). The actual plot of EQ and SQ looks like this:

The authors write, “In this particular case, it was possible to see immediately what combination of EQ and SQ govern the data, but in general this might require using a principal components analysis.” By this they mean the C and D axes (sum and difference of normalized EQ and SQ), writing, “The combination of the normalization steps and the rotation represents a principal components analysis of of this correlated bivariate data set.” This paper doesn’t quote standard deviations of EQ and SQ scores, but it’s “possible to see immediately” that (1) the male-female differences on both axes are much less than two standard deviations (as indicated by the earlier, schematic figure, which also appears in later publications), and (2) the male-female difference in EQ+SQ (C Axis) is insignificant, while the autistic group has much lower C-Axis scores. This should not have passed review.

Clearly, I’m not someone whose mind you’re trying to change, but I think your concern is on target. EQ-SQ is a bad measure, but even taken seriously it provides about as much evidence against the “extreme male brain” idea as it does in favor. I doubt that highlighting more problems with EQ-SQ is going to move people very much.