Accuracy of Autism Self-Diagnosis

This was inspired by my previous post using data from the Wrong Planet discussion forum. I noticed that a large chunk of my sample were self-diagnosed, and that these individuals scored no differently on a self-report measure of empathy than the officially diagnosed respondents. However, I had a small sample size, especially when splitting into groups, and the test I was comparing them on wasn't a diagnostic test for autism spectrum conditions.

But that gave me an idea, and I found another Wrong Planet thread, a stickied thread where a poster had linked to several online tests useful for self-assessment of autistic traits. Overall, 237 people (myself included) had posted their results on at least one of these tests, including 63 diagnosed with Asperger Syndrome, 20 with other autism spectrum conditions, 76 self-diagnosed Aspies, 65 who were unsure if they were on the spectrum or not, 11 with a family member on the spectrum and 2 neurotypicals. (The latter two groups were put together for further analysis.) There were 101 males and 136 females, similar to the gender ratio in my previous study. All the groups had similar gender ratios.

The tests taken were the Broad Autism Phenotype Questionnaire, the Autism Spectrum Quotient, the Empathizing and Systemizing Quotients, the Emotional IQ Test, the Highly Sensitive Person test, the 'Reading the mind in the eyes' test, the Cambridge Face Memory test (the link is broken), the Aspie Quiz, the Asperger Syndrome Self-Assessment test and the Toronto Alexithymia Scale. Of those 10 tests, 5 directly assess for autistic traits, 2 test for social skills but not other aspects of autism, and the last 3 assess conditions that commonly co-occur with autism.

A one-way ANOVA found significant differences between groups on the BAPQ aloof and pragmatic scales, ASQ, EQ and Aspie Quiz. In all of those cases, this difference was due to the family/NT group scoring as less autistic than the other groups. In most cases, all four autistic/possibly autistic groups scored significantly more autistic than the family/NT group, with the exception of the BAPQ pragmatic scale in which only the diagnosed Asperger Syndrome and the self-diagnosed group's differences reached significance (these groups also have the largest sample size). None of the other four groups showed any significant differences with each other.

Then I decided to see which percentage of individuals in each group met cutoff scores on the various tests.

The BAPQ is designed to assess the broader autistic phenotype, rather than autism spectrum conditions. As a result, the cutoff scores (89 for aloof, 81 for rigid and 79 for pragmatic) simply determine whether or not the person has a genetic predisposition to autism, and not whether they're autistic. However, autistic people would be expected to score quite high on this scale, and so they do (group differences almost significant at p=.073):

• diagnosed AS: 87% meet cutoff on all three scales, 4% on 2 scales and 9% on one scale only
• other autistic: 89% meet cutoff on all three scales and 11% on only 2 scales
• self-diagnosed: 91% meet cutoff on all three scales, 4% on 2 scales and 1% on one scale only
• unsure if autistic: 88% meet cutoff on all three scales, 10% on 2 scales and 2% on one scale
• family/NT: 67% meet criteria on all three scales and 33% meet criteria on none of the scales (sample size of 3)

The ASQ is a test for autism spectrum conditions. Studies have shown that a cutoff of 32 shows good reliability in distinguishing autistic people from non-autistic people. This cutoff clearly distinguished the family/NT group from the others (p<.001, though the other autistic group fell short of significance):

• diagnosed AS: 92% scored at or above 32
• other autistic: 69% scored at or above 32
• self-diagnosed: 95% scored at or above 32
• unsure if autistic: 83% scored at or above 32
• family/NT: 43% scored at or above 32

The Empathizing and Systemizing Quotients are based on the Extreme Male Brain theory of autism popularized by Simon Baron-Cohen. EQ is a measure of social skills and empathy, and SQ (the type used here is the SQ-R) is a measure of mechanical/scientific inclination. The ratio of EQ to SQ is used to classify individuals into four types - Extreme Empathizer, Empathizer, Balanced, Systemizer, and Extreme Systemizer. Autistic people usually score as Extreme Systemizers (family/NT differ from all other groups at p<.001):

• diagnosed AS: 90% Extreme S, 3% S, 5% Balanced and 2% E
• other autistic: 81% Extreme S, 13% S and 6% Balanced
• self-diagnosed: 96% Extreme S, 4% S
• unsure if autistic: 92% Extreme S, 6% S and 2% Balanced
• family/NT: 50% Extreme S and 50% Extreme E (sample size of 4)

The Emotional Intelligence test was not designed to assess autistic people. Instead, it was designed in reaction to the observation that full-scale IQ has relatively poor power to predict lifetime success, leading researchers to suspect that another factor was important. It is designed to be normed like an IQ test, and therefore a score under 75 would be deficient. Between-group differences on this test were not significant(p=.498), with 40-70% of each group scoring below that cutoff. In general, it seems that autistic people tend to perform poorly on this test, but the effect is not very strong.

The Asperger Syndrome Self-Assessment had very low response rates, with 3 diagnosed AS, 1 other autistic, 1 self-diagnosed and 3 unsure if autistic participants filling it out. As a result, I will not analyze it further.

The Highly Sensitive Person test assesses a personality construct of high reactivity to physical and emotional stimuli. The stereotypical highly-sensitive person is highly empathetic and therefore not autistic, but the test taps many questions regarding sensory sensitivities and therefore autistic people are expected to score highly. The Cambridge Face Memory test assesses for prosopagnosia, an impairment in facial recognition. Prosopagnosia appears to be a common comorbid condition for autism. The Toronto Alexithymia Scale assesses alexithymia, an impairment in understanding one's own emotions (in contrast to the autistic impairment in understanding other people's emotions). Alexithymia is more common in autism spectrum individuals than the general population. However, only 5 respondents filled out the TAS, so I excluded it from further analysis. I assessed presence of the other two conditions, using a cutoff of 14 or above for the HSP test and less than 55% correct for the CFMT (differences between groups non-significant at p=.429 for HSP and p=.121 for prosopagnosia):

• diagnosed AS: 94% highly sensitive, 22% prosopagnosic
• other autistic: 93% highly sensitive, 30% prosopagnosic
• self-diagnosed: 83% highly sensitive, 9% prosopagnosic
• unsure if autistic: 85% highly sensitive, 6% prosopagnosic
• family/NT: 67% highly sensitive, none prosopagnosic (sample size 3 for both)

In general, these results suggest that virtually all of the self-diagnosed and unsure if autistic groups probably are on the autism spectrum. But a single test does not determine this as well as multiple tests do. So I selected the three diagnostic tests with the best response rates - BAPQ, ASQ and EQSQ - and assessed how many scored in the autistic range on all three tests (above cutoff on all three scales of BAPQ, over 32 on ASQ and 'Extreme Systemizer'). Significant differences were found with a p=.037, post-hoc tests indicating that the family/NT group scored lower than all four other groups:

• diagnosed AS: 84% scored 3, 8% scored 2, 4% scored 1 and 4% scored 0
• other autistic: 67% scored 3, 8% scored 2 and 25% scored 1
• self-diagnosed: 88% scored 3, 9% scored 2 and 3% scored 1
• unsure if autistic: 70% scored 3, 23% scored 2 and 7% scored 1
• family/NT: 50% scored 3 and 50% scored 0 (sample size 6)

So far, all results have suggested that self-diagnosed autistics, and even those who aren't sure if they're on the spectrum, generally score the same as people who were officially diagnosed with an autism spectrum condition. But most of these tests are self-report tests, and therefore do not necessarily indicate if others would perceive the self-diagnosed person as autistic. However, two tests in this bunch are objective performance tests - the Cambridge Face Memory test, and the Eyes test. The latter taps an area thought to be central to autism spectrum conditions, and showed no group differences.

Oddly enough, though, most of the participants, even those with official diagnoses, scored in the normal range or better on this test (22 or more correct), with 47% of diagnosed AS, 39% of other autistic, 28% of self-diagnosed, 32% of unsure and none (out of 2) of the family/NT group scoring in the deficient range. The average scores (22.03, 21.23, 23.65, 23.05 in the four autistic groups) are similar to the findings of Baron-Cohen et al (2001), and suggest that many high functioning autistics have too subtle of deficits for this test to detect. One possible alteration could be to make this test timed, as many autistic participants reported that this test took a long time to complete. (Also, a few reported giving up on the test because they had no clue about any of them, suggesting that some of the most severely-impaired participants may have been omitted from the test's analysis.) Another possibility might be to make it free response instead of multiple choice, since some participants said that what they thought was the correct answer was not even one of the options.

Ideally, I would like to run a study in which a large sample of self-diagnosed autistics were assessed by a team of psychologists experienced in diagnosing adult high-functioning autistics. However, this study suggests that self-diagnosed autistics - at least those on the Wrong Planet forum - score about the same as officially diagnosed autistics on a wide variety of tests. Contrary to the stereotype that self-diagnosis of autism is inaccurate, this study suggests that it's actually very accurate in the majority of cases. Undoubtedly some self-diagnosed autistics are not in fact on the autism spectrum, but so are some officially diagnosed autistics. And from what I can tell, the misdiagnosis rate is pretty similar between self-diagnosis and official diagnosis.