Using Ngram to estimate depression prevalence over time

Link post

Historical language records reveal a surge of cognitive distortions in recent decades

My summary: People diagnosed with depression tend to exhibit characteristic patterns of language use that demonstrate the underlying cognitive distortions associated with depression.

For example… individuals label themselves in negative, absolutist terms (e.g., “I am a loser”). They may talk about future events in dichotomous, extreme terms (e.g., “My meeting will be a complete disaster”) or make unfounded assumptions about someone else’s state of mind (e.g., “Everybody will think that I am a failure”). Typologies of cognitive distortions generally differentiate between a number of partially overlapping types, such as “catastrophizing,” “dichotomous reasoning,” “disqualifying the positive,” “emotional reasoning,” “fortune telling,” “labeling and mislabeling,” “magnification and minimization,” “mental filtering,” “mindreading,” “overgeneralizing,” “personalizing,” and “should statements.”

The researchers looked for these language patterns in 14 million books, published over the past 125 years in English, Spanish, and German, that are available via Google Ngram, to see how their prevalence has changed over time.

They found that in general the prevalence of such language patterns decreased or stayed stable over the course of the 20th century up until around 1978. There were some local and temporary spikes (e.g. German-language texts between the world wars and after World War II, English-language texts in 1899 for some reason). In 1978, prevalence began to rise slowly, and then in 2000 more rapidly, leveling out again around 2008 at a historically-high level.

The authors conclude that there has been a recent rapid and strong rise in the use of language patterns that suggest the cognitive distortions associated with depression, in recent years in published books.

Some Concerns

  • Could it be that language fashions change in ways that are independent of depression but that overwhelm the effect of depression on the data? For example, if hyperbole goes in and out of fashion for purely aesthetic reasons, so will “catastrophizing,” “overgeneralizing,” “magnification and minimization.”

  • Similarly, writing today seems more direct, less baroque. A writer today might simply say “Everyone thinks I should exercise if I want to look better,” while a writer 125 years ago might say “It is a truth universally acknowledged that my countenance suffers from want of regular forthright bodily exertion.” Cognitive distortions expressed in a more modern, to-the-point way might be easier to discover in the way the researchers searched. (The authors note that sentence length, as a possible proxy for sentence baroqueness, has been mostly stable since the 1920s.)

  • Might the panel of CBT experts who created the set of phrases to search for have been more aware of current ways of expressing cognitive distortions (those they might hear examples of in their day-to-day lives or work), and less aware of archaic ways of doing so (those that may only exist today in books)? The authors tried to adjust for this by comparing their results to a “null model” of random n-grams, where they chose the sample for this null model to have a recency bias (more n-grams chosen from recently-published books). But their graph of the null model strangely shows more prevalence of those n-grams early in the 20th century than later, so I’m not sure I believe it.

  • I notice a somewhat similar pattern of slow decline through the 20th century followed by a spike in the 21st century for an Ngram search for first-person markers (e.g. I, me, my). Could the whole phenomenon just be explained by a trend toward more first-person narration? The authors say that “the prevalence of the CDS n-grams in the language of individuals with depression is not affected by… the presence of personal pronouns” so they don’t think that’s a factor. But I’m not convinced that really addresses the problem.

  • Authors of books are a peculiar sample of the population at large, as are narrators of fiction. The way those samples have been taken over time could bias the results.