video games > IQ tests

IQ tests are a type of test given to people. What makes them different from other tests given to people?

Is it “consistency of results on a single IQ test type”? Not really. Wikipedia says:

For modern tests, the confidence interval can be approximately 10 points and reported standard error of measurement can be as low as about three points. Reported standard error may be an underestimate, as it does not account for all sources of error.

That’s a best-case scenario for tests designed with that criteria as a priority, and the range is still significant.

Is it “consistency of results across different IQ test types”? Not really; that’s obviously worse than the above, and many “non-IQ” tests have comparable consistency.

Is it “practice being mostly irrelevant”? Not really. A few practice runs can often be worth +8 points, and that’s for kids that already to some extent do IQ-test-like stuff in school. This just mostly doesn’t come up, because institutions don’t generally use IQ test results.

Is it “working for an unusually wide range of intelligence”? Not really. IQ tests are notorious for working poorly above ~135, and I’d say they only really work well for −20 to +0 relative to the designers, with a somewhat wider range for teams.

Is it “unusually high correlation with general intelligence”, as represented by a “g factor”? I don’t think so. IQ tests, in general, don’t seem to be any better for that than the SAT. Anyway, given modern understanding of AI and biology, I consider the entire “g factor” framework an archaic and crude way of understanding intelligence. Humans have multiple mental systems, those systems have performance on various tasks which vary depending on the amount of specialization, amount of training, quality of training data, and multiple types of management by other (meta-level) systems. Then there’s correlation between performance of various systems on various tasks for obvious reasons.

If you’re trying to evaluate intelligence in a broad sense, you should use tests with problems that are big enough to use most of a system and broad enough to use many systems. For example, chess is suboptimal because it’s both too small and narrow to effectively apply the full power of the sort of general-purpose systems that are most important to evaluate when testing for “general intelligence”. The same is true of, say, questions on IIT entrance exams. Leetcode, Math Olympiad, and Physics Olympiad problems have been proposed as better alternatives, but there’s a lot of memorization of a specialized “toolbox” with those.

My view is that some combinations of video games are better “IQ tests” than actual IQ tests are, and better general standardized tests than the SAT. The term “video game” is very general (video games are a generalization of animation is a generalization of film is a generalization of photos are a generalization of text) so let me clarify: I’m talking about existing video games which were developed as entertainment.

I’m caught by a catch-22 here: if I don’t talk about standardized test scores I got, people will think I’m bitter about doing poorly, but if I do, then I’m bragging about scores on standardized tests which is super lame. Plus, no matter how well you did on standardized tests, there’s usually somebody around who did better and wants to brag about it. Well.

My ACT score was in the top 0.1%, but I don’t feel particularly proud of that, because it wasn’t evaluating any of my actual strengths. I left college after a semester (while that was a failure from the perspective of society, school was holding me back intellectually) but I still took the GRE for...reasons...and got a top 1% score without studying, but that’s not something I consider particularly meaningful either. Here’s a theory of Alzheimer’s I developed—what test score does that correspond to? As for IQ tests, I had a couple proper ones as a kid, and my scores were probably as high as was very meaningful, but probably less impressive than reading Feynman in 3rd grade.

Yes, many video games have design goals opposite those of IQ tests, being designed to be played only once and to give the player a strong feeling of progression, but there are many video games, and some are more appropriate. Roguelikes, many strategy games, and many simulation games are designed to be played many times. It might not be as objective, but people could compete on aesthetics too. If practice effects are inevitable, it’s better for everyone to get practice to a point of diminishing returns instead of trying to prevent people from practicing (IQ tests) or charging money for it (SATs).

When people smarter than the test designers take an IQ test, they often have to guess what the designers were thinking, but with video games, evaluation can be completely objective.

The bandwidth and scope possible with video games is much higher than with IQ tests. You can test people with bigger problems, like remembering the units in Wargame Red Dragon, and multidisciplinary challenges, like optimizing both cost and visuals of fireworks in Kerbal Space Program; そういえば、ゲームのウィキの英語を理解することはまたテストのもう一つの側面でしょう.

Video games also have potential legal advantages over IQ tests for companies. You could argue that “we only hire people good at video games to get people who fit our corporate culture of liking video games” but that argument doesn’t work as well for IQ tests.

Jobs in the US now often require a college degree even when the content of the degree is irrelevant to the job. Perhaps you’re tempted here to note that college degrees aren’t just an indication of intelligence, but also of diligence and tolerance for doing pointless BS work. But! Video gamers have already gone out of their way to create a category of competition to test those same things! That’s right: speedruns. You can’t quite match American universities, but you can get somewhat close.