Adequacy as Levels of Play
Adequacy as Levels of Play
One method that I find very useful for evaluating adequacy concerns is “level of play”. If you look at different games or different leagues of the same game, it’s pretty apparent that the “level of play”—the amount of athleticism, effort, skill, planning, strategy, etc. that is on display—is quite different.
For instance, in the United States the NFL is operating at a higher level of play than college football. Similarly, baseball in the United States operates at a higher level of play than baseball in Japan, and either is significantly elevated compared to the rest of the world. In South Korea, Starcraft is treated as a professional sport, and is thus predictably operating at a higher level than Starcraft in the United States. This sort of consideration doesn’t just apply to sports—cryptocurrency trading operated at an obviously much lower level of play than normal finance for a long time (though that may be changing).
A similar concept can be applied to general adequacy—one can quite usefully analyze existing organizations or programs by asking “what’s the level of play here?” There are three basic questions that I think are quite useful for analyzing what level something is on:
Seriousness. Do people take this seriously? There are many areas that nobody really tries very hard at, and those areas usually operate at a low level of play—why bother to do it right if you don’t really care that much? In sports, think about the difference between a niche sport like jai alai and something like soccer.
Competitiveness. Do different groups compete to do better? If so, how close is the competition? There are some fields that are taken very seriously but are nevertheless not very competitive—medicine, for instance—and this tends to yield lower-level outcomes overall, since there is less incentive to integrate new advances into the system.
Aliveness. Are the conclusions being tested in actual practice? If not, are they being tested under realistic training conditions? It may surprise you to learn that there are matters that are taken seriously and competitively and yet not actually tested, but this is the state of the world! The biggest example of this I can think of would, surprisingly enough, be military strategy. History is rife with examples of situations where war planners, despite being very serious, well-funded and in direct competition (sometimes to the point of arms races), nevertheless made grevious errors—errors which led to the deaths of thousands. One of the main reasons such drastically wrong decisions can be made is that it is very difficult to actually test various military strategies under realistic conditions, and so false theories easily propagate in the absence of actual wars.
Asking these questions is a great quick way to evaluate “level of play” and hence adequacy. For instance, to take three quick examples of my evaluations:
The level of play in investment banking is quite high. People take it quite seriously, it’s very competitive, and people’s conclusions are tested all the time.
The level of play in Magic: the Gathering is lower than that but still pretty high—people take it quite seriously (as games go), it’s very competitive, and people’s conclusions are tested all the time.
The level of play in asteroid impact avoidance, on the other hand, is not very high. While this is certainly a serious affair in one sense, it isn’t highly funded, it isn’t very competitive, and it’s hard to do tests.
I find this method and framing quick and easy to use—when I frame things in terms of “adequacy” it seems at times a little distanced from the case, but when I frame them in terms of levels of play they end up much easier for me to work with.
Try this out and see if it works for you!
I completely agree with seriousness and aliveness, but think competitiveness is only applicable in extremely narrow, well-defined circumstances like some games, and that these circumstances aren’t present in the real world. Sports is an edge case, since the boundaries are artificial, but not as abstract as the rules of, say, chess, and so have real-world gray area which is exploitable.
So I would argue that, most of the time, competitiveness leads to a much, much lower level of play in individuals, not higher. I see several routes to this:
Goodhart’s Law can attack, and people can start trying to game the metric and play the system to “win” at the expense of making real hard-to-measure progress, and the field suffers greatly.
People under pressure perform much worse on the candle problem. Offering cash prizes or deadline threats can only motivate well-defined, mechanical thinking, except by drawing attention to the desired problems.
#2 seems to be the bigger contributor, from my perspective.
(As a point of clarrification: I fully agree that if you make sports a multi-billion dollar industry, you will attract the interest of enough people capable of ignoring their incentives and just exploring the problem philosophically. However, my point is that for an individual already focused on a particular problem, a deadline or cash prize is the last thing they need. It had the distracting effect of focusing the mind. (Also, more generally, these sorts of phenomenon are known as the overjustification effect, which is why any system built on incentives alone is doomed to failure.) )
This (#2) is likely to sound highly surprising, so let me expand the model a bit: The actual frame of mind you want to be in to make real, novel breakthroughs is one of detached, unhurried curiosity. The world is usually way to eager for tangible, measurable results, and so is way way way to far toward the “exploit” side of the “explore/exploit” trade-off (formally, the multi-armed bandit problem). Goodhart’s Law strikes again.
Examples (of #2) to try and triangulate meaning:
If you actually wanted to win at a competitive sport, the way to do it is NOT to practice or compete. Everyone’s already doing that, so it’s an efficient market. The way to win is to look for loopholes which haven’t been exploited yet.
Example 1: Moneyball fundamentally changed the way people thought about sports, by playing a fundamentally different game with a different goal (optimizing for return on investment, not winning/prestige/warm fuzzies).
So, of the top of my head, you might take a close look at which sorts of transhumanist enhancements might not be illegal yet. Can you stretch leg bones, create new drugs that aren’t banned yet, (example 2) or recruit from untapped demographics which have some unique trait? Lets explore that avenue of the transhumanist approach.
I happen to randomly recall that ADHD meds stunt growth, so that might help make smaller gymnasts who are still old enough to participate in the Olympics. (Example 3) (I also happen to vaguely recall that China lied about the age of their gymnasts to get yunger into the Olympics, because smaller is apparently better for gymnastics.) So, since no one is going to ban a drug which a kid legitimately needs for non-sports reasons, ADHD meds sound promising. There are presumably health and moral reasons why kids are often taken off of ADHD drugs for the summer, to allow them to grow, but if you’re a true creative Machiavellian focused solely on winning, you could probably put them in a school system without a summer break or something.
Meta discussion of these examples:
Note that all we’ve done here is randomly mash up transhumanism + sports, and see what relevant knowledge our brains already have which might be useful. Having 2 things I randomly recall suggested one approach, but we’ve hardly scratched the surface of the search space. In order to generate a few thousand new, better approaches, we might read more about extreme features (height/weight/body proportions/endurance/hormones/etc.) or different athletes. (I happen to also randomly recall that some tour de france champoin had testicular cancer and ridiculously high testosterone levels or something. Similarly, the swimmer Michael Phelps is bizarrely proportioned, with short legs and long arms.) Combining that with looking up whether it might be possible to induce such features should be fertile ground for ideas.
But, even that approach is extremely narrow. We could broaden it to looking for the key limiting factors in various sports, physical or not, and then researching how to overcome them. Or better yet, spend a lot of time thinking about which search spaces are excluded even by that broader methodology, and then search in them. The entire point is not to get sucked into one concrete path until you are reasonably sure you’ve exhausted all the search space reachable by human-like minds, and are at least starting you implementation in the right ballpark. You should still expect to pivot to different sports or approaches though, even halfway through implementation. Discard sunk costs quickly.
Back to gesturing at #2:
Some of this sort of creative problem solving may occur in sports from time to time. I don’t follow sportsball close enough to guess exactly how inadequate/low-level-of-play sports is generally. But, the mere fact that people are bothering to waste time playing the object-level game, rather than concentrating all their efforts on the meta-game, isn’t a sign of competence. That’s probably for the best though, since any competent arms-race in just about anything is just a paperclip maximizer built out of human institutions instead of AI.
If there’s something important that you actually want to make substantive contributions too though, then the trick is to figure out how to revolutionize the entire field, rather than wasting time making incremental improvements. That means thinking outside the box, and outside the “outside the box” box. Douglas Hofstadter calls this JOOTSing, for “Jumping Out Of The System”. This is entirely distinct from merely repeating the innovation of a previous jump.
Saying “…so I can clearly not choose the wine in front of [me/you]...” over and over again isn’t actually going any additional meta-levals deep, which is why Yudkowsky’s Law of Ultrafinite Recursion applies to such situations. To win such a battle of wits, you have to completely upend the previous level of play, such as by building up an immunity to iocaine powder.
And such creative solutions require precisely the opposite mindset from the hyper-focused blinders which narrow attention to nothing but the object-level task at hand, to the exclusion of everything else. The odds of the ideal solution actually lying in such a tiny search space are infinitesimal.
Excellent comment! I don’t agree completely but I appreciate your taking the time to engage in such detail.
I think that concentrating efforts on the meta-game can be useful, and as you say can be one of the best ways to deal with situations that are currently extremely competitive. That being said, to me this seems almost evidence for competitiveness as a relevant aspect; these sorts of approaches become necessary only when applying standard methods won’t work because the level of competition is already high.
In many fields, you can win by just showing up and “doing it right”—only in fields with serious competition, where you can reasonably expect that people have reached the limits of the current approaches, is the meta approach really necessary. That said, as you point out you often need to take a new approach in order to do something really revolutionary, even if it’s easy to beat the current competitors in the field...
Just because this is a rare chance to share some jai alai knowledge, the sport is actually very hard to be good at in the “you’re good and win more often” sense because the way the ball bounces makes the outcomes more a matter of luck than skill. Yes, there is some skill involved, but the skill cap is relatively low beyond which being better at the game doesn’t help you win more because it can’t overcome the entropy introduced by the unpredictability of the ball. This is related to why it’s primarily a sport that, when played competitively, is primarily of interest for paramutual betting: it’s just a source of random noise you can use to try to exploit people’s misunderstandings of how paramutual betting works. It’s not interesting otherwise because in leagues skill doesn’t seem to help teams win consistently enough for there to be much point in playing.
Sources: I lived in Florida and knew people who knew way more about jai alai than me.
Huh, interesting. I should maybe edit this post with an alternate example, but I think the point more or less stands.
Agreed, I don’t think an edit is necessary.
Related question it’s critical to ask: Is the game I’m interested in the game that’s being played?
Amateur Jai Alai is probably full of people who love Jai Alai and are genuinely trying. On the other hand, Facebook isn’t trying very hard to make a social network that’s good for you—it’s trying to make a social network that consumes your attention, and is very good at that game, as long as you don’t take any strategic- or tactical-level actions against it. Likewise, equity prices are efficient at something such that dilettantes get burned, but they’re not necessarily efficient at estimating underlying fundamental value of businesses fairly. (For instance, a stock could be undervalued relative to its potential in part because investors correctly believe there’s a stable equilibrium of collusion to undervalue it, which would starve the company of capital needed to actualize its potential.)
Presumably the value of thinking in terms of levels of play for you is in drawing on your experience with various games being played at various levels of play; it might be good to be more explicit about this since presumably not everybody has such experience.
Competitiveness and aliveness seem clearly good and useful to track to me. I’m less sold on seriousness; I’m worried that it’s susceptible to problems analogous to problems applying the absurdity heuristic, e.g. if you don’t play video games much you might not think people take video games seriously and you would be very, very wrong about that.
Related but maybe easier to get a sense of is how strong the incentives are to try hard, e.g. do people get paid a lot for doing this, do they accrue a lot of status, that kind of thing (although this heuristic still seems like it risks coming to the wrong conclusion about video games, and also it mostly predicts Goodharting rather than getting good at the thing per se).
I think the problem with not recognizing speedrunning is less the absurdity heuristic and more just having the wrong information? There are many things that are not “respectable” or “mainstream” but are taken very seriously (many video games, fanfiction, editing certain Wikipedia pages, etc. etc.). My sense is that it’s quite possible for an outsider to assess this by checking out fan communities and the like.
Another distinction worth making: there are games that are being played for their own sake, but only within their legitimate domain, and then there are games being played in ways that trade off success in other games people ordinarily play. Financial fraud and cheating on tests are instance of the latter, but so are war and professional sports, where athletes/soldiers often do things that wreck their long-run health in order to win a local contest. Bans on performance-enhancing drugs / chemical and biological weapons are attempts to limit this sort of osmotic effect of transferring resources across membranes, without altering the underlying incentive type. By contrast, consider the original Olympics, and Greek city-state warfare, which for the most part (excepting late Sparta) managed to remain domain-specific. This is very different from “not trying hard”—Greek amateur hoplite armies held their own against a massive Persian invasion including a large number of professional soldiers.
Agreed. The modern Olympics used to mandate amateurism, and while this was originally classist in nature, it evolved into a means of preventing other excesses. (That said, amateurism is now gone from the Olympics for every sport other than wrestling). College sports in the NCAA currently have a weird pseudo-amateurism where student-athletes are not supposed to be compensated for their skills so as to prevent undue focus on athletics, but perks and benefits (of varying degrees of subtlety) are very obviously still present.