Dan walks through some extremely clear-cut sports examples where people continued making decisions that were wrong costing them wins in ways that were easily calculable. They did eventually change, but even that took years, sometimes decades. Given that this was happening in something as easy to measure as sports, he suggests we should expect bad decision making in corporations or governments to take even longer to correct.
My major concern with this post is that the goal of pro sports is a mix of winning and making money via attention from fans. This first means that the feedback loop isn’t as tight as it would be otherwise. Additionally, many of the optimizations that lead to more wins make games more boring, which ultimately costs the entire league money. Using these optimizations is in some ways defecting in a prisoner’s dilemma: you win more, which your fans like and other fans don’t, but if everyone does it the game becomes more boring, which drives away everyone’s fans. Under this light, base stealing attempts that cost you counterfactual wins could be viewed as a prosocial sacrifice to the entire league’s benefit.
This is true of some but not all optimizations. NFL teams punt too often on 4th down, and punting is boring; (in a large set of cases where teams have conventionally decided to punt) keeping your offense on the field to run a play increases your chances of winning and also makes the game more interesting for fans. (Teams have gradually been getting better at these decisions, over the years.)
Another complication is that various people judge the coach (or team) based on process and not just on results, using their own views about which process is best. So there’s a cost to making a decision that other people consider to be a bad decision, even if it maximizes your team’s chances of winning.
If the fans think the coach made a bad decision, they might like the team a bit less, spend less money on the team, or want the coach to be fired.
If the players think the coach made a bad decision, they might be a bit less on board with the what the team is doing or less eager to sign a contract with the team.
If the owner/GM thinks that the coach made a bad decision, or that the fans or players don’t support the coach as much, they might be a bit more likely to fire the coach.
So if we start in a situation where the fans, players, owner/GM, and coach all believe the conventional wisdom about what decisions are good ones, then the coach doesn’t necessarily have much incentive to search for unconventional approaches which are widely seen as bad ideas but actually increase the team’s chances of winning.
I also wonder — the effect size of forbidding base-stealing would be small, <1%. Maybe the cost to player morale of forbidding an interesting part of the game is worse than the benefit. It’s unclear and I wouldn’t bet on it. But I think it does fit in with the view I have, that he’s arguing against, which is that it’s really hard to show that any individual agent is acting irrationally.