I feel like ‘efficiency’ is often scowled at. It is associated with factories and killing and commercialization, and people who are no fun. Things are openly criticized for being oriented toward efficiency. Nobody hopes to give their children an efficient childhood or asks for an efficient Valentine’s day, unless they want to get it over with. I expect wariness in listeners at talk of efficient charity.
This intrigues me, because in what I take to be its explicit definition, ‘efficiency’ is almost the definition of goodness manifest. The efficiency of a process is the rate with which it turns what you have into what you want.
I usually wince when people criticize efficiency, and think they are confused and should be criticizing the goal that is being pursued efficiently. Which does seem basically always true. For instance, if they are saying their childcare center cares only for efficiency, they probably mean that it is doing something like trying to minimize financial costs without breaking the law. Perhaps by fitting many children into a room with minimal oversight or attention to thriving. Here, I would complain that the childcare center cares only about its profits and not breaking the law. If it was fulfilling my own values efficiently, that would be awesome.
However I think there is more merit to efficiency’s poor reputation than I have given credit for. Because pursuing efficiency does seem to systematically lead to leaving things out. Which I suppose perhaps makes sense, for creatures who don’t explicitly know what their values are, and especially who have trouble quantifying them. If you set out to build an efficient daycare center, chances are that you don’t know exactly what makes a daycare center good, and are even less well equipped to put these things into numbers and build machinery to optimize those numbers. (This would be much like the AI alignment problem but where the AI you are trying to direct is made of your own explicit reasoning. It might also what Seeing Like a State is about, but I haven’t read it.) It’s still not clear to me why this would systematically turn out actively worse than if you didn’t aim for efficiency, or whether it does (my guess is that it usually doesn’t, but sometimes does, and is notable on those occasions). If efficiency has really earned its poor reputation, I wonder if I should be more worried about this.
What is efficient on one side of the coin is 0 slack on the other.
Aspects which are hard to put into numbers get short end of the stick if numbers try to go up.
If someone optimises for some measures that are proxies for success the quality of the proxiness of those numbers goes down. An airplane that uses many instruments is likely to be safer than one that uses less instruments. For a complex system a good solution is unlikely to be optimal if the dimensionality of the problem was artifically lowered. So exact solutions can be warning signs that maybe important aspects were assumed away to make the problem overtly tractable.
Yes, this feels related to slack. No-one has really figured out how to get both slack and efficiency.
They feel like opposites and I it strikes me as strange to think they can be achieved at the same time.
This probably roots on how I tried to understand slack in terms of the magic the gathering colors terms as a very green property. Evolution can either be radiative or convergent. You can either have natural selection or natural expression. In a cuthtroat world only the very best survive and the smallest mistake gets you killed. In a world of abundance flexibility lets you go into niches you didn’t know you could thrive in.
With color wheel one could think efficiency as a black concept ie which costs are worth taking (Can I and do I want to cut off my arm in order to get my objectives?) or blue concept (What is the easiest and minimum amount of effort to get me to my objective?). A slack green would oppose as enemy color to black that having an arm is big chunk of flexibility just thrown away, it would make sense to lose this battle in order to go into future battles with both arms. It is okay to carry around limbs that are not crucial for the issue at hand. As opposition to blue, achieving only your goals mean you miss out on growth. Doing one thing good means you are only going to do one thing in your life. Being vibrant and adaptative lets you grab the sunshine where it happens to be rather where you are looking for it.
Efficiency enables more slack in a well functioning priority stack.
How often is the world actually a well functioning priority stack?
the world doesn’t have a priority stack, people do.
I haven’t read Zvi’s post, but would have thought that the good of slack can be cashed out in efficiency, if you are optimizing for the right goals (e.g. if you have a bunch of tasks in life which contribute to various things, it will turn out that you contribute to those things better overall if you have spare time between the tasks).
If you aren’t in the business of optimizing for the ultimately right goals though, I’d think you could also include slack as one of your instrumental goals, and thus mostly avoid serious conflict e.g. instead of turning out as many cookies per minute as I can, aim to turn out as many cookies as I can while spending half my time not working on it, and setting aside a bag of each ingredient. Perhaps the thought is that this doesn’t work because ‘slack’ is hard to specify, so if you just say that I have to be away from the cookie cutters, I might spend my time strategizing about cookies instead, and that might be somehow not slack in the right way? Plus part of the point is that if things go awry, you want me to be able to put all my time back into cookies briefly?
There is probably a process to burn slack to get efficiency and to use efficiency to create slack. I am somewhat skeptical that just having spare time would make overall time spent less. The way I would imagine a slack approach working out for greater efficiency would be like having a drinking pause and conversing with a buddy who gives a tip about the cookie making that makes it go more smoothly.
I would also think that instead of using only half the time a slack approach would be just bake cookies to achieve mastery better by being deliberate and slow instead of setting tighter and higher bars. Like pausing to wonder about the philosophy of baking.
I have also this analog about anticipating things go wrong. In a military setting having a reseve can be used to address when a default operation goes wrong. A optimization focused mind might think that they need to assign the minimum amount of soldiers to get each task done so that reserve is as big as it can get so it can more forcefullly address problems. But overassigning soldiers to tasks makes each of them less likely to fall. So being slack about it could mean that you want a reserve so that there is flexibility if main plan goes awry but you want the main plan to be flexible enough so it doesn’t brittle immidietly on the first hiccups.
If it was important that the cookies are made (big party or something) I would probably do them slowly rather than doing them quick and then idling. For big things the difference of being able to withstand 0,1 or 2 catasrophes is pretty big. If you do it quick and have some probability of having to do it from scratch again there is some probablity of spending double time on it. So one approach of increasing the expectancy of success would be to buff up the reliability rather than the number of shots. If you can aim that one bullseye it doesn’t matter how many arrows per second you can shoot at the target. One could think this as bullseyes per arrow in eficiency terms. But one could also think about all the factors taken into account: wind, gravity, breathing rythm, arrow conditions and material, muscle tension, sunglare, followthrought, degree of pull on the string. If any of the factors would have been different would you have known to adjust for it? Slack is the degree of disruption permitted while maintaining goals, the multitude of ways things could have unfolded.
The tails come apart. If you aim for extremes, you wind up selecting against other forms of goodness. It’s not robust. This is bad both for your goals, if you are incorrect about your goals at all, and socially bad, because it moves you off of (?)cooperative ground(?) where seeking your values correlates with seeking theirs.
A thoughtful discussion of efficiency. Points for metacognition: “If efficiency has really earned its poor reputation, I wonder if I should be more worried about this.”
This discussion seems (to me) to conflate a number of issues that it seems to me should be separated, in the manner of a scalar getting substituted for a vector, or, even better, a dynamic vector field (i.e., tensor).
Issues conflated seem to me (I could be wrong) to include:
[1] Substitution of mathematical optimization for human values—the difference twixt quantity and quality. Is this an issue we ought to worry about when we discuss efficiency? If so, how should we tackle it?
[2] Difference twixt making purely mechanical or mathematical or natural processes more efficient, and making social processes more efficient.
[3] The potential incommensurability of applying a mathematical optimization method like an optimal point or optimal set of values (usually obtained in the calculus of variations by setting the derivative to zero, or in a set of simultaneous linear equations by solving them all at once (e.g., Kantorovich’s optimization program for the Soviet economy in the 1950s-1960s) to human activities.
[4] The proven mathematical problems with applying optima and efficiency to ranking problems and other examples of human choice.
Let me try to give specific examples of these issues.
1 - Substitution of mathematical optimization for human values—rape is more efficient than marriage. Slavery is with torture as motivation is more efficient than workplace bonuses. The maximally efficient solution to global warming includes killing roughly 6 billion of the world’s people. And so on. You can immediately see the problem here. Imagine a hypothetical superhuman AI given the task of eliminating crime in a large city. “Easy,” the AI says, “kill everyone.” So you respond, OK, but we want another solution. “Fine,” the AI retorts, “just put everyone in the city in jail.” OK, you say, but that’s not gonna work either, we need another solution. “Sure, just sedate everyone in the city 24/7/365.” And it spirals downhill from there.
The problem here is that every problem in the real world has invisible constraints which involve unspoken social values. Moreover, different cultures solve the same problems in very different ways depending on their social values. For instance, in America we try to minimize out-of-wedlock births by distributing birth control and increasing education and making abortion expensive and difficult to obtain. Japan, however, minimizes out-of-wedlock births by socially stigmatizing it. As a result, Japan has a much lower out-of-wedlock birth rate than America, but this solution (which very efficient for Japan) does not carry over to America because of the difference in our cultures.
2 - “This intrigues me, because in what I take to be its explicit definition, ‘efficiency’ is almost the definition of goodness manifest. The efficiency of a process is the rate with which it turns what you have into what you want.”
For a purely mechanical or mathematical or natural process, this definition works. For example, we can numerically measure the efficiency of a Carnot engine very well. In part this is because purely mechanical or mathematical or natural efficiency can be unambiguously defined, and in part it’s because the system isn’t reflexive and typically doesn’t change its behavior when being measured/observed by humans.
The Keynesian beauty contest exemplifies the problem with reflexivity in human systems:
“It is not a case of choosing those [faces] that, to the best of one’s judgment, are really the prettiest, nor even those that average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practice the fourth, fifth and higher degrees.” (Keynes, General Theory of Employment, Interest and Money, 1936).
As Nassim Taleb has noted, game theory fails when predictions become warnings which change the participants’ behavior. “Fooled by Randomness,” Taleb, N., 2008.
For human systems, this definition of efficiency also seems to fail because different human observers will arrive at different numerical measures for efficiency depending on their value systems, and in part because of Goodhart’s Law, which tells us that “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Also Campbell’s Law, which states that “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
Examples here involve defining efficiency of, say, the economy—does efficiency mean maximizing GDP? Or maximizing individual happiness? Or minimizing global warming? Or producing the greatest good for the greatest number of people? There’s no one obvious “right” answer here, which creates problems. The classic example of Goodhart’s Law is the Soviet nail factory that demanded maximum unit production, so the workers responded by manufacturing millions of tiny 1-millimeter-long nails per day. The factory then changed to demanding maximum weight of the individual unit, so the factor workers responded by producing a handful of useless 500-kilogram nails per day.
https://en.wikipedia.org/wiki/Goodhart%27s_law
https://en.wikipedia.org/wiki/Campbell%27s_law
3 - Maximizing efficiency means optimizing some value or set of values, and this proves highly dependent on our ability to generate objective numbers from processes or systems. With non-human processes or systems, like an engine or a chemical reaction, we have well-defined and reliable ways of doing this: enthalpy, Carnot efficiency, mechanical static and dynamic friction, and so on. With human processes, applying the methods of science and mathematics becomes extremely dubious. Science depends on objectively measuring quantities in a reproducible way. Social systems tend to resist objective measurement and also tend not to be reproducible, when we’re talking about large-scale social processes like the French Revolution or the Industrial Revolution or the American Civil War, etc. Was the American Civil War the most efficient way of ending U.S. slavery? Was the French Revolution the most efficient way of redistributing income in 18th century France? These judgments depends not only on human values, but on counterfactuals that involve guesses, since we can’t re-run history via a time machine. This would seem to place a hard limit on our ability to measure, let alone define, efficiency in these contexts.
4 - “I usually wince when people criticize efficiency, and think they are confused and should be criticizing the goal that is being pursued efficiently. Which does seem basically always true.”
Maximizing efficiency has in many cases been shown to be mathematical impossible. For example, the Sonnenschein-Mantel-Debreu Theorem tells us”the excess demand curve for a market populated with utility-maximizing rational agents can take the shape of any function that is continuous, has homogeneity degree zero, and is in accordance with Walras’s law.[5] This implies that market processes will not necessarily reach a unique and stable equilibrium point.” If the system has no unique stable equilibrium point, the search for an optimal equilibrium, viz., Pareto optimum, would appear to be futile.
https://en.wikipedia.org/wiki/Sonnenschein%E2%80%93Mantel%E2%80%93Debreu_theorem
This tends to explain why game theory is junk science and why the Nash Equilibrium seldom shows up in the real world.
https://www.businessinsider.com/prisoners-dilemma-in-real-life-2013-7Secy.
All tests of the Prisoner’s Dilemma with real-world subject shows that the overwhelming majority of people prefer to cooperate rather than defect, going back to the original RAND test using RAND secretaries, which failed to confirm the conclusions of John Nash’s original paper. As neuroeconomics researchers have noted, “The only subjects who react as Nash predicted are sociopaths and economists.” (The paper is “Neuroeconomics,” but I can’t immediately find a link to it right now.)
“While it is common to report that experimental subjects do not make choices that comport with Nash equilibrium strategies (or even von Neumann-Morgenstern utility maximization), we should not infert hat human reasoning is thus somehow flawed. It is perhaps the case that ourexisting, deductive,modelsof human reasoning are too limited. Humans are able to solve many tasks that are quite difficult.”
http://mccubbins.us/mccubbins_files/icai2012.pdf
Efficiency measures like Pareto optimiality and the Nash equilibrium use a definition of efficiency that’s not only far too limited to account for human reason, but which fails at a basic mathematical level. Viz., Arrow’s Impossibilty Theorem:
https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem
Another essential problem with applying efficiency to any human endeavour is that data is always to some extend theory-driven, and thus our data always depends on the theoretical framework we use to decide which data to measure and how. We cannot measure everything, so in any experiment we must choose which variables to measure and which to exclude from consideration. This leads to quagmires like Secy. of Defense Robert McNamara’s incorrect use of body counts to measure the progress of the Viet Nam war:
https://chronotopeblog.com/2015/04/04/the-mcnamara-fallacy-and-the-problem-with-numbers-in-education/
https://www.technologyreview.com/2013/05/31/178263/the-dictatorship-of-data/
This is a problem which besets the entire rationality community, in my opinion. Mathematician Cathy O’Neil has written eloquently about it in her book Weapons of Math Destruction:
https://www.amazon.com/Weapons-Math-Destruction-Increases-Inequality/dp/0553418815
In the real world, most economic/social issues to which the mathematical tools of efficiency get applied tend to be so-called wicked problems. “In planning and policy, a wicked problem is a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. It refers to an idea or problem that cannot be fixed, where there is no single solution to the problem; and “wicked” denotes resistance to resolution, rather than evil.[1] Another definition is “a problem whose social complexity means that it has no determinable stopping point.”
“Wicked Problems”. Management Science. 14 (4): B-141–B-146, C. West Churchman, 1967.
Churchman defined wicked problems as having the following characteristics:
There is no definitive formulation of a wicked problem.
Wicked problems have no stopping rule.
Solutions to wicked problems are not true-or-false, but better or worse.
There is no immediate and no ultimate test of a solution to a wicked problem.
Every solution to a wicked problem is a “one-shot operation”; because there is no opportunity to learn by trial and error, every attempt counts significantly.
Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.
Every wicked problem is essentially unique.
Every wicked problem can be considered to be a symptom of another problem.
The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem’s resolution.
The social planner has no right to be wrong (i.e., planners are liable for the consequences of the actions they generate).
Purely mathematical/scientific measurements and definitions of efficiency appear to become extremely problematic in these situations (speaking for myself).
Goodhart effects are robust and it takes very little optimization pressure for them to show up.
It might be instructive to consider some specific examples. An internal combustion engine is particularly easy to analyze, for instance. The efficiency with which fuel is turned to kinetic energy is almost entirely determined by temperature gradient between the engine and its surroundings. This means that, apart from better cooling technologies, the only way to make an engine more efficient is to make it less powerful. In practice, we also make entire vehicles more fuel efficient by making them lighter and more aerodynamic.
But the “goal” of an engine in many cases is just to produce the most power. If you’re trying to win a race, get into orbit, you’re limited by how much thrust or torque you can generate per unit of mass and time. This goal is directly antithetical to efficiency.
While I don’t think this generalizes to the extent that efficiency as a goal is always antithetical to whatever you’re really trying to optimize, it is at least in most cases orthogonal to whatever you’re actually trying to optimize.
My friends circle has less universal distaste for the idea of efficiency, and it seems to be quite contextual in popular media (“If you can be one thing, be efficient.”—Wayne, Letterkenny).
I think it’s not popular in far-mode discussions because it’s explicitly about tradeoffs. Acknowledging that there are competing desires kind of sucks when you want to signal wholehearted support for whatever harebrained idea your friends or parents are supporting now. It’s extremely popular in near-mode planning—nobody I know argues in favor of waste (OK, that’s not true—we’ll argue any side of any issue, just for drill. But when we’re trying to actually do something, efficiency is accepted as important.)
This definition seems to imply that what you have is not what you want (fair enough, I guess), and that the process is a means to get what you want, and the process itself is also not a part of what you want.
Since life is a process (I do not believe that the one who dies with the most toys wins), efficiency under this definition is trading away a part of life for some end result that you want. (And I think that people tend to object to efficiency in human things, not in purely mechanical things.) May or may not be a good trade, but intuitively, a process that is less efficient in producing the end result but more life-affirming (for lack of a better word) sounds good to me, and I don’t agree that efficiency is definitionally equivalent to goodness. There is a tradeoff.
(On the other hand, if you consider the process as part of the end result, and the process as producing itself, and the efficiency as the rate with which the process produces itself in a form that’s what you want, I suppose you can get around this definitionally, but I find that definition less intuitive and more difficult to work with, as you can probably tell by the awkward way I described it. I don’t think this is usually what is meant by efficiency.)
I think it is a combination of the following things that often happen in the name of “efficiency”:
Goodhart’s law;
optimization for values that are not yours;
premature optimization;
removal of slack.
Note that the typical context when people consider “efficiency” explicitly is when it is forced on them by some authority. (If you want to do something more efficiently for yourself, you would probably just call it a “better” or “smarter” way of doing things.)
It is likely that the authority’s goals are not well aligned with yours, and “efficiency” means taking away part of your money or freedom, so that they can have greater profit or easier job. Sometimes the authority defends their solution as improving things for you, but your potential disagreement would be predictably ignored; they have already made their decision, and the important people have approved it, so the debate is over.
In context of employment, “efficiency” usually means that you will have to use cheaper tools or cheaper ingredients, your work will be monitored to make sure you are not taking breaks or otherwise stealing company time, there will be more micromanagement, possibly some of your colleagues will be fired and their tasks will be distributed among the remaining ones, while everyone’s salary remains constant. (Except for the managers who organized the whole thing; those will get huge bonuses.)
In context of services, “efficiency” means that the provider of service will shift some of their costs on you.
Of course none of this is popular.
I have often remarked “If we had viable third parties in this country, and there was a party that held no policy positions whatsoever but just vowed to make government more efficient, I would vote for them every time”. I feel that this is the forgotten element in so many things. We argue between American health-insurance healthcare and single-payer healthcare, but I believe both systems could be really good if they were run efficiently, and both systems could be horrific abominations if they are run inefficiently.
I really wish politicians would establish clear KPIs and track their progress against those KPIs. Even if the KPIs change from administration to administration, having clear goals and optimizing for them would be a huge benefit.
The pursuit of efficiency often produces myopia. If efficiency is the rate at which we turn “what we have” into “what we want,” then we first need to know what we want.
When processes are “efficient”, they tend to be efficient with respect to a single variable—the thing we purportedly want. Sometimes we want more than one thing, though. When multiple variables are thrown into the mix, tradeoffs and value judgments necessarily come into play.
A daycare might have low cost-per-student (lower cost) or just a few children-per-teacher (better care), but cost and care will be in tension. Which one is more efficient? A supply chain might be very centralized (optimized for cost) or it might be spread across many factories (optimized for resilience). Which one would be the better option?
The pushback against efficiency might be caused by a sense that you can’t always prudently optimize for any one variable. The concept of efficiency starts to lose its sway in situations that require a balance of numerous factors.
Note: This isn’t to say that efficiency is meaningless when optimizing for multiple variables. A low cost daycare with excellent care is obviously more “efficient” in every sense than the crappy, expensive daycare. It’s just that when multiple variables come into play, people want to be “holistic” rather than “efficient” in the sense that they want to consider and weight numerous important variables.
Despite being an EA and valuing efficiency in parts of my life and the world, I share the intimate rejection of efficiency in certain context that you’re describing (and not getting). Like, I don’t want a date with my girlfriend to be efficient. Now, you might answer that it’s because I’m talking about efficiency with regard to time management or money, and not about “having a great time with my girlfriend”. But my point is that “having a great time with my girlfriend” is the kind of objective for which the efficiency framing doesn’t seem appropriate. Indeed it’s not buying me anything and creates a shitload of pitfalls in which to fall (Goodhart, all that).
I would advise you to have a look at philosopher Joseph Heath’s work. He has a book, “The Efficient Society”, where, according to Wikipedia, “[h]e argues that Canada’s successes as a nation are largely attributable to its commitment to efficiency as a value”. In “Morality, Competition, and the Firm” he also discusses the central role of efficiency on ethics (business’ and beyond).