A thoughtful discussion of efficiency. Points for metacognition: “If efficiency has really earned its poor reputation, I wonder if I should be more worried about this.”
This discussion seems (to me) to conflate a number of issues that it seems to me should be separated, in the manner of a scalar getting substituted for a vector, or, even better, a dynamic vector field (i.e., tensor).
Issues conflated seem to me (I could be wrong) to include:
[1] Substitution of mathematical optimization for human values—the difference twixt quantity and quality. Is this an issue we ought to worry about when we discuss efficiency? If so, how should we tackle it?
[2] Difference twixt making purely mechanical or mathematical or natural processes more efficient, and making social processes more efficient.
[3] The potential incommensurability of applying a mathematical optimization method like an optimal point or optimal set of values (usually obtained in the calculus of variations by setting the derivative to zero, or in a set of simultaneous linear equations by solving them all at once (e.g., Kantorovich’s optimization program for the Soviet economy in the 1950s-1960s) to human activities.
[4] The proven mathematical problems with applying optima and efficiency to ranking problems and other examples of human choice.
Let me try to give specific examples of these issues.
1 - Substitution of mathematical optimization for human values—rape is more efficient than marriage. Slavery is with torture as motivation is more efficient than workplace bonuses. The maximally efficient solution to global warming includes killing roughly 6 billion of the world’s people. And so on. You can immediately see the problem here. Imagine a hypothetical superhuman AI given the task of eliminating crime in a large city. “Easy,” the AI says, “kill everyone.” So you respond, OK, but we want another solution. “Fine,” the AI retorts, “just put everyone in the city in jail.” OK, you say, but that’s not gonna work either, we need another solution. “Sure, just sedate everyone in the city 24/7/365.” And it spirals downhill from there.
The problem here is that every problem in the real world has invisible constraints which involve unspoken social values. Moreover, different cultures solve the same problems in very different ways depending on their social values. For instance, in America we try to minimize out-of-wedlock births by distributing birth control and increasing education and making abortion expensive and difficult to obtain. Japan, however, minimizes out-of-wedlock births by socially stigmatizing it. As a result, Japan has a much lower out-of-wedlock birth rate than America, but this solution (which very efficient for Japan) does not carry over to America because of the difference in our cultures.
2 - “This intrigues me, because in what I take to be its explicit definition, ‘efficiency’ is almost the definition of goodness manifest. The efficiency of a process is the rate with which it turns what you have into what you want.”
For a purely mechanical or mathematical or natural process, this definition works. For example, we can numerically measure the efficiency of a Carnot engine very well. In part this is because purely mechanical or mathematical or natural efficiency can be unambiguously defined, and in part it’s because the system isn’t reflexive and typically doesn’t change its behavior when being measured/observed by humans.
The Keynesian beauty contest exemplifies the problem with reflexivity in human systems:
“It is not a case of choosing those [faces] that, to the best of one’s judgment, are really the prettiest, nor even those that average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practice the fourth, fifth and higher degrees.” (Keynes, General Theory of Employment, Interest and Money, 1936).
As Nassim Taleb has noted, game theory fails when predictions become warnings which change the participants’ behavior. “Fooled by Randomness,” Taleb, N., 2008.
For human systems, this definition of efficiency also seems to fail because different human observers will arrive at different numerical measures for efficiency depending on their value systems, and in part because of Goodhart’s Law, which tells us that “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Also Campbell’s Law, which states that “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
Examples here involve defining efficiency of, say, the economy—does efficiency mean maximizing GDP? Or maximizing individual happiness? Or minimizing global warming? Or producing the greatest good for the greatest number of people? There’s no one obvious “right” answer here, which creates problems. The classic example of Goodhart’s Law is the Soviet nail factory that demanded maximum unit production, so the workers responded by manufacturing millions of tiny 1-millimeter-long nails per day. The factory then changed to demanding maximum weight of the individual unit, so the factor workers responded by producing a handful of useless 500-kilogram nails per day.
3 - Maximizing efficiency means optimizing some value or set of values, and this proves highly dependent on our ability to generate objective numbers from processes or systems. With non-human processes or systems, like an engine or a chemical reaction, we have well-defined and reliable ways of doing this: enthalpy, Carnot efficiency, mechanical static and dynamic friction, and so on. With human processes, applying the methods of science and mathematics becomes extremely dubious. Science depends on objectively measuring quantities in a reproducible way. Social systems tend to resist objective measurement and also tend not to be reproducible, when we’re talking about large-scale social processes like the French Revolution or the Industrial Revolution or the American Civil War, etc. Was the American Civil War the most efficient way of ending U.S. slavery? Was the French Revolution the most efficient way of redistributing income in 18th century France? These judgments depends not only on human values, but on counterfactuals that involve guesses, since we can’t re-run history via a time machine. This would seem to place a hard limit on our ability to measure, let alone define, efficiency in these contexts.
4 - “I usually wince when people criticize efficiency, and think they are confused and should be criticizing the goal that is being pursued efficiently. Which does seem basically always true.”
Maximizing efficiency has in many cases been shown to be mathematical impossible. For example, the Sonnenschein-Mantel-Debreu Theorem tells us”the excess demand curve for a market populated with utility-maximizingrational agents can take the shape of any function that is continuous, has homogeneity degree zero, and is in accordance with Walras’s law.[5] This implies that market processes will not necessarily reach a unique and stable equilibrium point.” If the system has no unique stable equilibrium point, the search for an optimal equilibrium, viz., Pareto optimum, would appear to be futile.
All tests of the Prisoner’s Dilemma with real-world subject shows that the overwhelming majority of people prefer to cooperate rather than defect, going back to the original RAND test using RAND secretaries, which failed to confirm the conclusions of John Nash’s original paper. As neuroeconomics researchers have noted, “The only subjects who react as Nash predicted are sociopaths and economists.” (The paper is “Neuroeconomics,” but I can’t immediately find a link to it right now.)
“While it is common to report that experimental subjects do not make choices that comport with Nash equilibrium strategies (or even von Neumann-Morgenstern utility maximization), we should not infert hat human reasoning is thus somehow flawed. It is perhaps the case that ourexisting, deductive,modelsof human reasoning are too limited. Humans are able to solve many tasks that are quite difficult.”
Efficiency measures like Pareto optimiality and the Nash equilibrium use a definition of efficiency that’s not only far too limited to account for human reason, but which fails at a basic mathematical level. Viz., Arrow’s Impossibilty Theorem:
Another essential problem with applying efficiency to any human endeavour is that data is always to some extend theory-driven, and thus our data always depends on the theoretical framework we use to decide which data to measure and how. We cannot measure everything, so in any experiment we must choose which variables to measure and which to exclude from consideration. This leads to quagmires like Secy. of Defense Robert McNamara’s incorrect use of body counts to measure the progress of the Viet Nam war:
This is a problem which besets the entire rationality community, in my opinion. Mathematician Cathy O’Neil has written eloquently about it in her book Weapons of Math Destruction:
In the real world, most economic/social issues to which the mathematical tools of efficiency get applied tend to be so-called wicked problems. “In planning and policy, a wicked problem is a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. It refers to an idea or problem that cannot be fixed, where there is no single solution to the problem; and “wicked” denotes resistance to resolution, rather than evil.[1] Another definition is “a problem whose social complexity means that it has no determinable stopping point.”
“Wicked Problems”. Management Science. 14 (4): B-141–B-146, C. West Churchman, 1967.
Churchman defined wicked problems as having the following characteristics:
There is no definitive formulation of a wicked problem.
Solutions to wicked problems are not true-or-false, but better or worse.
There is no immediate and no ultimate test of a solution to a wicked problem.
Every solution to a wicked problem is a “one-shot operation”; because there is no opportunity to learn by trial and error, every attempt counts significantly.
Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.
Every wicked problem is essentially unique.
Every wicked problem can be considered to be a symptom of another problem.
The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem’s resolution.
The social planner has no right to be wrong (i.e., planners are liable for the consequences of the actions they generate).
Purely mathematical/scientific measurements and definitions of efficiency appear to become extremely problematic in these situations (speaking for myself).
A thoughtful discussion of efficiency. Points for metacognition: “If efficiency has really earned its poor reputation, I wonder if I should be more worried about this.”
This discussion seems (to me) to conflate a number of issues that it seems to me should be separated, in the manner of a scalar getting substituted for a vector, or, even better, a dynamic vector field (i.e., tensor).
Issues conflated seem to me (I could be wrong) to include:
[1] Substitution of mathematical optimization for human values—the difference twixt quantity and quality. Is this an issue we ought to worry about when we discuss efficiency? If so, how should we tackle it?
[2] Difference twixt making purely mechanical or mathematical or natural processes more efficient, and making social processes more efficient.
[3] The potential incommensurability of applying a mathematical optimization method like an optimal point or optimal set of values (usually obtained in the calculus of variations by setting the derivative to zero, or in a set of simultaneous linear equations by solving them all at once (e.g., Kantorovich’s optimization program for the Soviet economy in the 1950s-1960s) to human activities.
[4] The proven mathematical problems with applying optima and efficiency to ranking problems and other examples of human choice.
Let me try to give specific examples of these issues.
1 - Substitution of mathematical optimization for human values—rape is more efficient than marriage. Slavery is with torture as motivation is more efficient than workplace bonuses. The maximally efficient solution to global warming includes killing roughly 6 billion of the world’s people. And so on. You can immediately see the problem here. Imagine a hypothetical superhuman AI given the task of eliminating crime in a large city. “Easy,” the AI says, “kill everyone.” So you respond, OK, but we want another solution. “Fine,” the AI retorts, “just put everyone in the city in jail.” OK, you say, but that’s not gonna work either, we need another solution. “Sure, just sedate everyone in the city 24/7/365.” And it spirals downhill from there.
The problem here is that every problem in the real world has invisible constraints which involve unspoken social values. Moreover, different cultures solve the same problems in very different ways depending on their social values. For instance, in America we try to minimize out-of-wedlock births by distributing birth control and increasing education and making abortion expensive and difficult to obtain. Japan, however, minimizes out-of-wedlock births by socially stigmatizing it. As a result, Japan has a much lower out-of-wedlock birth rate than America, but this solution (which very efficient for Japan) does not carry over to America because of the difference in our cultures.
2 - “This intrigues me, because in what I take to be its explicit definition, ‘efficiency’ is almost the definition of goodness manifest. The efficiency of a process is the rate with which it turns what you have into what you want.”
For a purely mechanical or mathematical or natural process, this definition works. For example, we can numerically measure the efficiency of a Carnot engine very well. In part this is because purely mechanical or mathematical or natural efficiency can be unambiguously defined, and in part it’s because the system isn’t reflexive and typically doesn’t change its behavior when being measured/observed by humans.
The Keynesian beauty contest exemplifies the problem with reflexivity in human systems:
“It is not a case of choosing those [faces] that, to the best of one’s judgment, are really the prettiest, nor even those that average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practice the fourth, fifth and higher degrees.” (Keynes, General Theory of Employment, Interest and Money, 1936).
As Nassim Taleb has noted, game theory fails when predictions become warnings which change the participants’ behavior. “Fooled by Randomness,” Taleb, N., 2008.
For human systems, this definition of efficiency also seems to fail because different human observers will arrive at different numerical measures for efficiency depending on their value systems, and in part because of Goodhart’s Law, which tells us that “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Also Campbell’s Law, which states that “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
Examples here involve defining efficiency of, say, the economy—does efficiency mean maximizing GDP? Or maximizing individual happiness? Or minimizing global warming? Or producing the greatest good for the greatest number of people? There’s no one obvious “right” answer here, which creates problems. The classic example of Goodhart’s Law is the Soviet nail factory that demanded maximum unit production, so the workers responded by manufacturing millions of tiny 1-millimeter-long nails per day. The factory then changed to demanding maximum weight of the individual unit, so the factor workers responded by producing a handful of useless 500-kilogram nails per day.
https://en.wikipedia.org/wiki/Goodhart%27s_law
https://en.wikipedia.org/wiki/Campbell%27s_law
3 - Maximizing efficiency means optimizing some value or set of values, and this proves highly dependent on our ability to generate objective numbers from processes or systems. With non-human processes or systems, like an engine or a chemical reaction, we have well-defined and reliable ways of doing this: enthalpy, Carnot efficiency, mechanical static and dynamic friction, and so on. With human processes, applying the methods of science and mathematics becomes extremely dubious. Science depends on objectively measuring quantities in a reproducible way. Social systems tend to resist objective measurement and also tend not to be reproducible, when we’re talking about large-scale social processes like the French Revolution or the Industrial Revolution or the American Civil War, etc. Was the American Civil War the most efficient way of ending U.S. slavery? Was the French Revolution the most efficient way of redistributing income in 18th century France? These judgments depends not only on human values, but on counterfactuals that involve guesses, since we can’t re-run history via a time machine. This would seem to place a hard limit on our ability to measure, let alone define, efficiency in these contexts.
4 - “I usually wince when people criticize efficiency, and think they are confused and should be criticizing the goal that is being pursued efficiently. Which does seem basically always true.”
Maximizing efficiency has in many cases been shown to be mathematical impossible. For example, the Sonnenschein-Mantel-Debreu Theorem tells us”the excess demand curve for a market populated with utility-maximizing rational agents can take the shape of any function that is continuous, has homogeneity degree zero, and is in accordance with Walras’s law.[5] This implies that market processes will not necessarily reach a unique and stable equilibrium point.” If the system has no unique stable equilibrium point, the search for an optimal equilibrium, viz., Pareto optimum, would appear to be futile.
https://en.wikipedia.org/wiki/Sonnenschein%E2%80%93Mantel%E2%80%93Debreu_theorem
This tends to explain why game theory is junk science and why the Nash Equilibrium seldom shows up in the real world.
https://www.businessinsider.com/prisoners-dilemma-in-real-life-2013-7Secy.
All tests of the Prisoner’s Dilemma with real-world subject shows that the overwhelming majority of people prefer to cooperate rather than defect, going back to the original RAND test using RAND secretaries, which failed to confirm the conclusions of John Nash’s original paper. As neuroeconomics researchers have noted, “The only subjects who react as Nash predicted are sociopaths and economists.” (The paper is “Neuroeconomics,” but I can’t immediately find a link to it right now.)
“While it is common to report that experimental subjects do not make choices that comport with Nash equilibrium strategies (or even von Neumann-Morgenstern utility maximization), we should not infert hat human reasoning is thus somehow flawed. It is perhaps the case that ourexisting, deductive,modelsof human reasoning are too limited. Humans are able to solve many tasks that are quite difficult.”
http://mccubbins.us/mccubbins_files/icai2012.pdf
Efficiency measures like Pareto optimiality and the Nash equilibrium use a definition of efficiency that’s not only far too limited to account for human reason, but which fails at a basic mathematical level. Viz., Arrow’s Impossibilty Theorem:
https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem
Another essential problem with applying efficiency to any human endeavour is that data is always to some extend theory-driven, and thus our data always depends on the theoretical framework we use to decide which data to measure and how. We cannot measure everything, so in any experiment we must choose which variables to measure and which to exclude from consideration. This leads to quagmires like Secy. of Defense Robert McNamara’s incorrect use of body counts to measure the progress of the Viet Nam war:
https://chronotopeblog.com/2015/04/04/the-mcnamara-fallacy-and-the-problem-with-numbers-in-education/
https://www.technologyreview.com/2013/05/31/178263/the-dictatorship-of-data/
This is a problem which besets the entire rationality community, in my opinion. Mathematician Cathy O’Neil has written eloquently about it in her book Weapons of Math Destruction:
https://www.amazon.com/Weapons-Math-Destruction-Increases-Inequality/dp/0553418815
In the real world, most economic/social issues to which the mathematical tools of efficiency get applied tend to be so-called wicked problems. “In planning and policy, a wicked problem is a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. It refers to an idea or problem that cannot be fixed, where there is no single solution to the problem; and “wicked” denotes resistance to resolution, rather than evil.[1] Another definition is “a problem whose social complexity means that it has no determinable stopping point.”
“Wicked Problems”. Management Science. 14 (4): B-141–B-146, C. West Churchman, 1967.
Churchman defined wicked problems as having the following characteristics:
There is no definitive formulation of a wicked problem.
Wicked problems have no stopping rule.
Solutions to wicked problems are not true-or-false, but better or worse.
There is no immediate and no ultimate test of a solution to a wicked problem.
Every solution to a wicked problem is a “one-shot operation”; because there is no opportunity to learn by trial and error, every attempt counts significantly.
Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.
Every wicked problem is essentially unique.
Every wicked problem can be considered to be a symptom of another problem.
The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem’s resolution.
The social planner has no right to be wrong (i.e., planners are liable for the consequences of the actions they generate).
Purely mathematical/scientific measurements and definitions of efficiency appear to become extremely problematic in these situations (speaking for myself).