Assuming Nails
Tangential followup to Defeating Ugh Fields in Practice.
Somewhat related to Privileging the Hypothesis.
Edited to add:
I’m surprised by negative/neutral reviews. This means that either I’m simply wrong about what counts as interesting, or I haven’t expressed my point very well. Based on commenter response, I think the problem is the latter. In the next week or so, expect a much more concise version of this post that expresses my point about epistemology without the detour through a criticism of economics.
At the beginning of my last post, I was rather uncharitable to neoclassical economics:
If I had to choose a single piece of evidence off of which to argue that the rationality assumption of neoclassical economics is totally, irretrievably incorrect, it’s this article about financial incentives and medication compliance.… [to maintain that this theory is correct] is to crush reality into a theory that cannot hold it.
Some mistook this to mean that I believe neoclassical economists honestly, explicitly believe that all people are always totally rational. But, to quote Rick Moranis, “It’s not what you think. It’s far, far worse.” The problem is that they often take the complex framework of neoclassical economics and believe that a valid deduction within this framework is a valid deduction about the real world. However, deductions within any given framework are entirely uninformative unless the framework corresponds to reality. But, because such deductions are internally valid, we often give them far more weight than they are due. Testing the fit of a theoretical framework to reality is hard, but a valid deduction within a framework feels so very satisfying. But even if you have a fantastically engineered hammer, you cannot go around assuming everything you want to use it on is a nail. It is all too common for experts to assume that their framework applies cleanly to the real world simply because it works so well in its own world.
If this concept doesn’t make perfect sense, that’s what the rest of this post is about: spelling out exactly how we go wrong when we misuse the essentially circular models of many sciences, and how this matters. We will begin with the one discipline in which this problem does not occur. The one discipline which appears immune to this type of problem is mathematics, the paragon of “pure” academic disciplines. This is principally because mathematics appears to have perfect conformity with reality, with no research or experimentation needed to ensure said conformity. The entire system of mathematics exists, in a sense, in its own world. You could sit in windowless room (perhaps one with a supercomputer) and, theoretically, derive every major theorem of mathematics, given the proper axioms. The answer to the most difficult unsolved problems in mathematics was determined the moment the terms and operators within them were defined—once you say a “circle” is “a convex polygon with every point equidistant from a center,” you have already determined every single digit of pi. The problem is finding out exactly how this model works—making calculations and deductions within this model. In the case of mathematics, for whatever reason, the model conforms perfectly to the real world, so any valid mathematical deduction is a valid deduction in the real world.
This is not the case in any true science, which by necessity must rely on experiment and observation. Every science operates off of some simplified model of the world, at least with our current state of knowledge. This creates two avenues of progress: discoveries within the model, which allow one to make predictions about the world, and refinements of the model, which make such predictions more accurate. If we have an internally consistent framework, theoretical manipulation within our model will never show us our error, because our model is circular and functions outside the real world. It would be like trying to predict a stock market crash by analyzing the rules of Monopoly, except that it doesn’t feel absurd. There’s nothing wrong with the model qua the model, the problem is with the model qua reality, and we have to look at both of them to figure that out.
Economics is one of the fields that most suffers from this problem. Our mathematician in his windowless room could generate models of international exchange rates without ever having seen currency, once we gave him the appropriate definitions and assumptions. However, when we try using these models to forecast the future, life gets complicated. No amount of experimenting within our original model will fix this without looking at the real world. At best, we come up with some equations that appear to conform to what we observe, but we run the risk that the correspondence is incidental or that there were some (temporarily) constant variables we left out that will suddenly cease to be constant and break the whole model. It is all too easy to forget that the tremendous rigor and certainty we feel when we solve the equations of our model does not translate into the real world. Getting the “right” answer within the model is not the same thing as getting the real answer.
As an obvious practical example, an individual with a serious excess of free time could develop a model of economics which assumes that agents are rational paper-clip maximizers—that agents are rational and their ultimate concern is maximizing the number of existing paper-clips. Given even more free time and a certain amount of genius, you could even model the behaviour of irrational paper-clip maximizers, so long as you had a definition of irrational. But however refined these models are, they models will remain entirely useless unless you actually have some paper-clip maximizers whose behaviour you want to predict. And even then, you would need to evaluate your predictions after they succeed or fail. Developing a great hammer is relatively useless if the thing you need to make must be put together with screws.
There is an obvious difference in the magnitude of this problem between the sciences, and it seems to be based on the difficulty of experimenting within them. In harder sciences where experiments are fairly straightforwards, like physics and chemistry, it is not terribly difficult to make models that conform well with reality. The bleeding edge of, say, physics, tends to like in areas that are either extremely hard to observe, like the subatomic, or extremely computation-intensive. In softer sciences, experiments are very difficult, and our models rely much more on powerful assumptions, social values, and armchair reasoning.
As humans, we are both bound and compelled to use the tools we have at our disposal. The problem here is one of uncertainty. We know that most of our assumptions in economics are empirically off, but we don’t know how wrong or how much that matters when we make predictions. But the model nevertheless seeps into the very core of our model of reality itself. We cannot feel this disconnect when we try to make predictions; a well-designed model feels so complete that there is no feeling of error when we try to apply it. This is likely because we are applying it correctly, but it just doesn’t apply to reality. This leads people to have high degrees of certainty and yet frequently be wrong. It would not surprise me if the failure of many experts to appreciate the model-reality gap is responsible for a large proportion of incorrect predictions.
This, unfortunately, is not the end of the problem. It gets much worse when you add a normative element into your model, when you get to call some things, “efficient” or “healthful,” or “normal,” or “insane.” There is also a serious question as to whether this false certainty is preferable to the vague unfalsifiability of even softer social sciences. But I shall save these subjects for future posts.
Right now, it is fashionable to criticize economic models. Reading a thesis on LW that I can see in many other places is not so much fun. Furthermore, the thesis as presented is much too strong. For many purposes classical economics works. For example, classical economics does a decent job predicting how different sorts of goods will influence each other, such as how closely related products will interact.
I was surprised to see such negative response to something I found so interesting. This comment and its support suggest either I’ve written this poorly, or using Economics as an example has gotten people sidetracked from my main point (which is I suppose a more specific way of saying I have written it poorly). I shall have to attempt to make the point in a more clear and concise manner in the near future; this is not intended as a criticism of economics, it is about a particular error in our manner of thinking. The fact that economics gets a lot right is actually besides the point, and as I did not comment on the degree of conformity modern economic models have to reality, nothing I wrote was intended to say it is useless.
I assume the point you were trying to make was about the general phenomenon of theorists assuming that reality is more like their theories than it really is. I agree that this is very common and saying that people who are falling into this general trap are “assuming nails” seems like a nice shorthand. I think you missed a few things though…
For example, scientific theories are generally not eliminated from practice by demonstrating that they are ill-founded or incorrect. To “win”, what you need to do is show a better thing for people who know the existing jargon and have the relevant skillsets to do, instead of what they are already doing. Otherwise they may as well keep proving theorems and explaining that they are working out what “the ideal case” looks like, with a nod to physics (where friction is usually neglected until it is explicitly factored in as a correction term). The other thing you could do is attack their funding, but good luck with that :-P
The other issue I see, is that your general point is an area of a substantial and very long running “stylist disagreement” between experimentalists and theorists. My impression is that historically, theorists win the money and fame while experimentalists are mostly remembered as manual laborers. Tesla would be a more recent famous example of this, but a nice non-famous example can be found in the docs for python’s difflib module:
Can’t you just hear the gritting of teeth here? The resentment at unacknowledged brilliance defeated by clever marketing? (Also: XKCD!)
Most of the arguments in this very broad “debate” over theory versus practice are not concrete enough to falsify and I know of no way to change someone’s stylistic approach in this dimension of human variation using nothing but reasoned discussion. This is one of those “limits of reason” things that I’ve never found a way to deal with other than by taking an audience’s temperature and avoiding arguments that would feel too hot or too cold to them, given their existing and functionally immutable prejudices on the subject.
For reference, I think a lot of the people at LW love ethereal etherealness so much that the community would marry it if we could. That would be my first guess as to why your article’s substantive point is not a wild success.
Could you elaborate on what you mean by “ethereal etherealness”? What Eliezer is talking about in that post looks to me like what most philosophers would call abstract Platonic entities. And I get the sense (though I may be projecting) that most people here are pretty uncomfortable with those. LWers seem to think it worthwhile to eliminate any reference to anything other than concrete physical referents.
I find the discussions of Decision Theories a little ethereal sometimes. There is a base assumption that Eliezer has made that the manner of making the decision doesn’t matter. So questions of energy efficiency or computational resources used when making a decision don’t come into the discussion. I personally cannot justify that assumption looking at the evolutionary history of brains, i.e. stuff that has worked in the real world. It matters how big the brain is, the smaller the better if you can get away with it. The simpler the better, if you can get away with it, as well.
Quote from a Newcomb’s Problem article
I’ve spent a few days mulling a response to this and tried writing a response with a lot of text that needed to be boiled down with a summary at the top… and then I waited a while and read it again and it didn’t hang together the way I was hoping it would.
I stand by my general assertion as being a useful working hypothesis for guiding behavior relative to this community, but I think I may in incapable of backing it up in a way that is vivid and succinct and comprehensive all at the same time.
I think it is useful to point out that in your worthwhile link, it contains a link to “belief in the implied invisible” which explains why we should believe in the “existence” of the necessarily unobservable by arguments based on the incomputable.
Which is not to say I think solomonff induction isn’t totally sweet, but I think its cool the way I think spherical cows and classical economic assumptions are cool—they are inspiring and offer a nice first draft estimate of the “upper bound” of how things could work.
At the same time I think Jaron Lanier (who coined the term “cybernetic totalism” in order to criticize an over-hyped and over-politicized version of the computer inspired zeitgeist) is very cool… but he would have to speak with a measure of “delicacy” around here if he wanted up votes...
You should post your thoughts anyway :). Even if they don’t “hang together”, I bet that they would be an illuminating expression of the impression that this community gives you. And maybe comprehension and vividness would following from a dialogue about your impressions. (Succinctness is harder to promise ;))
But do people here like that it’s incomputable? Or do they just tolerate that it’s incomputable, because they think that they can make adequate computable approximations? I think that most people here wish that Solomonoff induction were computable (except for those who worry that it would make building an unFriendly AI too easy).
You might find this Bloggingheads.tv conversation between Eliezer and Jaron Lanier interesting. (Here’s the corresponding Overcoming Bias thread.)
Other than that BHtv diavlog, I haven’t looked at Lanier’s stuff much. I’ll check out your YouTube link.
ETA: This comment thread from February’s Open Thread did not leave me expecting to find much insight in Lanier’s work.
I would suspect it is the latter combined with a third factor: your points, so far as I can determine, are (a) models only predict reality if their assumptions are valid, and (b) it’s easy to think that your model is good even with the assumptions aren’t valid.
Point (a) would be interesting if it weren’t trivial.
Point (b) would be interesting if you showed it convincingly.
The ideal post to make these points would, instead of continuing from “If this concept doesn’t make perfect sense [...]”, demonstrate this phenomenon in several examples detailed enough to eliminate other reasonable hypotheses.
I agree completely. I love “etherial etherialness”, and I think (a) is a good point, which was terribly uninteresting to read because I’ve heard it before both on LW and elsewhere.
Not only that, the thesis is presented in a really annoying font. I actually suspect this has at least as much influence on the reception of the post.
Yikes, bad definition. Better: “the set of all points a given distance from a center”. (And, of course, such a set isn’t going to be convex; you may be thinking of a disc) .)
As for the relation of mathematics to “reality”, the reason no “empirical confirmation” of mathematical truths is needed is because “mathematics” is defined that way: it’s the subject in which you examine models internally, for their own sake. (Once upon a time, some people in ancient Greece decided it would be a good idea to have a subject like this; they seem to have been right.) If it turns out that mathematics always somehow manages to describe the world accurately....well, there’s always going to be some model that works, isn’t there? Mathematics is the study of models.
Related article by me: The role of mathematical truths: Math is applicable to the world to the extent that you can be sure it behaves in a way isomorphic to a particular axiom set.
You seem to be beating up on neoclassical econ, but doesn’t this same problem hold for other econ schools, like neo-Keynesian econ, Marxian econ, Austrian econ? All of these seem to be even less empirically rigorous than the neoclassicals, so I’m not quite sure what point is made by focusing on them.
I will not engage with your core thesis but I give a plus one for this excerpt. A problem I see with experts at times is the tendency to say explicitly or convey implicitly that the world should be a nail.
The “rational paper clip maximizers” link seems to be broken.
Can you point us to actual evidence that neoclassical economics is totally broken? Because there does seem to be pretty good evidence that demand curves slope downward, etc.
This whole line of argument reminds me of my biology major friend explaining to me that he knows psychology is bogus because the idea of the Oedipal complex is stupid.
Is there good experimental evidence supporting neoclassical economics or Freudian psychology? It is not the critic’s duty to supply the evidence—it is the duty of those that want to use the models in real life situations to show that they work in the real world.
My point was not that there is evidence to support Freudian theory, it really is bunk. My point was that non-experts have a tendency to dismiss entire fields of inquiry as bogus because of single imperfect ideas within the subject.
For instance, my Women’s Studies professor was flat out wrong when she said observed sex differences are purely “socially constructed”. This does not, however, give one a right to say that Women’s Studies can be readily dismissed by an outsider as totally worthless.
Getting “good experimental evidence” to support any macroeconomic theory is all but impossible, but that doesn’t mean there is no evidence whatsoever. A good rule of thumb is that if two absolute ideologues like Paul Krugman and Milton Friedman agree about an issue, there must be pretty damn good evidence.
knb:
I happen to agree with them about the issue of rent control, but taken as a general rule, this is absolutely fallacious. Within any belief system, leaders of different factions will often hate each other ferociously over their differences, even though from an outside perspective, they may well look almost indistinguishable. Just look at various religious and ideological disputes over incomprehensibly obscure points of doctrine.
(And yes, I do think that modern economics has some fundamentally unsound beliefs that are a matter of virtual consensus nowadays. See e.g. this discussion for some examples.)
I agree that a part of a theory can be shown to be invalid even though many other parts are shown to be valid. Even the basic assumptions of a model can be wrong and parts of the model may still work. That is not the point I was making. I was saying that the onus is on the model to be shown to work in a real life situation. It should not be assumed to work and its critics have the onus to show that it is “totally broken”. As I am not sure what experiments have been done to show the validity of neoclassic economics in the real world—I ask. You want “actual evidence that neoclassical economics is totally broken” but I think it is more reasonable to ask for evidence that it works. Is there good evidence?
My impression is that a lot of Freud’s predictions failed to bear out, and where they turned out right, they are usually better explained (more coverage of phenomenon with a simpler model) by less elaborate theories.
For example, the idea of “Freudian slips” (where accidents of speech reveal prurient obsessions) was somewhat born out, but priming theory explains the same general phenomenon without the obsessive focus on sex that was so characteristic of Freud… and which probably accounted for a large part of his success in popular culture. Also, some of his claims were just plain fraud.
Nowadays, Freudian psychology has much more relevance to literature majors than psych majors, because he offered a theory of human souls that claimed to be scientific, was full of sex, and worked at about the same level of explanation that animist theories of nature do (with lots and lots of use of the intentional stance). It was a very useful theory for novelists, but not so useful to neuro-physiologists.
I read knb’s point as “It is silly to dismiss all of psychology as bogus just because of Freudian psychology, just like it is worth considering whether all of neoclassical economics is broken or useless because some parts that are flawed.”
Yes, this is what I meant.
So, OK, great—neoclassical economics has been making predictions well in advance of its ability to test them against reality for decades. People are waking up to that, and one response is the development of neuroeconomics. Instead of assuming that people are rational, neuroeconomics tracks actual behavior based on brain states, and tries to extrapolate from that behavior.
Is that enough? If not, are there alternatives? What do you suggest we do about the fact that one of the most important branches of social science has long since gone off the deep end? It’s really frustrating reading such a well-worded critique of neoclassical economics without even a suggestion as to what should come next.
Related LW glossary word: “nonapples”.
Mathematics is about advancing human understanding of mathematics (see also). It’s not about our world (you’d have to find it first).
Case in point.
In short, “In theory there is no difference between theory and practice—but in practice there is.”
Not that it isn’t interesting, but it seems confused, and somewhat trivial.
Trivial, because it basically says: Keep in mind that the map is not the territory applies even if the map is a scientific model. A good thing to keep in mind, nevertheless.
But in the details, you seem to misunderstand some of the problems “mathematics appears to have perfect conformity with reality” is, as Vladmir Nesov points out, exactly backwards. Mathematics qua mathematics has no relation to reality, and (properly) makes no claim as to reflections of reality. Your linked article, on the surface, is perfectly in line with classical incentive economics: remembering to take meds is costly, so some people don’t do it. Give an incentive, and more people will do it. Not that there aren’t important flaws in the perfect rationality assumption, and some of them show up beneath the surface of that behavior. But show it to computer programmed to do classical economics, and it will happily calculate marginal costs of remembering to take drugs, etc.
Further, you seem to miss some of the important roots of the problem. Economics is not the only discipline where good models are lacking (turbulent flow comes to mind). But it’s easy to create a turbulent flow in a laboratory. So, is it the difficulty of experiments that cause problems, or the complexity of the phenomenon?
Or is it lack of self-awareness or honesty? Do economists imagine they understand the economy better than aeronautical engineers imagine they understand flow? And if so, why?
I’d say lack of honesty because claims are hard to verify and therefore it’s all about signaling competence to gain status. On the other hand the basics of austrian economics are almost trivial and since you don’t get points for stating the obvious austrian economics is marginalized although overall it leads to better results.