Are Cognitive Biases Design Flaws?
I am a newbie so today I read the article by Eliezer Yudkowski “Your Strength As A Rationalist” which helped me understand the focus of LessWrong, but I respectfully disagreed with a line that is written in the last paragraph:
It is a design flaw in human cognition...
So this was my comment in the article’s comment section which I bring here for discussion:
Since I think evolution makes us quite fit to our current environment I don’t think cognitive biases are design flaws, in the above example you imply that even if you had the information available to guess the truth, your guess was another one and it was false, therefore you experienced a flaw in your cognition.
My hypotheses is that reaching the truth or communicating it in the IRC may have not been the end objective of your cognitive process, in this case just to dismiss the issue as something that was not important anyway “so move on and stop wasting resources in this discussion” was maybe the “biological” objective and as such it should be correct, not a flaw.
If the above is true then all cognitive bias, simplistic heuristics, fallacies, and dark arts are good since we have conducted our lives for 200,000 years according to these and we are alive and kicking.
Rationality and our search to be LessWrong, which I support, may be tools we are developing to evolve in our competitive ability within our species, but not a “correction” of something that is wrong in our design.
Edit 1: I realize there is change in the environment and that may make some of our cognitive biases, which were useful in the past, to be obsolete. If the word “flaw” is also applicable to describe something that is obsolete then I was wrong above. If not, I prefer the word obsolete to characterize cognitive biases that are no longer functional for our preservation.
No. This does not work on an individual level; nearly every human is dead, because of inability to invent medicine and cure aging. Nor does it seem to work on a species level; the wikipedia page for our genus names fifteen extinct species, some of which are believed to have been as or nearly as intelligent as we were.
Perhaps even smarter. Along certain metrics at least.
The paragraph in question doesn’t talk about a cognitive bias. It talks about how the feeling of a model not matching a story is too weak.
Evolution didn’t design us. As such the phrase “design flaw” isn’t meant to be taking 100% literally. For the purpose that Eliezer uses the word in that context being supoptimal for our times is enough.
The designer had a specific design goal : “thou shalt replicate adequately well under the following environmental conditions”...
Given the complex, intricate mechanisms that humans seem to have that allow for this, the “designer” did a pretty good job.
Cognitive biases boost replication under the environmental conditions they were meant for, and they save on the brainpower required.
So yes, I agree with you. If the human brain system were an engineered product, it clearly meets all of the system requirements the client (mother nature) asked for. It clearly passes the testing. The fact that it internally takes a lot of shortcuts and isn’t capable of optimal performance in some alien environment (cities, virtual spaces, tribes larger than a few hundred people) doesn’t make it a bad solution.
Another key factor you need to understand in order to appreciate nature is the constraints it is operating under. We can imagine a self-replicating system that has intelligence of comparable complexity and flexibility to humans that makes decisions that optimal to a few decimal places. But does such a system exist inside the design space accessible to Earth biology? Probably not.
The simple reason for this is 3 billion years of version lock in. All life on earth uses a particular code-space, where every possible codon in DNA maps to a specific amino acid. With 3 bases per codon, there’s 4^3 possibilities, and all of them map to an existing amino acid. In order for a new amino acid to be added to the code base, existing codons would have to be repurposed, or an organism’s entire architecture would need to be extended to a 4 (or more) codon base system.
We can easily design a state machine that translates XXX → _XXX, remapping an organisms code to a new coding scheme. However, such a machine would be incredibly complicated at the biological level—it would be a huge complex of proteins and various RNA tools, and it would only be needed once in a particular organism’s history. Evolution is under no forces to evolve such a machine, and the probability of it occurring by chance is just too small.
To summarize, everything that can ever be designed by evolution has to be made of amino acids from a particular set, or created as a derivative product by machinery made of these amino acids.
An organism without cognitive biases would probably need a much more powerful brain. Nature cannot build such a brain with the parts available.
So the extent to which various traits are adaptive vs. maladaptive is an interesting question. There are a lot of hidden trade-offs, especially when you start discussing cognitive heuristics. Modern life also has some fairly different selection pressures than our species has historically been exposed to, so maybe some of those instincts are getting out-dated.
But all of that is secondary to a much larger consideration. Evolution doesn’t share my goals. Evolution designed my brain for gene propagation. It does a decent job at survival, resource acquisition, and many other problems because those are useful for gene propagation. But I have almost no interest in gene propagation! I’m interested in the truth, even if the truth won’t get me laid. My deep suspicion of many of my biological impulses isn’t because I suspect natural selection of being a limited bumbling algorithm, but is instead rooted in my conviction that those biological impulses have a different goal in mind than I do.
As a side note, tool development isn’t a super useful competitive advantage because it’s a lot easier to steal or copy a tool than it is to develop one. The advantage you get from making a new tool is always temporary.
This is a key definition, the feeling of beauty, good, bad, justice, etc. are our conscious interpretations of reality, but their functional advantages all respond to our basic needs: to get laid and self preservation.
I guess if we pay attention to our conscious interpretation of reality then yes, our biases are have flaws because they are not entirely aligned with our values. But if we see how functional they are towards our biological needs maybe they are perfectly good.
200,000 years? Pfah. That’s a poor measure of success.
Goblin sharks have been around for 600 times as long. I don’t think we can say we’re particularly successful as a species for at least a 30 million years.
With luck you and I as individuals will be around to see it, but to do so, we’ll probably have deal with our biases.
Also, evolution happens across a species in an environment and within a species across a population. You don’t have to be faster than the bear, just faster than the slowest guy being chased by the bear.
Lacking a designer, a design flaw is a pointless term. Are our biases wrong—compared to what exactly? We are just what we are. We really need to adopt value-neutral language here. Our biases are simply our traits. Or hardware features to work with.
This feels like some kind of naturalistic fallacy—it was made by evolution, therefore it is somehow meaningful.
Evolution is just a stupid process that sometimes even can bring you to extinction. Worshiping evolution is like worshiping the water for flowing downhill. Of course I don’t mean “worship” in the original religious sense here, but rather in the sense which atheists often do without realizing they are doing it: verbally denying the supernatural, but still alieving that the thing has some higher meaning, higher purpose, is an exception to the rules, a separate magisterium, or any of the dozens of excuses that all say: “the usual rules of thinking do not apply here.”
What evolution does, is what evolution does, nothing more. If evolution caused X, all that it means is that historically, under specific circumstances, the animals with X had a higher chance of reproduction. First, it is probably a crude hack, not a finely tuned solution. It is merely better than nothing, or better than the previous version; that is enough to make an evolutionary advantage. Second, as they say in finance: “past performance does not guarantee future results”. Just because eating everything that contained a lot of sugar increased your fittness in the ancient jungle, it does not mean it will do the same thing now. If this is true for your digestion, then it is also true for your mind.
The mind is not a separate magisterium. It is yet another system which evolved by crude hack after crude hack, with no intelligent design, with short-term improvements prefered to hypothetical better long-term alternatives, with a lot of randomness, interdependent with other things in the environment. If you can believe that your digestion is not perfect, by the same reasoning you should be able to believe that your mind is not perfect. Imagining that human body consists of 99 imperfect organs and 1 perfect one, that would be a very unlikely hypothesis requiring a lot of evidence; not something we should assume as a null hypothesis. (Unless you would believe that the whole human body is flawless. Then I would have a question about why we are not super resistant to illness, why we have allergies, etc.)
Okay, your turn: Is there any reason to believe in perfection of human mind, without believing in the perfection of all the other human organs? (Hint: all of them were shaped by evolution.)
Even that is only true for a narrow reading of the sentence.
That’s where it stops being true. Genetic drift is part of evolution and can lead to worse versions.
Yes, I may be incurring in the fallacy “it is a product of evolution therefore it is not flawed”, but I think it hasn’t been refuted in this discussion.
I think this is the key, out cognitive biases are there because they were useful for our biological objectives, but they were built in an evolving environment so if that environment changes then they can become obsolete.
Therefore I would prefer to characterize our cognitive biases as potentially obsolete rather than flaws. A flaw seems to be the product of error, that it was never useful, which I think is not possible in evolution, but obsolete is the product of change so the “past performance” phrase is more suitable in my opinion.
Because of the same reason I believe our cognitive biases are not flawed, I don’t agree our organs have flaws either. All organisms with their internal organs seems to be fit for their niche in nature.
But that doesn’t mean some organs don’t need to change, e.g. 10,000 years ago we didn’t eat wheat, but now that we do we all need adapt our digestive system to that.
Conclusion: Since evolution is an adaptation to change and change is constant then we may experience some traits to be obsolete, therefore I cannot say we are entirely “fit” to the current environment, but that does not mean they are flaws since they were functional at least in some past environment, if not still i the present.
Adaptation takes time. Maybe the environment is changing faster than the organs evolve. Also, evolution finds local optima, not global ones, so it can get stuck at suboptimal designs which any incremental changes would only make worse.
Depends on what you define success, actually.
Most people here like science and everything around it so eliminating cognitive biases is EXTREMELY important in order to reach their goal.
Most people on the outside however are more obsessed about money or status and so are probably going to benefit from some degree of rationality, but anything else is probably dimnishing returns for one reason or another.ff
I’d say Eliezer put a higher standard on “Human” rather than what your average clubgoer thinks of.
Again it depends on how you define success.
In other words, epistemic rationality is not instrumental rationality.
The potential rewards of epistemic rationality for a society are very high.
However, it doesn’t follow that everyone needs to be an epistemic rationalist, and it also doesn’t necessarily follow that anyone has to remove all their biases individually, since biases can be allowed to cancel out in collective rationality.
Are they for the individual, too? There’s only two parents for each child.
I agree that we can define an evolutionary trait as a flaw or as perfect depending on what is the definition of success of the observer, but my options are between the “Human” and the biological justification. My objective is to work with the truth too!
We’d also need to separate the first from the second.
The first one probably has little advantage to reproduction (It would make sense to say that rationalists SHOULD be at the top of the social ladder but in fact that’s not often true.)
The second is probably your best bet of reproducing.
I don’t understand this. Care to elaborate?
Above you mention two types of people:
But I am clarifying that my observation is from the point of view of human conscious objectives human may have vs biological objectives.
I think all evolved traits respond to biological goals and that we may regard some biases as flawed, but that’s from the human perspective.
If rationalist means instrumental rationalist, they often are winning.
I thought instrumental and epistemic rationality feed each other, no? Can you be one but not the other?
An epistemic rationalist is an instrumental rationalist who values truth. An instrumental rationalist values something else. If they value typical things like wealth and status, then there is some evidence that the winners in society have won by systematic winning. But epistemic rationalists don’t often win in those terms.
That doesn’t really answer the question though.
How can you make a plan (instrumental rationality) without having solid premises? (epistemic rationality)
How can you know what works and what not (epistemic rationality) if you haven’t tried something? (instrumental rationality)
There’s a difference between the two in theory,because an idealized ageny either has true knowledge as a terminal value or not.
The extent to which a given agent can stick to instrumental rationality depends on its nature., how fuzzy or leaky it is. An instrumental rationalist that habitually gathers knowledge of no obvious use might mutate into what is FAPP an epistemic rationalist.
I cant see why experimentation should be more connected to IR than ER.
It’s impossible to say, really. Yes, some kind of approximation in rational thinking is necessary for building a practical brain. But it’s also a fallacy to think evolution created the perfect thinking machine that is optimally fit, even for its environment. Our brains are obviously suboptimal in many areas (and in my personal opinion are deeply and horrifyingly suboptimal in most areas). It’s hard to say, though, whether something is a feature or a bug without examining the neural circuitry behind them in detail.