Positive Bias: Look Into the Dark

I am teach­ing a class, and I write upon the black­board three num­bers: 2-4-6. “I am think­ing of a rule,” I say, “which gov­erns se­quences of three num­bers. The se­quence 2-4-6, as it so hap­pens, obeys this rule. Each of you will find, on your desk, a pile of in­dex cards. Write down a se­quence of three num­bers on a card, and I’ll mark it ‘Yes’ for fits the rule, or ‘No’ for not fit­ting the rule. Then you can write down an­other set of three num­bers and ask whether it fits again, and so on. When you’re con­fi­dent that you know the rule, write down the rule on a card. You can test as many triplets as you like.”

Here’s the record of one stu­dent’s guesses:

4-6-2 No
4-6-8 Yes
10-12-14 Yes .

At this point the stu­dent wrote down their guess at the rule. What do you think the rule is? Would you have wanted to test an­other triplet, and if so, what would it be? Take a mo­ment to think be­fore con­tin­u­ing.

The challenge above is based on a clas­sic ex­per­i­ment due to Peter Wa­son, the 2-4-6 task. Although sub­jects given this task typ­i­cally ex­pressed high con­fi­dence in their guesses, only 21% of the sub­jects suc­cess­fully guessed the ex­per­i­menter’s real rule, and repli­ca­tions since then have con­tinued to show suc­cess rates of around 20%.

The study was called “On the failure to elimi­nate hy­pothe­ses in a con­cep­tual task.” Sub­jects who at­tempt the 2-4-6 task usu­ally try to gen­er­ate pos­i­tive ex­am­ples, rather than nega­tive ex­am­ples—they ap­ply the hy­po­thet­i­cal rule to gen­er­ate a rep­re­sen­ta­tive in­stance, and see if it is la­beled “Yes.”

Thus, some­one who forms the hy­poth­e­sis “num­bers in­creas­ing by two” will test the triplet 8-10-12, hear that it fits, and con­fi­dently an­nounce the rule. Some­one who forms the hy­poth­e­sis X-2X-3X will test the triplet 3-6-9, dis­cover that it fits, and then an­nounce that rule.

In ev­ery case the ac­tual rule is the same: the three num­bers must be in as­cend­ing or­der.

But to dis­cover this, you would have to gen­er­ate triplets that shouldn’t fit, such as 20-23-26, and see if they are la­beled “No.” Which peo­ple tend not to do, in this ex­per­i­ment. In some cases, sub­jects de­vise, “test,” and an­nounce rules far more com­pli­cated than the ac­tual an­swer.

This cog­ni­tive phe­nomenon is usu­ally lumped in with “con­fir­ma­tion bias.” How­ever, it seems to me that the phe­nomenon of try­ing to test pos­i­tive rather than nega­tive ex­am­ples, ought to be dis­t­in­guished from the phe­nomenon of try­ing to pre­serve the be­lief you started with. “Pos­i­tive bias” is some­times used as a syn­onym for “con­fir­ma­tion bias,” and fits this par­tic­u­lar flaw much bet­ter.

It once seemed that phlo­gis­ton the­ory could ex­plain a flame go­ing out in an en­closed box (the air be­came sat­u­rated with phlo­gis­ton and no more could be re­leased). But phlo­gis­ton the­ory could just as well have ex­plained the flame not go­ing out. To no­tice this, you have to search for nega­tive ex­am­ples in­stead of pos­i­tive ex­am­ples, look into zero in­stead of one; which goes against the grain of what ex­per­i­ment has shown to be hu­man in­stinct.

For by in­stinct, we hu­man be­ings only live in half the world.

One may be lec­tured on pos­i­tive bias for days, and yet over­look it in-the-mo­ment. Pos­i­tive bias is not some­thing we do as a mat­ter of logic, or even as a mat­ter of emo­tional at­tach­ment. The 2-4-6 task is “cold,” log­i­cal, not af­fec­tively “hot.” And yet the mis­take is sub-ver­bal, on the level of imagery, of in­stinc­tive re­ac­tions. Be­cause the prob­lem doesn’t arise from fol­low­ing a de­liber­ate rule that says “Only think about pos­i­tive ex­am­ples,” it can’t be solved just by know­ing ver­bally that “We ought to think about both pos­i­tive and nega­tive ex­am­ples.” Which ex­am­ple au­to­mat­i­cally pops into your head? You have to learn, word­lessly, to zag in­stead of zig. You have to learn to flinch to­ward the zero, in­stead of away from it.

I have been writ­ing for quite some time now on the no­tion that the strength of a hy­poth­e­sis is what it can’t ex­plain, not what it can —if you are equally good at ex­plain­ing any out­come, you have zero knowl­edge. So to spot an ex­pla­na­tion that isn’t helpful, it’s not enough to think of what it does ex­plain very well—you also have to search for re­sults it couldn’t ex­plain, and this is the true strength of the the­ory.

So I said all this, and then I challenged the use­ful­ness of “emer­gence” as a con­cept. One com­menter cited su­per­con­duc­tivity and fer­ro­mag­netism as ex­am­ples of emer­gence. I replied that non-su­per­con­duc­tivity and non-fer­ro­mag­netism were also ex­am­ples of emer­gence, which was the prob­lem. But be it far from me to crit­i­cize the com­menter! De­spite hav­ing read ex­ten­sively on “con­fir­ma­tion bias,” I didn’t spot the “gotcha” in the 2-4-6 task the first time I read about it. It’s a sub­ver­bal blink-re­ac­tion that has to be re­trained. I’m still work­ing on it my­self.

So much of a ra­tio­nal­ist’s skill is be­low the level of words. It makes for challeng­ing work in try­ing to con­vey the Art through words. Peo­ple will agree with you, but then, in the next sen­tence, do some­thing sub­de­liber­a­tive that goes in the op­po­site di­rec­tion. Not that I’m com­plain­ing! A ma­jor rea­son I’m writ­ing this is to ob­serve what my words haven’t con­veyed.

Are you search­ing for pos­i­tive ex­am­ples of pos­i­tive bias right now, or spar­ing a frac­tion of your search on what pos­i­tive bias should lead you to not see? Did you look to­ward light or dark­ness?