“Outside View!” as Conversation-Halter

Fol­lowup to: The Out­side View’s Do­main, Con­ver­sa­tion Halter­s
Re­ply to: Refer­ence class of the unclassreferenceable

In “con­ver­sa­tion halters”, I pointed out a num­ber of ar­gu­ments which are par­tic­u­larly per­ni­cious, not just be­cause of their in­her­ent flaws, but be­cause they at­tempt to chop off fur­ther de­bate—an “ar­gu­ment stops here!” traf­fic sign, with some im­plicit penalty (at least in the mind of the speaker) for try­ing to con­tinue fur­ther.

This is not the right traf­fic sig­nal to send, un­less the state of knowl­edge is such as to make an ac­tual halt a good idea. Maybe if you’ve got a repli­ca­ble, repli­cated se­ries of ex­per­i­ments that squarely tar­get the is­sue and set­tle it with strong sig­nifi­cance and large effect sizes (or great power and null effects), you could say, “Now we know.” Or if the other is blatantly priv­ileg­ing the hy­poth­e­sis—start­ing with some­thing im­prob­a­ble, and offer­ing no pos­i­tive ev­i­dence to be­lieve it—then it may be time to throw up hands and walk away. (Priv­ileg­ing the hy­poth­e­sis is the state peo­ple tend to be driven to, when they start with a bad idea and then wit­ness the defeat of all the pos­i­tive ar­gu­ments they thought they had.) Or you could sim­ply run out of time, but then you just say, “I’m out of time”, not “here the gath­er­ing of ar­gu­ments should end.”

But there’s also an­other jus­tifi­ca­tion for end­ing ar­gu­ment-gath­er­ing that has re­cently seen some ad­vo­cacy on Less Wrong.

An ex­per­i­men­tal group of sub­jects were asked to de­scribe highly spe­cific plans for their Christ­mas shop­ping: Where, when, and how. On av­er­age, this group ex­pected to finish shop­ping more than a week be­fore Christ­mas. Another group was sim­ply asked when they ex­pected to finish their Christ­mas shop­ping, with an av­er­age re­sponse of 4 days. Both groups finished an av­er­age of 3 days be­fore Christ­mas. Similarly, Ja­panese stu­dents who ex­pected to finish their es­says 10 days be­fore dead­line, ac­tu­ally finished 1 day be­fore dead­line; and when asked when they had pre­vi­ously com­pleted similar tasks, replied, “1 day be­fore dead­line.” (See this post.)

Those and similar ex­per­i­ments seem to show us a class of cases where you can do bet­ter by ask­ing a cer­tain spe­cific ques­tion and then halt­ing: Namely, the stu­dents could have pro­duced bet­ter es­ti­mates by ask­ing them­selves “When did I finish last time?” and then ceas­ing to con­sider fur­ther ar­gu­ments, with­out try­ing to take into ac­count the speci­fics of where, when, and how they ex­pected to do bet­ter than last time.

From this we learn, allegedly, that “the ‘out­side view’ is bet­ter than the ‘in­side view’”; from which it fol­lows that when you’re faced with a difficult prob­lem, you should find a refer­ence class of similar cases, use that as your es­ti­mate, and de­liber­ately not take into ac­count any ar­gu­ments about speci­fics. But this gen­er­al­iza­tion, I fear, is some­what more ques­tion­able...

For ex­am­ple, taw alleged upon this very blog that be­lief in the ‘Sin­gu­lar­ity’ (a term I usu­ally take to re­fer to the in­tel­li­gence ex­plo­sion) ought to be dis­missed out of hand, be­cause it is part of the refer­ence class “be­liefs in com­ing of a new world, be it good or evil”, with a his­tor­i­cal suc­cess rate of (allegedly) 0%.

Of course Robin Han­son has a differ­ent idea of what con­sti­tutes the refer­ence class and so makes a rather differ­ent pre­dic­tion—a prob­lem I re­fer to as “refer­ence class ten­nis”:

Tak­ing a long his­tor­i­cal long view, we see steady to­tal growth rates punc­tu­ated by rare tran­si­tions when new faster growth modes ap­peared with lit­tle warn­ing. We know of per­haps four such “sin­gu­lar­i­ties”: an­i­mal brains (~600MYA), hu­mans (~2MYA), farm­ing (~1OKYA), and in­dus­try (~0.2KYA)...

Ex­cess in­side view­ing usu­ally con­tinues even af­ter folks are warned that out­side view­ing works bet­ter; af­ter all, in­side view­ing bet­ter show offs in­side knowl­edge and abil­ities. Peo­ple usu­ally jus­tify this via rea­sons why the cur­rent case is ex­cep­tional. (Re­mem­ber how all the old rules didn’t ap­ply to the new dot­com econ­omy?) So ex­pect to hear ex­cuses why the next sin­gu­lar­ity is also an ex­cep­tion where out­side view es­ti­mates are mis­lead­ing. Let’s keep an open mind, but a wary open mind.

If I were to play the game of refer­ence class ten­nis, I’d put re­cur­sively self-im­prov­ing AI in the refer­ence class “huge mother#$%@ing changes in the na­ture of the op­ti­miza­tion game” whose other two in­stances are the di­vide be­tween life and non­life and the di­vide be­tween hu­man de­sign and evolu­tion­ary de­sign; and I’d draw the les­son “If you try to pre­dict that things will just go on sorta the way they did be­fore, you are go­ing to end up look­ing pa­thet­i­cally over­con­ser­va­tive”.

And if we do have a lo­cal hard take­off, as I pre­dict, then there will be noth­ing to say af­ter­ward ex­cept “This was similar to the ori­gin of life and dis­similar to the in­ven­tion of agri­cul­ture”. And if there is a non­lo­cal eco­nomic ac­cel­er­a­tion, as Robin Han­son pre­dicts, we just say “This was similar to the in­ven­tion of agri­cul­ture and dis­similar to the ori­gin of life”. And if noth­ing hap­pens, as taw seems to pre­dict, then we must say “The whole foofaraw was similar to the apoc­a­lypse of Daniel, and dis­similar to the ori­gin of life or the in­ven­tion of agri­cul­ture”. This is why I don’t like refer­ence class ten­nis.

But mostly I would sim­ply de­cline to rea­son by anal­ogy, prefer­ring to drop back into causal rea­son­ing in or­der to make weak, vague pre­dic­tions. In the end, the dawn of re­cur­sive self-im­prove­ment is not the dawn of life and it is not the dawn of hu­man in­tel­li­gence, it is the dawn of re­cur­sive self-im­prove­ment. And it’s not the in­ven­tion of agri­cul­ture ei­ther, and I am not the prophet Daniel. Point out a “similar­ity” with this many differ­ences, and re­al­ity is li­able to re­spond “So what?”

I some­times say that the fun­da­men­tal ques­tion of ra­tio­nal­ity is “Why do you be­lieve what you be­lieve?” or “What do you think you know and how do you think you know it?”

And when you’re ask­ing a ques­tion like that, one of the most use­ful tools is zoom­ing in on the map by re­plac­ing sum­mary-phrases with the con­cepts and chains of in­fer­ences that they stand for.

Con­sider what in­fer­ence we’re ac­tu­ally car­ry­ing out, when we cry “Out­side view!” on a case of a stu­dent turn­ing in home­work. How do we think we know what we be­lieve?

Our in­for­ma­tion looks some­thing like this:

  • In Jan­uary 2009, stu­dent X1 pre­dicted they would finish their home­work 10 days be­fore dead­line, and ac­tu­ally finished 1 day be­fore dead­line.

  • In Fe­bru­ary 2009, stu­dent X1 pre­dicted they would finish their home­work 9 days be­fore dead­line, and ac­tu­ally finished 2 days be­fore dead­line.

  • In March 2009, stu­dent X1 pre­dicted they would finish their home­work 9 days be­fore dead­line, and ac­tu­ally finished 1 day be­fore dead­line.

  • In Jan­uary 2009, stu­dent X2 pre­dicted they would finish their home­work 8 days be­fore dead­line, and ac­tu­ally finished 2 days be­fore dead­line.

  • And so on through 157 other cases.

  • Fur­ther­more, in an­other 121 cases, ask­ing stu­dents to vi­su­al­ize speci­fics ac­tu­ally made them more op­ti­mistic.

There­fore, when new stu­dent X279 comes along, even though we’ve never ac­tu­ally tested them be­fore, we ask:

“How long be­fore dead­line did you plan to com­plete your last three as­sign­ments?”

They say: “10 days, 9 days, and 10 days.”

We ask: “How long be­fore did you ac­tu­ally com­plete them?”

They re­ply: “1 day, 1 day, and 2 days”.

We ask: “How long be­fore dead­line do you plan to com­plete this as­sign­ment?”

They say: “8 days.”

Hav­ing gath­ered this in­for­ma­tion, we now think we know enough to make this pre­dic­tion:

“You’ll prob­a­bly finish 1 day be­fore dead­line.”

They say: “No, this time will be differ­ent be­cause—”

We say: “Would you care to make a side bet on that?”

We now be­lieve that pre­vi­ous cases have given us strong, veridi­cal in­for­ma­tion about how this stu­dent func­tions—how long be­fore dead­line they tend to com­plete as­sign­ments—and about the un­re­li­a­bil­ity of the stu­dent’s plan­ning at­tempts, as well. The chain of “What do you think you know and how do you think you know it?” is clear and strong, both with re­spect to the pre­dic­tion, and with re­spect to ceas­ing to gather in­for­ma­tion. We have his­tor­i­cal cases aplenty, and they are all as similar to each other as they are similar to this new case. We might not know all the de­tails of how the in­ner forces work, but we sus­pect that it’s pretty much the same in­ner forces in­side the black box each time, or the same rough group of in­ner forces, vary­ing no more in this new case than has been ob­served on the pre­vi­ous cases that are as similar to each other as they are to this new case, se­lected by no differ­ent a crite­rion than we used to se­lect this new case. And so we think it’ll be the same out­come all over again.

You’re just draw­ing an­other ball, at ran­dom, from the same bar­rel that pro­duced a lot of similar balls in pre­vi­ous ran­dom draws, and those pre­vi­ous balls told you a lot about the bar­rel. Even if your es­ti­mate is a prob­a­bil­ity dis­tri­bu­tion rather than a point mass, it’s a solid, sta­ble prob­a­bil­ity dis­tri­bu­tion based on plenty of sam­ples from a pro­cess that is, if not in­de­pen­dent and iden­ti­cally dis­tributed, still pretty much blind draws from the same big bar­rel.

You’ve got strong in­for­ma­tion, and it’s not that strange to think of stop­ping and mak­ing a pre­dic­tion.

But now con­sider the analo­gous chain of in­fer­ences, the what do you think you know and how do you think you know it, of try­ing to take an out­side view on self-im­prov­ing AI.

What is our data? Well, ac­cord­ing to Robin Han­son:

  • An­i­mal brains showed up in 550M BC and dou­bled in size ev­ery 34M years

  • Hu­man hunters showed up in 2M BC, dou­bled in pop­u­la­tion ev­ery 230Ky

  • Farm­ers, show­ing up in 4700BC, dou­bled ev­ery 860 years

  • Start­ing in 1730 or so, the econ­omy started dou­bling faster, from 58 years in the be­gin­ning to a 15-year ap­prox­i­mate dou­bling time now.

From this, Robin ex­trap­o­lates, the next big growth mode will have a dou­bling time of 1-2 weeks.

So far we have an in­ter­est­ing ar­gu­ment, though I wouldn’t re­ally buy it my­self, be­cause the dis­tances of differ­ence are too large… but in any case, Robin then goes on to say: We should ac­cept this es­ti­mate flat, we have prob­a­bly just gath­ered all the ev­i­dence we should use. Tak­ing into ac­count other ar­gu­ments… well, there’s some­thing to be said for con­sid­er­ing them, keep­ing an open mind and all that; but if, fool­ishly, we ac­tu­ally ac­cept those ar­gu­ments, our es­ti­mates will prob­a­bly get worse. We might be tempted to try and ad­just the es­ti­mate Robin has given us, but we should re­sist that temp­ta­tion, since it comes from a de­sire to show off in­sider knowl­edge and abil­ities.

And how do we know that? How do we know this much more in­ter­est­ing propo­si­tion that it is now time to stop and make an es­ti­mate—that Robin’s facts were the rele­vant ar­gu­ments, and that other ar­gu­ments, es­pe­cially at­tempts to think about the in­te­rior of an AI un­der­go­ing re­cur­sive self-im­prove­ment, are not rele­vant?

Well… be­cause...

  • In Jan­uary 2009, stu­dent X1 pre­dicted they would finish their home­work 10 days be­fore dead­line, and ac­tu­ally finished 1 day be­fore dead­line.

  • In Fe­bru­ary 2009, stu­dent X1 pre­dicted they would finish their home­work 9 days be­fore dead­line, and ac­tu­ally finished 2 days be­fore dead­line.

  • In March 2009, stu­dent X1 pre­dicted they would finish their home­work 9 days be­fore dead­line, and ac­tu­ally finished 1 day be­fore dead­line.

  • In Jan­uary 2009, stu­dent X2 pre­dicted they would finish their home­work 8 days be­fore dead­line, and ac­tu­ally finished 2 days be­fore dead­line...

It seems to me that once you sub­tract out the scary la­bels “in­side view” and “out­side view” and look at what is ac­tu­ally be­ing in­ferred from what—ask “What do you think you know and how do you think you know it?”—that it doesn’t re­ally fol­low very well. The Out­side View that ex­per­i­ment has shown us works bet­ter than the In­side View, is pretty far re­moved from the “Out­side View!” that taw cites in sup­port of pre­dict­ing against any epoch. My own similar­ity met­ric puts the lat­ter closer to the analo­gies of Greek philoso­phers, ac­tu­ally. And I’d also say that try­ing to use causal rea­son­ing to pro­duce weak, vague, qual­i­ta­tive pre­dic­tions like “Even­tu­ally, some AI will go FOOM, lo­cally self-im­prov­ingly rather than global-eco­nom­i­cally” is a bit differ­ent from “I will com­plete this home­work as­sign­ment 10 days be­fore dead­line”. (The Weak In­side View.)

I don’t think that “Out­side View! Stop here!” is a good cog­ni­tive traf­fic sig­nal to use so far be­yond the realm of home­work—or other cases of many draws from the same bar­rel, no more dis­similar to the next case than to each other, and with similarly struc­tured forces at work in each case.

After all, the wider refer­ence class of cases of tel­ling peo­ple to stop gath­er­ing ar­gu­ments, is one of which we should all be wary...