[Question] Understanding information cascades

Meta: Be­cause we think un­der­stand­ing info cas­cades are im­por­tant, we re­cently spent ~10 hours try­ing to figure out how to quan­ti­ta­tively model them, and have con­tributed our think­ing as an­swers be­low. While we cur­rently didn’t have the time to con­tinue ex­plor­ing, we wanted to ex­per­i­ment with see­ing how much the LW com­mu­nity could to­gether build on top of our pre­limi­nary search, so we’ve put up a ba­sic prize for more work and tried to struc­ture the work around a cou­ple of open ques­tions. This is an ex­per­i­ment! We’re look­ing for­ward to read­ing any of your con­tri­bu­tions to the topic, in­clud­ing things like sum­maries of ex­ist­ing liter­a­ture and build­ing out new mod­els of the do­main.

Background

Con­sider the fol­low­ing situ­a­tion:

Bob is won­der­ing whether a cer­tain pro­tein in­jures the skele­tal mus­cle of pa­tients with a rare dis­ease. He finds a hand­ful pa­pers with some ev­i­dence for the claim (and some with ev­i­dence against it), so he sim­ply states the claim in his pa­per, with some cau­tion, and adds that as a cita­tion. Later, Alice comes across Bob’s pa­per and sees the cited claim, and she pro­ceeds to cite Bob, but with­out trac­ing the cita­tion trail back to the origi­nal ev­i­dence. This keeps hap­pen­ing, in var­i­ous shapes and forms, and af­ter a while a liter­a­ture of hun­dreds of pa­pers builds up where it’s com­mon knowl­edge that β amy­loid in­jures the skele­tal mus­cle of pa­tients with in­clu­sion body myosi­tis—with­out the claim hav­ing ac­cu­mu­lated any more ev­i­dence. (This real ex­am­ple was taken from Green­berg, 2009, which is a case study of this event.)

An in­for­ma­tion-cas­cade oc­curs when peo­ple up­date on each oth­ers be­liefs, rather than shar­ing the causes of those be­liefs, and those be­liefs end up with a ves­tige of sup­port that far out­strips the ev­i­dence for them. Satvik Beri might de­scribe this as the prob­lem of only shar­ing the out­puts of your think­ing pro­cess, not your in­puts.

The dy­nam­ics here are per­haps rem­i­nis­cent of those un­der­ly­ing var­i­ous failures of col­lec­tive ra­tio­nal­ity such as as­set bub­bles, by­stan­der effects and stam­pedes.

Note that his effect is differ­ent from other prob­lems of col­lec­tive ra­tio­nal­ity like the repli­ca­tion crisis, which in­volve low stan­dards for ev­i­dence (such as un­rea­son­ably lax p-value thresh­olds or co­or­di­na­tion prob­lems pre­vent­ing pub­lish­ing of failed ex­per­i­ments), or the de­gen­er­acy of much on­line dis­cus­sion, which in­volves tribal sig­nal­ling and UI en­courag­ing prob­le­matic se­lec­tion effects. Rather, in­for­ma­tion cas­cades in­volve peo­ple ra­tio­nally up­dat­ing with­out any ob­ject-level ev­i­dence at all, and would per­sist even if the repli­ca­tion crisis and on­line out­rage cul­ture dis­ap­peared. If no­body lies or tells un­truths, you can still be sub­ject to an in­for­ma­tion cas­cade.

Questions

Ben and I are con­fused about how to think about the nega­tive effects of this prob­lem. We un­der­stand the ba­sic idea, but aren’t sure how to rea­son quan­ti­ta­tively about the im­pacts, and how to trade-off solv­ing these prob­lems in a com­mu­nity ver­sus do­ing other im­prove­ments to over­all effi­cacy and effi­ciency of a com­mu­nity. We cur­rently know only how to think about these qual­i­ta­tively.

We’re post­ing a cou­ple of re­lated ques­tions that we have some ini­tial thoughts on, that might help clar­ify the prob­lem.

If you have some­thing you’d like to con­tribute, but that doesn’t seem to fit into the re­lated ques­tions above, leave it as an an­swer to this ques­tion.

Bounties

We are com­mit­ting to pay at least ei­ther $800 or (No. of an­swers and com­ments * $25), whichever is smaller, for work on this prob­lem recorded on LW, done be­fore May 13th. The prize pool will be split across com­ments in ac­cor­dance with how valuable we find them, and we might make awards ear­lier than the dead­line (though if you know you’ll put in work in x weeks, it would be good to men­tion that to one of us via PM).

Ben and Ja­cob are each re­spon­si­ble for half of the prize money.

Ja­cob is fund­ing this through Me­tac­u­lus AI, a new fore­cast­ing plat­form track­ing and im­prov­ing the state-of-the-art in AI fore­cast­ing, partly to help avoid info-cas­cades in the AI safety and policy com­mu­ni­ties (we’re cur­rently live and invit­ing beta-users, you can sign-up here).

Ex­am­ples of work each of us are es­pe­cially ex­cited about:

Jacob

  • Con­tri­bu­tions to our Guessti­mate model (linked here), such as re­duc­ing un­cer­tainty on the in­puts or us­ing bet­ter mod­els.

  • Ex­ten­sions of the Guessti­mate model be­yond biomedicine, es­pe­cially in ways that make it more di­rectly ap­pli­ca­ble to the ra­tio­nal­ity/​effec­tive al­tru­ism communities

  • Ex­am­ples and anal­y­sis of ex­ist­ing in­ter­ven­tions that deal with this and what makes them work, pos­si­bly sug­ges­tions for novel ones (though avoid­ing the trap of op­ti­mis­ing for good-seem­ing ideas)

  • Dis­cus­sion of how the prob­lem of info-cas­cades re­lates to forecasting

Ben

  • Con­cise sum­maries of rele­vant pa­pers and their key contributions

  • Clear and con­cise ex­pla­na­tions of what other LWers have found (e.g. turn­ing 5 long an­swers into 1 medium sized an­swer that links back to the oth­ers while still con­vey­ing the key info. Here’s a good ex­am­ple of some­one dis­till­ing an an­swer sec­tion).