An in­for­ma­tion cas­cade is a prob­lem in group ra­tio­nal­ity. Wikipe­dia has ex­cel­lent in­tro­duc­tions and links about the phe­nomenon, but here is a meta-ish ex­am­ple us­ing like­li­hood ra­tios.

Sup­pose in some fu­ture ver­sion of this site, there are sev­eral well-known facts:

• All posts come in two kinds, high qual­ity (in­sight­ful and rele­vant) and low qual­ity (old ideas re­hashed, long hy­po­thet­i­cals).

• There is a well-known prior 60% chance of any­thing be­ing high qual­ity, rather than low qual­ity. (We’re do­ing well!)

• Read­ers get a pri­vate sig­nal, ei­ther “high” or “low”, their per­sonal judge­ment of qual­ity, which is wrong 20% of the time.

• The num­ber of up and down votes is dis­played next to each post. (Note the differ­ence from the pre­sent sys­tem, which only dis­plays up minus down. This hy­poth­e­sis makes the math eas­ier.)

• Read­ers are com­pe­tent in Bayesian statis­tics and strive to vote the true qual­ity of the post.

Let’s talk about how the very first reader would vote. If they judged the post high qual­ity, then they would mul­ti­ply the prior like­li­hood ra­tio (6:4) times the bayes fac­tor for a high pri­vate sig­nal (4:1), get (6*4:4*1) = (6:1) and vote the post up. If they judged the post low qual­ity then they would in­stead mul­ti­ply by the bayes fac­tor for a low pri­vate sig­nal (1:4), get (6*1:4*4) = (3:8) and vote the post down.

There were two sce­nar­ios for the first reader (pri­vate in­for­ma­tion high or low). If we spec­u­late that the first reader did in fact vote up, then there are two sce­nar­ios for the sec­ond sce­nario: There are two sce­nar­ios for the sec­ond reader:

1. Per­sonal judge­ment high: (6:4)*(4:1)*(4:1) = (24:1), vote up.

2. Per­sonal judge­ment low: (6:4)*(1:4)*(4:1) = (6:4), vote up against per­sonal judge­ment.

Note that now there are two ex­pla­na­tions for end­ing up two votes up. It could be that the sec­ond reader ac­tu­ally agreed, or it could be that the sec­ond reader was fol­low­ing the first reader and the prior against their per­sonal judge­ment. That means that the third reader gets zero in­for­ma­tion from the sec­ond reader’s per­sonal judge­ment! The two sce­nar­ios for the third reader, and ev­ery fu­ture reader, are ex­actly analo­gous to the two sce­nar­ios for the sec­ond reader.

1. Per­sonal judge­ment high: (6:4)*(4:1)*(4:1) = (24:1), vote up.

2. Per­sonal judge­ment low: (6:4)*(1:4)*(4:1) = (6:4), vote up against per­sonal judge­ment.

This has been a night­mare sce­nario of group­think af­flict­ing even dili­gent bayesi­ans. Pos­si­ble con­clu­sions:

• Don’t strive to vote the true qual­ity of the post, strive to vote your per­sonal judge­ment.

• Try to avoid even notic­ing the score. (Maybe scores could even be oc­cluded, like spoiler-text?)

• In­for­ma­tion cas­cades are dan­ger­ous and in­ter­est­ing. We should de­velop good cog­ni­tive cit­i­zen­ship tech­niques.

• Broad­cast novel ev­i­dence, not con­clu­sions.

Note: Olle found an er­ror that ne­ces­si­tated a rewrite. I apol­o­gize.

• The con­fu­sion is be­cause you are try­ing to do two con­tra­dic­tory things with your “vote”. If the goal is to in­form fu­ture read­ers as to whether a post is high qual­ity or low qual­ity, clearly you should just give your per­sonal as­sess­ment. The only rea­son you’d take pri­ors into ac­count was if your “vote” was ac­tu­ally a “bet” and there was some ad­van­tage to “bet­ting” cor­rectly.

• Les­son learned: If you want use­ful feed­back, avoid mak­ing it a bet/​com­pe­ti­tion.

• But what about pre­dic­tion mar­kets?

• Let’s say we strive to vote ac­cord­ing to our per­sonal judg­ment. Should we vote strate­gi­cally or not?

For ex­am­ple, let’s say I read a post that seems marginally good. It has a score that’s sig­nifi­cantly higher than other posts which seem su­pe­rior. Should I down­vote the post to in­di­cate that I think its score should be lower, or up­vote to in­di­cate that I think the post was marginally good?

• An­noy­ance wrote, “There’s no point to the rat­ing sys­tem if we don’t ex­er­cise our eval­u­a­tions in­de­pen­dently. Tak­ing other peo­ple’s opinions into con­sid­er­a­tion may be a good way to reach a fi­nal con­clu­sion, but it’s a ter­rible way to form your own opinion.”

It’s also a ter­rible way to con­tribute to a good group eval­u­a­tion. One of the points Surow­iecki wrote about in “The Wis­dom of Crowds” is the ne­ces­sity of ag­gre­gat­ing in­de­pen­dent judge­ments. If the judge­ments to be ag­gre­gated are not in­de­pen­dent you get bub­bles and runs.

• One way to en­courage peo­ple to not just vote stuff up be­cause it is already pop­u­lar would be to oc­ca­sion­ally put up a sub­tle non­sense post, prefer­ably by some­one high sta­tus like EY or robin, seed it with an already fairly high score (such as 10) and then heav­ily pe­nal­ize peo­ple’s karma if they vote it up. One might call this a “hon­ey­pot for phoney ra­tio­nal­ists”

This would re­quire some in­cen­tive for peo­ple to vote, to com­pen­sate for the small prob­a­bil­ity of be­ing hit with a penalty. Over­all this would make the Karma sys­tem more com­plex.

• Should we be wor­ried that peo­ple will vote stuff up just be­cause it is already pop­u­lar? There is cur­rently no penalty for vot­ing against the crowd, so wouldn’t peo­ple (rightly) want to do this?

(Of course, we as­sume peo­ple are vot­ing based on their per­sonal im­pres­sions. It’s clear that votes bases on Bayesian be­liefs are not are use­ful here.)

• When mod­er­at­ing com­ments, the goal is not to vote good posts up and bad posts down, but to make the vote to­tal most ac­cu­rately re­flect the sig­nals of all the peo­ple who voted on it. Since vot­ers don’t gain or lose any­thing by vot­ing ac­cu­rately, be­sides the satis­fac­tion of know­ing that their votes help the scores more ac­cu­rately re­flect post qual­ity, they should always vote ac­cord­ing to their pri­vate sig­nal, and ig­nore the sig­nals that oth­ers have given.

On the other hand, when sig­nal­ing is tied to­gether with some other choice, then in­for­ma­tion cas­cades can hap­pen. The ex­am­ple that was given in my net­works class was a case of two restau­rants next to each other, where each po­ten­tial pa­tron can see how busy each restau­rant is. In that case, peo­ple don’t care about their sig­nal, but just want to visit the bet­ter restau­rant, and an in­for­ma­tion cas­cade is likely to oc­cur. A similar oc­cur­rence hap­pens with book pur­chases: if a book ap­pears on a best-sel­ler list, then that sig­nals to ev­ery­one that it’s good, but it may only be there be­cause peo­ple bought it based on that sig­nal. There are doc­u­mented ex­am­ples of clever pub­lish­ers have buy­ing copies of their own books to kick-start this effect.

• Hold on, Johni­cholas, isn´t there a slip in the calcu­la­tion con­cern­ing the third reader, case 4? You say

1. Low, one vote up and one vote down: (6:4)(4:1)(1:4)(14) = (6:1), vote up against judgement

...but shouldn’t this pro­duce the an­swer (3:8) rather than (6:1)? The con­clu­sion seems to be that as long as ei­ther the score is tied or “down” leads by one, read­ers will keep on vot­ing ac­cord­ing to their judge­ment, while as soon as ei­ther “up” leads by one or “down” leads by one, the next reader and all the fol­low­ing will ig­nore their judge­ments and fol­low suit.

Slightly more com­pli­cated, but still a great ex­am­ple!

• You are ENTIRELY CORRECT! I am em­bar­rassed and I apol­o­gize.

I jug­gled the num­bers re­peat­edly, try­ing to get a brief ex­am­ple that only uses num­bers, not sym­bols; when it seemed like I had suc­ceeded, I stopped.

I’ll think about how to cor­rect the post.

• It is in­ter­est­ing to ob­serve the dis­tri­bu­tion of scores on re­cent posts.

0, 0, 2, 3, 3, 14, 19, 23

these are fairly ob­vi­ously clus­tered into “high scor­ing” and “very low scor­ing”, in­di­cat­ing that a non­lin­ear effect is in play, per­haps some­thing like an in­for­ma­tion cas­cade.

• If karma was hid­den, would you ex­pect it to be lin­ear?

Plus, as far as I know, we can’t see the to­tal up votes and down votes i.e. more rele­vant in­for­ma­tion.

• Well I’ll have to be care­ful here that I don’t say some­thing stupid, be­cause my grasp of statis­tics needs work. I would cer­tainly not ex­pect the dis­tri­bu­tion to have two clear peaks un­less the un­der­ly­ing qual­ity of the posts was similarly two-peaked.

• I think the ex­pla­na­tion for this on LessWrong is the same as the ex­pla­na­tion on Red­dit (which, from what I un­der­stand, served as the code base for LW).

Peo­ple don’t have un­limited time, and they are will­ing to spend time on LW read­ing good posts, but un­will­ing to waste time read­ing bad posts. Thus many peo­ple will some­how filter lower posts (I do so by sim­ply sort­ing the posts from high­est rated to low­est rated, and read un­til I run out of time or get bored).

If many peo­ple do this, then posts which are “gen­er­ally agreed as good” will then to shoot up as ev­ery­one votes them up. Posts which are bad will gen­er­ally drop down to the filter point (usu­ally around 0), and then stay there rather than go fur­ther be­low.

If ev­ery­one voted on ev­ery post, you would ex­pected bad posts to con­tinue drop­ping down, and a post which was “gen­er­ally agreed as bad” would have just as large a mag­ni­tude as posts which are “gen­er­ally agreed as good”, ex­cept in the nega­tive di­rec­tion.

But we don’t see posts with a score of −23. They seem to ei­ther be neu­tral, or good. So my the­ory that peo­ple filter to only see (and thus only vote on) neu­tral or bet­ter ar­ti­cle seems to be able to pre­dict what ac­tu­ally hap­pens with the scores.

• I may end up link­ing this from the About page when it comes time to ex­plain sug­gested vot­ing poli­cies.

Of course, your per­sonal vote up or down should not be in­fluenced by that.

(Also, I did a quick edit to in­sert a page break /​ con­tinu­a­tion thingy, hope you don’t mind.)

• Thanks; I’ll learn to use those.

• Great ex­am­ple. In gen­eral, when there have been many votes, most peo­ple would do best to be­lieve the con­sen­sus (as­sum­ing it is mean­ingful). But if ev­ery­one just then votes based on their up­dated opinion, they will just re­in­force the con­sen­sus, mak­ing it mean­ingless. So I think your sug­ges­tion is right that you should vote your per­sonal judge­ment, as it was be­fore you were in­formed by the vot­ing data.

I have oc­ca­sion­ally ar­gued that this is kind of a pris­oner’s dilemma game. Co­op­er­at­ing means try­ing not to be per­suaded by the con­sen­sus and vot­ing your per­sonal opinion. Defect­ing means up­dat­ing from the con­sen­sus, thereby los­ing the abil­ity to make an un­bi­ased con­tri­bu­tion to the col­lec­tive.

• 6 Mar 2009 6:22 UTC
6 points
Parent

deleted

• I think this is very im­por­tant we have a lot of peo­ple who are un­sure and don’t vote, this I think is a rele­vant in­for­ma­tion that must be taken ac­count of.

• Per­haps LW could ran­domly hide scores of some ar­ti­cles for a while af­ter they’re posted. If this were done with enough ar­ti­cles that the sam­ple in­cluded a wide range of ar­ti­cle types and qual­ities, we could eas­ily see just how sig­nifi­cant an effect there is from hav­ing scores visi­ble.

• If we were se­ri­ous about this, I’d sug­gest a dou­ble-blind ex­per­i­ment where for a ran­domly se­lected minor­ity of posts or com­ments, half of us see a score higher than the real score and half see a lower score. Some­thing like +/​- per , so they still look be­liev­able and change as ex­pected as the user makes a vote. We then see how this af­fected vot­ing, and whether be­ing in­fluenced by scor­ing cor­re­lates to other fac­tors. While it’s on, users would be asked not to dis­cuss spe­cific scores.

• Great idea. One po­ten­tial prob­lem though for these sorts of ex­per­i­ments is that knowl­edge (or rea­son­able sus­pi­cion) of the ex­per­i­ments would al­ter users’ be­hav­ior.

• Yes, but I’m hop­ing us­ing a ran­domly se­lected minor­ity posts or com­ments would help, and I’d ex­pect our es­ti­ma­tions as to which posts have been raised or low­ered would be in­ter­est­ingly in­ac­cu­rate. Maybe we could sub­mit our guesses along with the prob­a­bil­ity we as­sign to each guess, then the cal­ibra­tion test re­sults could be posted… :-)

• A re­lated is­sue is that com­ments on older posts are less likely to be read, and less likely to be voted on. Ear­lier com­ments are, on av­er­age, some­what bet­ter com­ments (See one anal­y­sis of this here—http://​​www.marginalrev­olu­tion.com/​​marginalrev­olu­tion/​​2008/​​02/​​does-the-qualit.html ). But I still think the sta­tus quo is sub-op­ti­mal, es­pe­cially when most peo­ple view com­ments sorted by pop­u­lar­ity or in chronolog­i­cal or­der.

Now the fun part:

If this com­ment re­ceives pos­i­tive votes, how will that af­fect your as­sess­ment of its qual­ity?

• I don’t agree that re­hash­ing old ideas is a bad thing, es­pe­cially when good old ideas or their im­pli­ca­tions are be­ing ig­nored. Novelty is valuable, but highly over­rated.

There’s no point to the rat­ing sys­tem if we don’t ex­er­cise our eval­u­a­tions in­de­pen­dently. Tak­ing other peo­ple’s opinions into con­sid­er­a­tion may be a good way to reach a fi­nal con­clu­sion, but it’s a ter­rible way to form your own opinion.

What pre­cisely is the differ­ence be­tween be­ing a “good Bayesian” as you de­scribe it, and be­ing a group­thinker? Is it only that the Bayesian has an ex­plicit equa­tion while the group­thinker prob­a­bly doesn’t?

• The end­points 1,2 and 4 are more or less equiv­a­lent; they are worth re­peat­ing though. There isn’t re­ally any worth in a score of votes on the true qual­ity, at least not for bayesi­ans. A score of votes on in­di­vi­d­ual judg­ments would con­tain all use­ful in­for­ma­tion.

A thought ex­per­i­ment: You could use a dou­ble vot­ing sys­tem: you make one vote on your be­liefs be­fore up­dat­ing on the con­sen­sus and an­other vote in a sep­a­rate count on your up­dated be­lief. The point would be to up­date on the con­sen­sus of the first vote count and use the sec­ond vote count for all other pur­poses, eg. pro­mot­ing on the front page. This would al­low broad­cast­ing of each per­sons novel ev­i­dence (their in­di­vi­d­ual judge­ment) as well as keep­ing some kind of ag­gre­gate score for the sites al­gorithms to work with. It would prob­a­bly be easy to cre­ate an al­gorithm that makes full use of the first score though and as long as one can’t think of a good use of the sec­ond count, one shouldn’t vote on ones up­dated be­liefs in a sin­gle vote sys­tem I guess.

A minor point about the calcu­la­tions: An ideal bayesian wouldn’t do the calcu­la­tion you did. Know­ing the vot­ing pro­ce­dure, they would dis­miss any votes not con­tribut­ing new in­for­ma­tion. As the or­der of the votes isn’t pub­lic, they would have to keep a prior for the differ­ent or­ders and up­date on that. This is of course a minor quib­ble as this would lead to far too much calcu­la­tions to be a rea­son­able model for any real reader.

• “An ideal bayesian wouldn’t...” I apol­o­gize, I’m not fol­low­ing.

I was dis­miss­ing votes not con­tribut­ing new in­for­ma­tion. The or­der of the votes is partly de­duced. Re­gard­ing the part that isn’t de­duced, there is no ev­i­dence to up­date on, and the prior is in­cluded—it’s the (6:4) fac­tor.

Would you mind post­ing what the ideal bayesian’s calcu­la­tions would look like?

• [Sorry for not an­swer­ing ear­lier, I didn’t find the in­box un­til re­cently.]

I per­haps was a bit un­clear, but when I say “ideal bayesian” I mean a math­e­mat­i­cal con­struct that does full bayesian up­dat­ing i.e. in­cor­po­rates all prior knowl­edge into its calcu­la­tions. This is of course im­pos­si­ble for any­one not ex­tremely ig­no­rant of the world, which is why I called it a minor point.

An ideal bayesian calcu­la­tion would in­clude mas­sive de­duc­tive work on e.g. the psy­chol­ogy of vot­ing, knowl­edge of the func­tion­ing of this com­mu­nity in par­tic­u­lar etc.

My com­ment wasn’t re­ally an ob­jec­tion. To do a full bayesian calcu­la­tion of a real world prob­lem is com­pa­rable to us­ing quan­tum me­chan­ics for macro­scopic sys­tems. One must use ap­prox­i­ma­tions; the hard part is know­ing when they break down.

• We already have sec­tions for both pop­u­lar (up—down > thresh­old) and con­tro­ver­sial (up + down > thresh­old). Is it that posts are au­to­mat­i­cally ele­vated to these states, or does that still need to be done by mod­er­a­tors? Is the throt­tling of post ele­va­tion that EY re­cently men­tioned han­dled au­to­mat­i­cally or man­u­ally?

If ele­va­tion is han­dled man­u­ally by mod­er­a­tors, it seems it makes most sense to keep the tal­lies pri­vate, and let the mod­er­a­tors use bayesian math to ad­just for pri­ors. (I per­son­ally think that’s overkill—might make a fun au­toma­tion task how­ever.)

The only rea­son to leave them pub­lic is so peo­ple can de­cide which posts to read. There’s not enough time in the day for me to keep up with all the posts here—hell, I can barely keep up with Eliezer’s posts on OB.

In­stead, it seems they should be kept pri­vate to avoid the bi­ases pointed out.

Those like my­self will likely only ever read pop­u­lar posts—at which point it’s too late to vote (ele­va­tion has already hap­pened), and only oc­ca­sion­ally dip into the new or con­tro­ver­sial sec­tions when par­tic­u­larly bored. I ex­pect I’m in the ma­jor­ity. (Are we keep­ing stats of views vs votes? Prob­a­bly a bit early to tell at this point.)

• it seems they should be kept pri­vate to avoid the bi­ases pointed out.

Per­son­ally I find the scores quite in­ter­est­ing. I like hav­ing a sense of what other as­piring ra­tio­nal­ists read­ing alongside me are think­ing.

I’m in fa­vor of an op­tion not to see the scores, but surely those here gath­ered to over­come bias should be al­lowed to strive to do that them­selves with­out hav­ing to be pro­tected from the in­for­ma­tion.

• In other words, be aware that pop­u­lar­ity breeds pop­u­lar­ity.

• Let’s talk about how the very first reader would vote. If they judged the post high qual­ity, then they would mul­ti­ply the prior like­li­hood ra­tio (6:4) times the bayes fac­tor for a high pri­vate sig­nal (4:1), get (6 4:4 1) = (6:1) and vote the post up.

Vot­ing is a de­ci­sion, so you need util­ity in or­der to go from 6:1 to “vote up”.

• I agree. I should have said some­thing about it, but at that point I’m us­ing the as­sump­tion “Vot­ers strive to vote the true qual­ity of the post.”

• Isn’t this vot­ing busi­ness all rather ….. ju­ve­nile? Ra­tion­al­ity (what­ever it is) is not based on the sim­ple ad­di­tion of vot­ing! 70 peo­ple can vote up and 69 vote down re­sult­ing in an ob­jec­tively mea­sured mediocre post. Or 100 can read the post and move on with­out vot­ing and only two vote down—re­sult­ing in a nega­tive karma. We have a list of boasts on the side­bar—Yud­kowsky is 6 times more ra­tio­nal thanY­vain and 15 times more ra­tio­nal than me and Han­son is lag­ging be­hind. Now we know whom we need to af­fili­ate with or cheer on—Come on Han­son you need my vote. This is silly and rife with cog­ni­tive bias.

• I don’t think that the idea was that the higher your score the more ra­tio­nal you are, but I do agree that the “Top Con­trib­u­tors” thing seems to be more trou­ble than it’s worth.