One Website To Rule Them All?

Epistemic sta­tus: I’m not the first per­son to think of this, and I’m not sure if this is pos­si­ble. But I want to flesh out some op­tions for con­sid­er­a­tion and get feed­back. Sorry if it’s long.

WANTED:

A crowd-sourced site that re­li­ably pre­sents im­por­tant pros and cons on any/​all top­ics and hope­fully leads us col­lec­tively closer to con­sen­sus.

This has been on my mind for a few years now. Figur­ing out truth in the digi­tal age is of­ten tricky and some­times down­right im­pos­si­ble. (Is min­i­mum wage a good thing? Should I adopt a pa­leo or keto or ve­gan or Shangri-la diet? What do we re­ally know and not know about the his­tor­i­cal Je­sus?) Espe­cially when it comes to poli­ti­cal dis­cus­sion, an­swers of­ten de­pend on more de­tails than an av­er­age per­son has time or en­ergy to re­search—and even if they did, they may still leave out con­sid­er­a­tions or solu­tions that some­one else may think of.

Some links where other ra­tio­nal­ists think semi-re­lated things: Robin Han­son wants AI to help us to­wards con­sen­sus be­cause Wikipe­dia isn’t up for the job. Scott Alexan­der pro­moted ad­ver­sar­ial col­lab­o­ra­tions to see if they could reach con­sen­sus. (I swear, I saw more ex­am­ples of peo­ple wish­ing for some­thing like a de­bate web­site, but I wasn’t writ­ing them down and search­ing for them has failed.)

THE WISHFUL THINKING PART

Now let me be more spe­cific about what my imag­ined web­site (here­after, Site) needs to do.

On mat­ters of truth, it needs to sup­port epistemic ar­gu­ments for why we should be­lieve or not be­lieve par­tic­u­lar claims. On mat­ters of ac­tion, it needs to provide im­por­tant pro/​cons of tak­ing that ac­tion. Site must have a method of al­low­ing the best ar­gu­ments to rise to the top. That’s the most ba­sic func­tion­al­ity.

Ad­di­tional Fea­ture A: Be­cause the Site is crowd-sourced, any­one can pro­pose pos­si­ble ac­tions and pos­si­ble mod­ifi­ca­tions to ac­tions. In the­ory, this could mean that Site could be a plat­form where a crowd-writ­ten law could have its de­tails ham­mered out by many af­fected par­ties. Creative solu­tions could have a chance to be re­viewed and, if well-thought-out, could rise higher in the pub­lic con­scious­ness.

Op­tional Fea­ture B: I re­ally, re­ally would like it if, within the struc­ture of ar­gu­ments (for ei­ther epistemic-truth top­ics or pro­posed-ac­tions), Site dis­t­in­guished be­tween truth-claims and value-claims. This might be es­pe­cially un­re­al­is­tic, since no one nat­u­rally lays out ar­gu­ments that way. But gosh dang it, I re­ally want it to.

Op­tional Fea­ture C: A num­ber of ra­tio­nal­ist posts have dis­cussed how use­ful it would be for so­ciety to have a way to co­or­di­nate on cer­tain prob­lems: speci­fi­cally, hav­ing a con­di­tional “I will adopt X if Y% of other rele­vant peo­ple also do”. It might be pos­si­ble to set up Site so that pro­posed ac­tions that need a min­i­mum num­ber of peo­ple to adopt it in or­der for it to be worth­while offer the op­tion of con­di­tion­ally com­mit­ting to adopt it and send­ing out no­tifi­ca­tions when thresh­olds are reached.

THE NITTY-GRITTY

Ob­vi­ously, this is a LOT to ask from a web­site. Be­fore I go into ev­ery­thing that could go wrong and all the rea­sons it might not even be pos­si­ble, let me lay out a cou­ple of op­er­a­tional op­tions.

Con­trast Voting

Red­dit and other up/​down-vote sys­tems suffer from cer­tain effects, an im­por­tant one be­ing that late-to-the-game com­ments don’t stand a chance against early pop­u­lar com­ments, even if the later ones are higher qual­ity. To com­bat this, Site should not al­low up/​down votes on in­di­vi­d­ual items/​com­ments/​what­ever. In­stead, Site should pre­sent two items and ask for a vote on which one is bet­ter. In this way, Site can give bet­ter rank­ings to items. A new ar­gu­ment, per­haps one that pre­sents a new, more re­cent study find­ing, can quickly out­rank the pre­vi­ous top ar­gu­ment if, when those two ar­gu­ments are pre­sented, most peo­ple vote that the new one is bet­ter.

Con­trast Topics

The Con­trast Topics op­tion would not have a page ti­tled “Is God real?” nor a page for “We should out­law chicken cage farm­ing.” In­stead, top­ics would be pre­sented in terms of pos­si­ble al­ter­na­tives. “The Abra­hamic God is real” could be sep­a­rately con­trasted with “There is no God” and, on an­other page, con­trasted with “There is a di­v­ine en­ergy”. “We should out­law chicken cage farm­ing” could be con­trasted with “We should eat less chicken and let the mar­ket ad­just chicken cage farm­ing down­ward” or “We should slowly phase out chicken cage farm­ing ac­cord­ing to this de­tailed plan I’ve writ­ten out” or “We should wait un­til chicken-like lab meat is available, and then out­law chicken cage farm­ing”.

Vi­su­ally, this might be tricky, but a “Chicken cage farm­ing” page might list all of the pro­posed ac­tions re­lated to it, and a user could se­lect two ac­tions and click “next” to see the pros and cons of those two ac­tions rel­a­tive to each other. (Th­ese would be more like pros of A vs pros of B, not pro/​cons of each; a con of B would be listed as a pro of A and vice versa.) Th­ese head­ings might be treated as tags/​la­bels rather than a strict hi­er­ar­chy of cat­e­gories, so that a pos­si­ble ac­tion or topic might ap­pear un­der mul­ti­ple head­ings if it’s rele­vant to all of them. Head­ings might, in turn, have both epistemic truth-claims and po­ten­tial ac­tions listed un­der them.

Open Forum

In this op­tion, in­di­vi­d­ual users com­pose com­plete ar­gu­ments, ei­ther pro or con. They are al­lowed and even en­couraged to steal what oth­ers have writ­ten, add/​sub­tract/​mod­ify it as they like, and sub­mit it as a new ar­gu­ment. Con­trast Vot­ing is used within “pro” and within “con” to rank these ar­gu­ments and pre­sent the high­est ranked at the top. Prob­a­bly only the top one or two sub­mis­sions will show by de­fault (the rest reached by an ex­pand­able + or “show more” or such), since sub­mis­sions are ex­pected to (even­tu­ally) ag­gre­gate all the best ar­gu­ments.

Wikibate

This is an al­ter­na­tive to Open Fo­rum. It works more like Wikipe­dia; ev­ery­one is al­lowed to di­rectly edit one sin­gle page. A pro and con side are pro­vided, and we see if a ver­sion comes out that makes enough peo­ple happy enough to be sta­ble. A be­hind-the-scenes con­trast vot­ing op­tion might be used to de­cide which or­der the pos­si­ble ar­gu­ments ap­pear in, but the phras­ing of the ar­gu­ments would have to satisfy peo­ple or be ed­ited by some­one.

Per­sonal Ex­per­tise Stories

In nor­mal life, peo­ple rely heav­ily on per­sonal ex­pe­riences when form­ing their be­liefs. If Site doesn’t some­times take this into ac­count, it will feel shal­low to many peo­ple. Ar­gue the facts of abor­tion all you want, but if you can’t in­clude some­one’s vivid de­scrip­tion of hav­ing a loved one die from a back-alley abor­tion or an­other per­son’s hor­ror at see­ing what they were told is a “lump of tis­sue” that turns out to look ex­actly like a baby, you’re miss­ing im­por­tant con­text.

It is also the case that some­times hear­ing ex­pert tes­ti­mony is helpful. Law­mak­ers of­ten rely on it. Ex­perts don’t always agree, but a claim from an ex­pert can and should carry a differ­ent weight than the same claim from a non-ex­pert.

On a crowd-sourced site, any­one can claim they’re an ex­pert or make up b.s. sto­ries about what hap­pened to them. A pos­si­ble solu­tion is to al­low peo­ple to sub­mit a “per­sonal ex­per­tise/​story”. If sub­mit­ting one, you have to in­clude per­sonal con­tact info like a phone num­ber. Some num­ber, say 10, peo­ple can ap­ply to ver­ify a story. Only ap­proved ver­ifiers are given ac­cess to the story sub­mit­ter’s con­tact info. The ver­ifiers promise to come back and re­port on the sta­tus of their ver­ifi­ca­tion and not to abuse the con­tact info. (A com­plaint from a story sub­mit­ter re­sults in that ver­ifier be­ing un­able to do any fur­ther ver­ifi­ca­tions, and pos­si­bly banned from the Site.) The ver­ifier calls the per­son and talk to them, get de­tails if pos­si­ble, like other peo­ple or com­pa­nies they can talk to, to ver­ify parts of the story or claim. In the end they re­port back, maybe on a 5-point scale, whether they be­lieve the per­son’s claim, and the net ver­ifi­ca­tion re­sult from all the ver­ifiers is dis­played with the story. (E.g. a green bar show­ing 3.5/​5, with a (2) next to it in­di­cat­ing only 2 ver­ifiers have sub­mit­ted re­sponses.)

Con­se­quences Be­fore Arguments

For pro­posed ac­tions, be­fore the ac­tual pro/​con ar­gu­ments ap­pear, I would like to see a list of pos­si­ble con­se­quences that might re­sult from tak­ing that ac­tion. While I don’t want Site to adopt con­se­quen­tial­ism as an offi­cial philos­o­phy, I re­mem­ber (Google, you fail me) an ar­ti­cle that claimed that peo­ple demon­strated less par­ti­san­ship when they were asked to think about the con­se­quences of pos­si­ble laws. A po­ten­tial con­se­quence is it­self a truth-claim (“If we do X, then Y will hap­pen”), so that should link to its own page.

Lo­gin Voting

Gen­er­ally, I want peo­ple not to have to cre­ate ac­counts in or­der to par­ti­ci­pate on the web­site. IP ad­dresses can be recorded, like Wikipe­dia does, for sub­mit­ting/​mod­ify­ing ar­gu­ments, propos­ing ac­tions, and cre­at­ing top­ics. That should min­i­mize the bar­rier to en­try and en­courage par­ti­ci­pa­tion. How­ever, all vot­ing should re­quire log­ging in, to min­i­mize one per­son vot­ing mul­ti­ple times, and to ac­cu­rately track if some­one changes their rel­a­tive rank­ings of two ar­gu­ments.

Par­allel Axes Voting

There is no one stan­dard for what makes an ar­gu­ment “bet­ter” than an­other. Rele­vant mea­sure­ments in­clude: ac­cu­racy, im­por­tance/​ap­pli­ca­bil­ity, thor­ough­ness, kind­ness, for­mat­ting/​gram­mar/​spel­ling. One op­tion might be to al­low users to Con­trast Vote two ar­gu­ments in mul­ti­ple cat­e­gories sep­a­rately: Ar­gu­ment A has more im­por­tant points than B, but B has bet­ter gram­mar than A, and they are the same on kind­ness. Par­allel Axes Vot­ing is more rele­vant if Site uses the Open Fo­rum style, and not so much if it uses Wik­i­bate.

THE PROBLEMS, OH SO MANY PROBLEMS

  • I don’t know how to make this web­site. I know a lit­tle html and css, and I have a vague idea that Ama­zon Web Ser­vices could be ini­tially used to host the site. But I know noth­ing about how to use AWS, how to make a database and a web­site talk to each other, how to have user ac­counts and se­cure lo­gins, etc. I’d ei­ther need a lot of help or some­one else would have to do it en­tirely or I’d need to in­vest a ton of time into ba­si­cally learn­ing a new pro­fes­sion to make some­thing that might fail.

  • The most pop­u­lar ar­gu­ments aren’t nec­es­sar­ily the best.

    • Peo­ple might sac­ri­fice truth for sim­plic­ity. Given two ar­gu­ments that make the same point, one that pre­sents the point more sim­ply and un­der­stand­ably is bet­ter than one that doesn’t. How­ever, users will some­times be offered the choice be­tween a sim­ple, un­der­stand­able ar­gu­ment that isn’t ac­cu­rate and a difficult one that is more cor­rect. I don’t know of a way to dis­cour­age up­vot­ing the sim­pler one over the more ac­cu­rate one. Worse, if all the ar­gu­ments on the in­ac­cu­rate side are sim­ple and easy to un­der­stand, and all the ar­gu­ments on the more cor­rect side are difficult to un­der­stand, Site could back­fire and have the effect of con­vinc­ing peo­ple of some­thing that isn’t true.

    • Some top­ics re­quire a whole back­ground course to un­der­stand. Is­rael/​Pales­tine, the mort­gage crisis...there’s some things that re­quire so much back­ground knowl­edge (even to be fa­mil­iar with the rele­vant terms) that I’m not sure it’s pos­si­ble to cre­ate ar­gu­ments that the av­er­age reader could un­der­stand with­out also pre­sent­ing some sort of back­ground course on the topic. [Maybe this could be ame­lio­rated by limit­ing the vot­ing on cer­tain top­ics to peo­ple who have ac­cessed and agreed that they have read a back­ground page that ver­ified ex­perts have ap­proved?]

  • I don’t know if Site will ac­tu­ally pro­mote con­sen­sus. I think see­ing your op­po­nent’s best ar­gu­ments in­stead of only their worst will re­duce par­ti­san­ship a lit­tle. I think that com­par­ing mul­ti­ple pos­si­ble ac­tions on a topic in­stead of only one or two will help some. But that might not be enough.

    • Hav­ing all ar­gu­ments di­vided into ei­ther pro or con in­stead of one sin­gle nar­ra­tive might not shift peo­ple into a con­sen­sus. Re­lat­edly, it might be difficult for the two sides to ad­e­quately in­ter­act with each other and re­spond to each other’s views when ev­ery­thing is listed on one side or the other.

    • See­ing the pros and cons of both sides might ac­tu­ally make un­true be­liefs (flat Earth, for ex­am­ple) seem more le­gi­t­i­mate or rea­son­able than they are; the epistemic im­bal­ances might not come across effec­tively.

    • Much dis­agree­ment in­cludes trust­ing sources differ­ently. If an ar­gu­ment about whether some­thing did/​would hap­pen or not boils down to “Bre­it­bart/​Slate said so”, Site might not have a way to re­solve that.

    • A web­site might be in­nately in­suffi­cient; per­son­ally car­ing about an­other per­son you know might be re­quired for con­sen­sus.

    • A sub-point on this one is that in the Wik­i­bate style, I don’t know if ar­gu­ments can reach a rea­son­ably sta­ble state. Wikipe­dia does some­times have those back­ground pages for ar­gu­ing about whether an ar­ti­cle should say one thing or an­other; Wik­i­bate might try those for set­tling dis­putes, de­cid­ing whether sub­tle differ­ences are similar enough to count as the same thing or not, and such. But con­tentious things are con­tentious, and that might not be enough.

  • Some al­gorithms and UI de­tails need to be worked out.

    • How can the con­di­tional-co­op­er­a­tion op­tion be im­ple­mented? If differ­ent peo­ple have differ­ent thresh­olds that need to be met be­fore they will do it, how does Site take that into ac­count? How does it han­dle iden­ti­fy­ing the sub­pop­u­la­tion of the whole planet that would need to con­di­tion­ally com­mit to cer­tain mea­sures? (E.g. all cor­po­ra­tions will fol­low some en­vi­ron­men­tal rule if all the oth­ers do too—the CEOs or other higher-ups in the cor­po­ra­tions would be the only rele­vant pop­u­la­tion that would need to meet the thresh­old, not ev­ery in­di­vi­d­ual on the planet).

    • What for­mula should Site use for tak­ing user’s Con­trast Votes and turn­ing them into an over­all rank? What hap­pens when peo­ple be their in­con­sis­tent selves and like A bet­ter than B, B bet­ter than C, but C bet­ter than A?

    • How do we get users to rank new sub­mis­sions? Do we list new ones sep­a­rately above the already-ranked ones? Force a pop-up that asks peo­ple to rank a newer sub­mis­sion with a ran­dom older one be­fore they con­tinue read­ing that page? Offer a side­bar that says “Here’s some new ar­gu­ments, please re­view and rank them:“?

    • Num­bers. This is es­pe­cially rele­vant when list­ing pos­si­ble con­se­quences of ac­tions, but some­times Site needs to han­dle num­bers del­i­cately. “One more per­son each year will die if we do this” is differ­ent from “100,000 more peo­ple will die each year if we do this”. List­ing all the pos­si­ble es­ti­mates that in­di­vi­d­u­als pull out of their...err, hat would be over­whelming to look at and con­sider. Do we cre­ate a sep­a­rate page for ar­gu­ments over what the num­ber will be and in­sti­tute Con­trast Vot­ing on the re­sults so that the origi­nal page shows the most pop­u­lar es­ti­mate? List up­per and lower bounds: “Between 1 and 100,000 peo­ple will die”?

    • How does Site han­dle nest­ing? How far down do we let nest­ing con­tinue be­fore de­cid­ing that two op­tions are too similar to bother list­ing them sep­a­rately? Does Site make you com­pare only the low­est-nested level with each other, or you com­pare higher nested lev­els too? (That is, is there a page for Abra­hamic God vs. No God and also a page for prayer-an­swer­ing Bap­tist God vs. No God, even though prayer-an­swer­ing Bap­tist God is nested un­der Abra­hamic God? Or can you only com­pare prayer-an­swer­ing Bap­tist God to non-so-in­ter­ven­tion­ist Bap­tist God? If so, how do you get agree­ment on what as­pects of God all the Abra­ham­ics agree on?)

    • Is a sum­mary pos­si­ble? Would it be pos­si­ble for a page to pre­sent a sort of con­sen­sus sum­mary? (Some­thing like “Peo­ple pre­fer im­me­di­ate out­law­ing of chicken cage farm­ing if they value re­duc­ing the to­tal amount of suffer­ing of liv­ing crea­tures more than they value re­duc­ing hu­man suffer­ing alone, and peo­ple pre­fer grad­ual phas­ing out of chicken cage farm­ing if they value re­duc­ing hu­man suffer­ing alone more than they value re­duc­ing the to­tal amount of suffer­ing of liv­ing crea­tures”.)

    • How does Site bal­ance cov­er­ing all the op­tions with pre­sent­ing a num­ber of pos­si­bil­ities that peo­ple will ac­tu­ally read? Some­times it will be enough to list all the op­tions, put the most im­por­tant/​rele­vant ones at the top, and let peo­ple read as far down as they want. Some­times that might not be enough. For in­stance, if you want to list the con­se­quences of a pos­si­ble ac­tion be­fore you list the pro/​con ar­gu­ments for that ac­tion, then what do you do if there’s 50 con­se­quences pro­posed? Cut it off at an ar­bi­trary num­ber (say, only show the 5 most im­por­tant)? Dis­play any that, say, 50% of peo­ple vote should be visi­ble with­out be­ing hid­den be­hind a “see more” op­tion?

    • How is mod­er­a­tion han­dled? Like, I have no clue. This prob­a­bly varies de­pend­ing on whether Site uses Open Fo­rum or Wik­i­bate.

    • There’s a lot more de­tails to work out. I think I have some im­plicit imagery in my head as to how Site looks and op­er­ates that I might not have laid out ex­plic­itly here, but a lot of it has yet to be worked out. And doubtless huge new prob­lems will be en­coun­tered once ac­tu­ally try­ing to make the site.

  • Get­ting Site to be pop­u­lar enough to be use­ful is difficult. I definitely don’t have the so­cial net­work or fol­low­ing to pull some­thing like that off. I don’t re­ally know any­one who does. The best I could do is email a link (once it’s set up) to ran­dom fa­mous peo­ple and say “Please check this out”. Maybe if some­one writes up a “6 Rea­sons to Check Out This Web­site (And Laugh)” ar­ti­cle and sub­mits it to Buz­zfeed.

  • Fund­ing. If it does be­come pop­u­lar, de­ci­sions will need to be made about ad­ver­tis­ing and how to pay for server main­te­nance and what­not.

  • Site needs a cool name. All I’ve come up with is Wik­i­bate (if it’s set up that way) or Bet­terThink.

In the end, de­spite the prob­lems, I think Site would be more worth hav­ing than not.

So what do you think? Is Site even pos­si­ble? Is there a bet­ter setup than Open Fo­rum or Wik­i­bate? If not, which of those two is bet­ter? Should the Per­sonal Ex­per­tise Sto­ries op­tion be in­cluded? What other de­tails could Site im­ple­ment to be suc­cess­ful? Are any of you will­ing to work on it? Is it worth my time to work on it?