Voting Phase of 2018 LW Review

For the past 1.5 months on LessWrong, we’ve been do­ing a ma­jor re­view of 2018 — look­ing back at old posts and ask­ing which of them have stood the test of time.

The LessWrong 2018 Re­view has three ma­jor goals.

  • First, it is an ex­per­i­ment in im­prov­ing the LessWrong com­mu­nity’s longterm feed­back and re­ward cy­cle.

  • Se­cond, it is an at­tempt to build com­mon knowl­edge about the best ideas we’ve dis­cov­ered on LessWrong.

  • Third, af­ter the vote, the LessWrong Team will com­pile the top posts into a phys­i­cal book.

We spent about 2 weeks nom­i­nat­ing posts, and 75 posts re­ceived the the 2 re­quired nom­i­na­tions to pass through that round. (See all nom­i­na­tions on the nom­i­na­tions page.) Then we spent a month re­view­ing them, and users wrote 72 re­views, a num­ber of them by the post-au­thors them­selves. (See all re­views on the re­views page.)

And fi­nally, as the con­clu­sion of all of this work, we are now vot­ing on all the nom­i­nated posts. Vot­ing is open for 12 days, and will close on Sun­day, Jan­uary 19th. (We’ll turn it off on Mon­day dur­ing the day, en­sur­ing all time­zones get it through­out Sun­day.)

The vote has a sim­ple first sec­tion, and a de­tailed-yet-op­tional sec­ond sec­tion based on quadratic vot­ing. If you are one of the 430 users with 1000+ karma, you are el­i­gible to vote, then now is the time for you to par­ti­ci­pate in the vote by fol­low­ing this link.

For all users and lurk­ers, re­gard­less of karma, the next 12 days are your last op­por­tu­nity to write re­views for any nom­i­nated posts in 2018, which I ex­pect will have a sig­nifi­cant im­pact on how peo­ple vote. As you can see, all re­views are high­lighted when a user is vot­ing on a post. (To re­view a post, go to the post and click “Write A Re­view” at the top of the post.)

This is the end of this post. If you’d like to read more de­tailed in­struc­tions about how to vote, the rest of the text be­low con­tains in­struc­tions for how to use the vot­ing sys­tem.

How To Vote

Sort­ing Posts Into Buckets

The first part of vot­ing is sort­ing the nom­i­nated posts into buck­ets.

The five buck­ets are: No, Neu­tral, Good, Im­por­tant, Cru­cial. Sort the posts how­ever you think is best.

The key part is the rel­a­tive weight­ing of differ­ent posts. For ex­am­ple, it won’t make a differ­ence to your fi­nal vote if you put ev­ery post in ‘cru­cial’ or ev­ery post in ‘good’.

Fine-Tun­ing Your Votes

The sys­tem we’re us­ing is quadratic vot­ing (as I dis­cussed a few weeks ago).

Once you’re happy with your buck­ets, click ‘Con­vert to Quadratic’. At this point the sys­tem con­verts your buck­ets roughly into their quadratic equiv­a­lents.

The sys­tem will only as­sign in­te­ger num­bers of votes, which means that it will likely only al­lo­cate around 80-90% of the to­tal votes available to you. If you vote on a smaller num­ber of posts (<10), the au­to­matic sys­tem may not use your en­tire quadratic vot­ing bud­get.

If you’re happy with how things look, you can just leave at this point, and your votes will be saved (you can come back any time be­fore the vote closes to up­date them). But if you want to al­lo­cate 100% of your available votes, you’ll likely need to do fine-tun­ing.

There are two key parts of quadratic vot­ing you need to know:

  • First, you have a limited bud­get of votes.

  • Se­cond, votes on a post have in­creas­ing marginal cost.

This means that your first vote costs 1 point, your sec­ond vote on that post costs 2 points, your third costs 3 points. Your nth vote costs n points.

You have 500 points to spend. You can see how many points you’ve spent at the top of the posts.

The sys­tem will au­to­mat­i­cally weight the buck­ets differ­ently. For ex­am­ple, I just did this, and I got the fol­low­ing weight­ings:

  • Good: 2 votes.

  • Im­por­tant: 4 votes.

  • Cru­cial: 9 votes.

  • Neu­tral: 0 votes.

  • No: −4 votes.

(Note that nega­tive votes cost the same as pos­i­tive votes. The first nega­tive vote costs 1 point, the sec­ond nega­tive vote costs 2 points, etc.)

You’ll see your score at the top of the page. (When I ar­rived on the fine-tun­ing page, the sys­tem had spent about 416 points, which meant I had a sig­nifi­cant num­ber of votes left to buy.)

Once you’re happy with the bal­ance, just close the page; your votes will be saved.

You can re­turn to this page any­time un­til vot­ing is over, to re­con­figure your weights.

Leav­ing Com­ments (Anony­mous)

There’s a field to leave anony­mous thoughts on a post. All com­ments writ­ten here will be put into a pub­lic Google doc, and be linked to from the post that an­nounces the re­sult of the vote. If you want to share your thoughts, how­ever briefly, this is a space to do that.

I will likely be mak­ing a book of 2018 posts, and if I do I will use the votes as my guide to what to in­clude, so I’ll definitely be in­ter­ested in read­ing through peo­ple’s anony­mous thoughts and feel­ings about the 2018 LW posts.

Ex­tra Notes

Ad­di­tional info on voting

If you’d like to go back to the buck­ets stage, hit “Re­turn to ba­sic vot­ing”. If you do this, all of your fine-tun­ing will be thrown out the win­dow, and the sys­tem will re-calcu­late your weights en­tirely based on the new buck­ets you as­sign.

I find it re­ally valuable to be able to see the posts in the or­der I’ve cur­rently ranked them, so there’s a but­ton at the top to re-or­der the posts, which I ex­pect to be clicked dozens of times by each user.

If you click on a post, all nom­i­na­tions and re­views for that post will ap­pear in a box on the side of the page. You may want to read these to make a more in­formed de­ci­sion when vot­ing.

The results

The vot­ing will have many out­puts. Once we’ve had the time to analyse the data, we’ll in­clude a bunch of data/​graphs, all anonymised, such as:

  • For each win­ning post and each bucket, how many times peo­ple picked that bucket

  • The in­di­vi­d­ual votes each post got

  • The re­sults if the karma cut­off was 10,000 karma rather than 1,000

  • The out­put of peo­ple’s buck­ets com­pared with the out­put of peo­ple’s quadratic fine-tuning

  • The mean and stan­dard de­vi­a­tion of the votes

Vote Here (if you have more than 1000 karma)