2019 Review: Voting Results!

You can see the full voting results here: 1000+ karma voters (All voters)

The 2019 Review votes are in!

This year, 88 voters participated, evaluating 116 posts. (Of those voters, 61 had 1000+ karma, and will be weighted more highly in the moderation team’s decision of what to include in the Best of 2019 Books)

The LessWrong Moderation team will be reflecting on these results and using them as a major input into “what to include in the 2019 books.”

Top Results

The top 15 results from the 1000+ karma users are:

  1. What failure looks like by Paul Christiano

  2. Risks from Learned Optimization: Introduction, by evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

  3. The Parable of Predict-O-Matic, by Abram Demski

  4. Book Review: The Secret Of Our Success, by Scott Alexander

  5. Being the (Pareto) Best in the World, by Johnswentworth

  6. Rule Thinkers In, Not Out, by Scott Alexander

  7. Book summary: Unlocking the Emotional Brain, by Kaj Sotala

  8. Asymmetric Justice, by Zvi Mowshowitz

  9. Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists, by Zack Davis

  10. 1960: The Year The Singularity Was Cancelled, by Scott Alexander

  11. Selection vs Control, by Abram Demski

  12. You Have About Five Words, by Raymond Arnold

  13. The Schelling Choice is “Rabbit”, not “Stag”, by Raymond Arnold

  14. Noticing Frame Differences, by Raymond Arnold

  15. “Yes Requires the Possibility of No”, by Scott Garrabrant

Top Reviewers

Meanwhile, we also had a lot of great reviews. One of the most valuable things I found about the review process was that it looks at lots of great posts at once, which led me to find connections between them I had previously missed. We’ll be doing a more in-depth review of the best reviews later on, but for now, I wanted to shoutout to the people who did a bunch of great review work.

The top reviewers (aggregating the total karma of their review-comments) were:

Some things I particularly appreciated were:

  • johnswentworth, Zvi and others providing fairly comprehensive reviews of many different posts, taking stock of how some posts fit together.

  • Jacobjacob and magfrump who stuck out in my mind for doing particularly “epistemic spot check” type reviews, which are often more effortful.

Complete Results (1000+ Karma)

You can see the full voting results here: 1000+ karma voters (All voters)

To help users see the spread of the vote data, we’ve included swarmplot visualizations.

  • Only votes with weights between −10 and 16 are plotted. Outliers are in the image captions.

  • Gridlines are spaced 2 points apart.

  • Concrete illustration: The plot immediately below has 18 votes ranging in strength from −3 to 12.

1What failure looks like
2Risks from Learned Optimization: Introduction
3The Parable of Predict-O-Matic
4.5Being the (Pareto) Best in the World
4.5Book Review: The Secret Of Our Success
6Rule Thinkers In, Not Out
7Book summary: Unlocking the Emotional Brain
Outlier: +20
8.5Asymmetric Justice
8.5Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists
101960: The Year The Singularity Was Cancelled
11.5Selection vs Control
11.5You Have About Five Words
13The Schelling Choice is “Rabbit”, not “Stag”
14Noticing Frame Differences
15Yes Requires the Possibility of No
16″Other people are wrong” vs “I am right”
17Rest Days vs Recovery Days
18.5Seeking Power is Often Robustly Instrumental in MDPs
18.5The Forces of Blandness and the Disagreeable Majority
20.5The Costs of Reliability
20.5Chris Olah’s views on AGI safety
22Reframing Superintelligence: Comprehensive AI Services as General Intelligence
23.5Humans Who Are Not Concentrating Are Not General Intelligences
23.5The strategy-stealing assumption
26.5Reframing Impact
26.5Understanding “Deep Double Descent”
26.5Moloch Hasn’t Won
26.5Integrity and accountability are core parts of rationality
30.5Gears-Level Models are Capital Investments
30.5In My Culture
30.5Make more land
30.5Forum participation as a research strategy
33Unconscious Economics
34.5Mistakes with Conservation of Expected Evidence
36The Tale of Alice Almost: Strategies for Dealing With Pretty Good People
37Excerpts from a larger discussion about simulacra
38human psycholinguists: a critical appraisal
40AI Safety “Success Stories”
40Do you fear the rock or the hard place?
40Propagating Facts into Aesthetics
42Gradient hacking
44The Amish, and Strategic Norms around Technology
44Power Buys You Distance From The Crime
44Paper-Reading for Gears
48.5How to Ignore Your Emotions (while also thinking you’re awesome at emotions)
48.5The Real Rules Have No Exceptions
48.5Coherent decisions imply consistent utilities
48.5Alignment Research Field Guide
48.5The Curse Of The Counterfactual
52.5The Credit Assignment Problem
52.5Reason isn’t magic
54Mental Mountains
56.5Simple Rules of Law
56.5Utility ≠ Reward
56.5Is Rationalist Self-Improvement Real?
56.5Literature Review: Distributed Teams
59Steelmanning Divination
60Book Review: Design Principles of Biological Circuits
61Building up to an Internal Family Systems model
62Evolution of Modularity
63[Answer] Why wasn’t science invented in China?
64How Much is Your Time Worth?
65.5Book Review: The Structure Of Scientific Revolutions
65.5Everybody Knows
68.5Sequence introduction: non-agent and multiagent models of mind
68.5From Personal to Prison Gangs: Enforcing Prosocial Behavior
68.5Some Ways Coordination is Hard
68.5Circle Games
71Why Subagents?
73.5Healthy Competition
73.5Where to Draw the Boundaries?
73.5Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?
73.5Six AI Risk/​Strategy Ideas
76.5Thoughts on Human Models
76.5Classifying specification problems as variants of Goodhart’s Law
79Book Summary: Consciousness and the Brain
79Gears vs Behavior
79Strategic implications of AIs’ ability to coordinate at low cost, for example by merging
82Book Review: Secular Cycles
82Some Thoughts on My Psychiatry Practice
82Coordination Surveys: why we should survey to organize responsibilities, not just predictions
84.5The Hard Work of Translation (Buddhism)
84.5[Part 2] Amplifying generalist research via forecasting – results from a preliminary exploration
86.5Soft takeoff can still lead to decisive strategic advantage
86.5Total horse takeover
88.5Complex Behavior from Simple (Sub)Agents
88.5Less Competition, More Meritocracy?
90.5Megaproject management
90.5But exactly how complex and fragile?
92.5Trauma, Meditation, and a Cool Scar
92.5Turning air into bread
95AlphaStar: Impressive for RL progress, not for AGI progress
95Dishonest Update Reporting
95What are the open problems in Human Rationality?
97.5Integrating the Lindy Effect
97.5mAIry’s room: AI reasoning to solve philosophical problems
99S-Curves for Trend Forecasting
100.5Partial summary of debate with Benquo and Jessicata [pt 1]
100.5The Power to Teach Concepts Better
102.5Instant stone (just add water!)
102.5Autism And Intelligence: Much More Than You Wanted To Know
104Relevance Norms; Or, Gricean Implicature Queers the Decoupling/​Contextualizing Binary
105Rationality, Levels of Intervention, and Empiricism
106The Zettelkasten Method
107Two explanations for variation in human abilities
108No, it’s not The Incentives—it’s you
Outlier: −11
109Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think
110The Power to Demolish Bad Arguments
111Neural Annealing: Toward a Neural Theory of Everything (crosspost)
Outlier: −20
112Dual Wielding
113Approval Extraction Advertised as Production
Outlier: −15
114The AI Timelines Scam
Outlier: −13
115Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More

What does this mean, and what happens now?

(This section written by habryka, previous section written by Ray)

The goals of this review and vote were as follows:

  1. Create common knowledge about how the LessWrong community feels about various posts and the progress we’ve made.

  2. Improve our longterm incentives, feedback, and rewards for authors.

  3. Help create a highly curated “Best of 2019” Sequence and Book.

Over the next few months we will take the results of this vote and make it into another curated collection of essays, just as we did with last years results, which turned into the “A Map that Reflects the Territory” essay collection.

Voting, review and nomination participation was substantially greater this year than last year (something between a 30% and 80% increase, depending on which metrics you look at), which makes me hopeful about this tradition living on as a core piece of infrastructure for LessWrong. I was worried that participation would fall off after the initial excitement of last year, but I am no longer as worried about that.

Both this year and last year we have also seen little correlation with the vote results and the karma of the posts, which is an important sanity check I have for whether going through all the effort of this review is worth it. If the ranking was basically just the same as the karma scores of the post, then we wouldn’t be getting much information out of the review. But as it stands, I trust the results of this review much more than I would trust someone just pressing the “sort by karma” button on the all-posts page, and I think as the site and community continues to grow, the importance of the robustness of the review will only grow.

Thank you all for participating in this year’s review. I am pleased with results, and brimming with ideas for the new set of books that I am looking forward to implementing, and I think the above is already a valuable resource if someone wants to decide how to best catch up with all the great writing here on the site.