I’m an admin of this site; I work full-time on trying to help people on LessWrong refine the art of human rationality.
Longer bio: www.lesswrong.com/posts/aG74jJkiPccqdkK3c/the-lesswrong-team-page-under-construction#Ben_Pace___Benito
Ben Pace(Ben)
Karma: 16,151
Hey guicost. This is a great first post, thanks for the fascinating doc!
For ease of readability for users, I’ve copied the full doc into the post (I’m an admin so I have the power to edit people’s posts). If you disprefer that for any reason, please let me know right away and I’ll undo the change, or you can just delete everything yourself. I spent <15 mins on it, it’s no big deal.
The only edits I made were to put all the footnotes at the bottom (we don’t have superscript so footnotes have the slightly uglier [n] styling) and some of the images were a bit funny when I brought them over (e.g. some of them became bigger images with other stuff in them) so I just inserted screenshots instead. Again, it’s fully your post, and you are free to make whatever further edits you like.
Thx, edited.
I have some sense to engage you on particular posts, but I don’t know what would be compelling/cruxy for you.
I could say that Anna’s is in some ways like Swentworth‘s recent post “Making Vaccine”, not in being notably successful/groundbreaking but for being a move in an important direction to be rewarded — I think making your own vaccine is relatively easy and I am facepalming that I did not try to make a proper mRNA vaccine back in April. Similarly, I think Anna’s post is correctly moving from a common naive consequentialist refrain that I think is very damaging and contrasting it with a virtue ethics perspective that I think is a healthy remedy, and I regularly see people failing to live up to virtues when faced with naive consequentialist reasoning. No, it was not especially rigorous or especially brilliantly communicated like it was Tim Urban explaining how Neuralink works. But I think that there’s space for rigorous, worked out research like Cartesian Frames or Radical Probabilism, as well as off-the-cuff ideas like the PR/Honor one.
Or I could talk about how valuable new ideas have been explained and built on and discussed. I could talk about Mesa-Optimizers and then follow-on work where people have done Explain Like I’m 12. I could talk about discussion building on the Simulacra Levels ideas that I think LW has helped move along (although I expect you’ll point out that many of the people writing on it like Benquo, Zvi, and Elizabeth have their own blogs). I could talk about the time Jameson Quinn spent a month or two writing up a research question he had in voting theory and a commenter came in and solved it. I don’t know if you’ll find this stuff compelling, in each case I can imagine a reason to not be excited. But in my mind this is all contributions to our understanding of rationality and how minds work, and I think it’s pretty positive. And maybe you’ll agree and just think it’s nowhere near enough progress. And on that I might even agree with you, and would say I am planning something fairly more ambitious than this in the longer term.
The single best thing on LessWrong 2.0 so far I’d say is the Embedded Agency sequence. I think this was a lot of work done primarily by Scott and Abram (employed by MIRI), and I think LessWrong gave it a home and encouraged Abram to do it in the cartoon style (after the hit success of An Untrollable Mathematician) which I think improved it massively, making it more Feynman-esque in its attempt at simplicity, and would have probably stayed in the long drought of editing for far longer, and had the LW audience not been around for it and been read far less and built on far less. I would call this a big deal and a major insight. That would be somewhat cruxy for me and I’d be overall quite surprised if I came to think it didn’t represent philosophical progress in our understanding of rationality and LessWrong hadn’t helped it (and follow-up work like this and this) get written up well.
Added: You’re right, it wasn’t a babble, it was quite thoughtful. Edited.
“SuperDuperMetaHonestEpistemicOpus”
“If you try to Glomarize you will be too verbocious”
And if I’m honest about being SuperMetaHonest, then I’m: SuperDuperMetaHonest.
Good point.
In Eliezer’s defense I’ll note that the original proposal took pains to say “At least as honest as an unusually honest person AND THEN also truthful in communicating about your meta-level principles about when you’ll lie”, so the above isn’t a literal following of what Eliezer said (because I don’t think an unusually honest person would write that). But I think that was not a very natural idea, and I mostly think of meta-honesty as about being honest on the meta level, and that it’s important, but I don’t think of it as really tied up with the object level being super honest.
That sort of thing does encourage us to record the event, but I think we still wouldn’t have recorded the 3+ hours of discussion afterwards, some of which leaned into being more personal or private. I got a lot out of that convo.
I appreciate your thoughtful list of changes. But I don’t agree that weighted voting is bad. Overall I see the following things happening: 1) new writers coming to the site, writing excellent posts, getting reputation, and going on to positively shape the site’s culture (e.g. Alkjash, Wentworth, TurnTrout, Daniel Kokotajlo, Evan Hubinger, Lsusr, Alex Flint, Jameson Quinn, and many more) 2) very small amounts of internal politicking 3) very small amounts of brigading from external groups or the dominant culture. I agree that setting weighted karma is a strong bet on the current culture (or the culture at the time) being healthy and being able to grow into good directions, and I think overall that bet seems to be going fairly well.
I don’t want most people on the internet to have an equal vote here, I want the people who’ve proven themselves to have more say.
I do think that people goodhart on short-term rewards (e.g. karma, number of comments, etc) and to build more aligned long-term incentives the team has organized the annual review (2018, 2019) (the result of which notably does not just track post-karma) and we have published the top 40 posts in a professionally designed book set.
I agree the theoretical case would be pretty compelling alone, and I agree that some part of you should always be terrified by group incentive mechanisms in your environment, but I don’t feel the theoretical case here is strong and I think ‘what I see when I look’ is a lot of thoughtful and insightful writing about interesting and important ideas.
I also am pretty scared of messing up group incentives and coordination mechanisms – for example there are many kinds of growth that I have explicitly avoided because I think they would overwhelm our current reward and credit-allocation systems.
Fuck. I’m shocked to hear about a nice LessWronger like Corwin dying. That feels closer to home in some ways than many of the deaths I’ve known.
I’m also sorry to Gwern about your grandmother :(
Woop! I’m very pleased for you. I occasional plot about how to help people (including you-in-particular) to do this, and I’m very pleased that it has happened without my intervention! Beth continues to do good things, my appreciation to her.
I look forward greatly to your proportional increase in insights being posted to LessWrong :D
This was perhaps my favorite weekend event that we’ve done, both for the main event and the many hours of conversation afterward.
I’ll see if I can get the time to write up some of the thoughts and ideas I got from the discussion and put it into a post.
I had been considering the hypothesis that they would obviously not have kids, because look how much work they have to do. Just people most people have kids doesn’t mean a really small set of strongly selected outliers do – I expected that the most successful people also have very few major illnesses, have 99th percentile IQs, have married parents, etc. (I’m not sure that the health and parental ones check out either, but I think the 99th percentile IQ probably does.)
I had anticipated that most great scientists, inventors and builders would not have children, and was actively surprised when I found out (before today) that Elon and Demis did (and a few other people I had thought to check). It changed my attitude toward having kids substantially.
What I wouldn’t do to have Less Wrong available during my high school years… which makes me wonder, where are the teenage would-be rationalists hanging out today? There seem to be fewer of them here than I would have expected.
Some of us grew up and are a little more active on the site these days :)
I’d be interested in a post like this: https://www.lesswrong.com/posts/jcCMsg46RhcZJrTxP/survey-on-cortical-uniformity-an-expert-amplification
hurrah! betting ftw!
Aaah this was so much fun! I learned a lot from you looking back at your thinking from 12 months ago and seeing what you’ve learned since. This is one of my favorite posts in a while (along with this, incidentally, which I recommend everyone read because it was great).
We occasionally post them to the LessWrong YouTube channel: https://www.youtube.com/channel/UCt7yNB7lIW89N0hvISKJMpA
But mostly the best place to check is the event post (like this one), where it will eventually be updated.
Hey! Thanks. I notice you’re a brand new commenter and I wanted to say this was a great first (actually second) comment. Both your examples were on-point and detailed. Your second one FYI seems quite likely to me too. (My colleague Jacob Lagerros interacted with epidemiological modeling at many places early in the pandemic – including Imperial! – and I have heard many horror stories from him about the modeling that was being used to advise governments.)
I’ll leave an on-topic reply tomorrow, just wanted to say thanks for the solid comment.
Thanks for the examples! Those two sound like the second case I had in mind.
You’re welcome :)