I’m an admin of this site; I work full-time on trying to help people on LessWrong refine the art of human rationality.
Ben Pace(Ben)
Karma: 16,829
I think I will write a follow-up post in a day or so, after the site is back up. For now, I’ll leave y’all to figure out wtf to believe about the world in light of this.
In the meantime I’m happy to confirm any basic facts. Indeed, user Chris_Leong submitted his unique launch codes at 4:33am Pacific Time, bringing down the LessWrong Frontpage for 24 hours.
Hi :) I was trying to safely increase the number of people with codes, so I included about 40 good older users who had very high karma but hadn’t been active in the last year. It didn’t seem bad to offer you the codes, you all seemed quite good natured from your comments.
And not one of you nuked the site, so thanks for that...
(As I stayed up quite late setting this all up, I will answer questions tomorrow… or at least, I will if the site is still up!)
Thanks, fixed.
Honoring Petrov Day on LessWrong, in 2020
I agree with regard to Moraga. Habryka and a few housemates of mine drove down to have a look around, and I think their main updates were that each house had only like 2 bedrooms, were all ~5x the distance from each other relative to Berkeley, there were no sidewalks, and no natural meeting place (the place with the shops had no natural seating), which means people just wouldn’t see each other very much unless everyone had a car and made it a conscious and constant effort. Even though it was nice and clean and so on.
I also agree wrt CFAR/MIRI. I would be interested in talking with them more, to see if they‘re open to moving generally, and what their preferences are, I’d massively prefer (both personally and as the LW team) to move with them than away from.
On the side of small-scale moving, I’m curious how you feel about the South Bay? It’s been an idea that a few friends of mine have raised, that after the pandemic we could recongregate there. It’s cleaner, a bit cheaper, larger houses, doesn’t have as much protests and conflict as Berkeley, has far fewer homeless people, and the same distance from SF, and would still work for group houses. It does have less good public transport, which matters if a sizeable number of us don’t primarily Uber for such things, although I think perhaps most of us do.
I think this tag description is confused between humility and modesty, as was originally written in the Twelve Virtues:
To be humble is to take specific actions in anticipation of your own errors. To confess your fallibility and then do nothing about it is not humble; it is boasting of your modesty.
Might come back and edit the description a bit, but overall it seems that the list of posts is largely accurate and useful.
Yeah. Hover-overs that tell you whose distribution it is would help here.
I did start trying to make a survey on this sort of info once, and for me it quickly became unmanageably big, like it would have had over a hundred questions and would require lots of iteration loops with users. I spent a bunch of time trying to figure out how far from a major city people would be able to live, with lots of questions about earning, earning potential, openness and ability to do remote work, etc.
Probably someone else will figure out a clever smaller survey worth making though. (I’d be happy to give comments on any draft surveys people make.)
Suppose there’s a point where it’s clearly possible that humanity explores the universe and flourishes. Suppose there’s a point where it’s clearly no longer possible that humanity explores the universe and flourishes. Assign 0% of existential risk at the first point, and 100% at the second point. Draw a straight line between.
Then make everything proportional to the likelihood of things going off the rails at that time period.
I listened to this yesterday! Was quite interesting, I’m glad I listened to it.
I expect the examples Ajeya has in mind are more like sharing one-line summaries in places that tend to be positively selected for virality and anti-selected for nuance (like tweets), but that substantive engagement by individuals here or in longer posts will be much appreciated.
I also didn’t understand what your sentence was saying. It read to me as “I don’t want people to update on this post”. When you pointed specifically to LW’s culture (which is very argumentative) possibly being a key cause it was clearer what you were saying. Thanks for the clarification (and for trying to avoid negative misinterpretations of your comment).
Hah!
Yeah, I think we’ve made substantial progress in the last couple of years.
I like this experiment! Keep ’em coming.
The motivation seems trivial to me, which might be part of the proble
Yeah, for a long time many people have been very confused about the motivation of Paul’s research, so I don’t think you’re typical in this regard. I think that due to this sequence, Alex Zhu’s FAQ, lots of writing by Evan, Paul’s post “What Failure Looks Like”, and more posts on LW by others, a many more people understand Paul’s work on a basic level that was not at all the case 3 years ago. Like, you say “Most training procedures are obviously outer-misaligned”, and ‘outer-alignment’ was not a concept with a name or a write-up at the time this sequence was published.
I agree that this post talks about assuming human-level performance, whereas much of iterated amplification also relaxes that assumption. My sense is that if someone were to just read this sequence, it would still help them focus on the brunt of the problem, that being ‘helpful’ or ‘useful’ is not well-defined in the way many other tasks are, and help realize the urgency of this task and why it’s possible to make progress now.
Maybe Mensa specifically selects for a combination of high IQ and low RQ, dunno.
I’m curious what the actual cutoff for Mensa is.
Googling, I get that the cutoff for getting into Mensa is 98th percentile in IQ, which means in the US this means you’re in the top 6.5 million people in terms of IQ. This doesn’t seem to me to be a particularly high cutoff. I’d imagine most Math PhDs form a far more selective group, given that I expect they’re all in that set and also there’s probably only order-of-magnitude 10k Math PhDs, and also they have been selected more on conscientiousness and competence (insofar as Mensa has no work required other than the test, whereas mathematicians have way more intellectual hoops to jump through).
(I don’t know what exactly this implies, mostly just noticing things aloud here.)
Things this post does:
Explain what problem civilization is likely to face with the development of AI (i.e. we’ll be able to build AIs that solve well-defined tasks, but we don’t know how to well define the task of being ‘useful’).
Connect this problem to a clear and simple open problem, which is clear on its assumptions and helps me understand the hard parts of fixing the above problem regarding AI development.
Provides argument for the important and urgency of working on this sort of technical research.
This gives real-world motivation and a direct problem to be solved to help the real-world problem, and engages concretely with the bigger picture in a way other posts in the sequence don’t (e.g. “Clarifying AI Alignment” is a clarification and doesn’t explicitly motivate why the problem is important).
In general, especially at the time when these posts were published, when I’d read them I felt like I understood a particular detail very clearly, but I did not understand the bigger picture of why that particular detail was interesting, or why those particular assumptions should be salient, and this post helped me understand it a great deal.
I’m not sure if the above still helped? Does it make sense what experience I had reading the post? I’m also interested if other posts feel to you like they clearly motivate iterated amplification.
Two things:
I’ve taken vitamin D daily for a few months of the pandemic, will get back to taking it. I’ve also just told my mum she needs to be on it. Thanks for this post, and pushing me to get back on it.
You said we were on it back in February, which is right. It was discussed on LW in the coronavirus justified practical advice thread, where the top answer was Connor Flexman recommending Vitamin D. It’s still a little early, but if the effects are confirmed, we will definitely be celebrating Connor for getting it so right for the right reasons early on, and for telling the rest of us. (His comment is why I started taking it.)
I also think that XKCD would be quite appropriate had it been the site admins. But no, it was not us.