I’ve been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I’ve been interested in improving my own epistemic standards and helping others to do so as well.
Raemon(Raymond Arnold)
Karma: 36,096
Yeah a few people have also brought up this concern recently. Will think about it.
Something I’m unsure about (commenting from my mod-perspective but not making a mod pronouncement) is how LW should relate to posts that lay out ideas that may advance AI capabilities.
My current understanding is that all major AI labs have already figured out the chinchilla results on their own, but that younger or less in-the-loop AI orgs may have needed to run experiments that took a couple months of staff time. This post was one of the most-read posts on LW this month, and shared heavily around twitter. It’s plausible to me that spreading these arguments plausibly speeds up AI timelines by 1-4 weeks on average.
It seems important to be able to talk about that and model the world, but I’m wondering if posts like this should live behind a “need to log-in” filter, maybe with a slight karma-gate, so that the people who end up reading it are at least more likely to be plugged into the LW ecosystem and are also going to get exposed to arguments about AI risk.
nostalgiabraist, I’m curious how you would feel about that.
Curated. I’m not sure I endorse all the specific examples, but the general principles make sense to me as considerations to help guide alignment research directions.
FYI, I’ve found this concept useful in thinking, but I think “atomic” is a worse word than just saying “non-interruptible”. When I’m explaining this to people I just say “unbounded, uninterruptible optimization”. The word atomic only seems to serve to make people say “what’s that?” and then I say “uninterruptible”
My impression is that a ton of work at MIRI (and some related research lines in other places) went into answering this question, and indeed, no one knows the answer very crisply right now and yup that’s alarming.
See John Wentworth’s post on Why Agent Foundations? An Overly Abstract Explanation, which discusses the need to find the True Name of agents.
(Also, while I agree agents are “more mysterious than rocks or The Odyssey”, I’m actually confused why the circularity is particularly the problem here. Why doesn’t the Odyssey also run into the Abstraction for Whom problem?)
Not sure if this was intentional or not, but the post title and opening paragraph both saying “DM” instead of “DeepMind” seemed kinda confusing.
I also found this a good exercise in deliberate questioning/boggling.
From 2010-2014, when I was first forming my opinions on AI, it was really frustrating that anyone who objected to the basic AI arguments just… clearly hadn’t been paying attention and at all and didn’t understand the basic arguments.
My understanding is that perfect predictor decision-thought-experiments are just simplified versions of “pretty good predictor thought experiments.” Like, if you. know that Omega is 90% accurate, it’s probably still better to one-box.
Minor bug/sadness report: I did the first question, then it prompted me to log in before revealing the answer, and AFAICT the question-answer didn’t get recorded and now I can’t find that question again and am curious and sad.
Oh man this is great. I was just wishing for something like this.
I’ve thought about integrating bets and predictions more into LessWrong. An alternate option is to integrate with existing prediction platforms to see people’s track record on Metaculus, PredictionBook, etc.
Strong upvoted for this line:
I tried this out for about a week before posting, and found the results to be interesting enough that I plan to try to keep it up for a while longer.
(I don’t endorse strong upvoting everything with this property, but am feeling capriciously like rewarding this today. I think there’s tendency to post about clever ideas as soon as you think about them, and actually trying them and seeing how it plays out is a good habit)
That said: I disagree with the claim that “distance” metaphors are inaccurate – as I’ve played a bunch of semantle and semantle.pimanrul and noticed how I think about concepts elsewhere in life, I think spectrums that make sense to measure in distance are just… everywhere. “Bird-iness” is pretty scalar. There are birds that can’t fly, or are bad at flying. There are things with lots of birdlike properties on the dimension of flying and on other dimensions.
That said, your description of the general benefit of forcing yourself to avoid your usual conventions and explore new thought patterns makes sense.
Oh, I meant: I predict you will write a post comparing boundaries and pivotal acts, but it will be unconvincing to me that boundaries are a particularly key term in how to think about pivotal acts.
(random additional note: I’m also anticipating that this boundaries sequence isn’t really going to address the core cruxes around pivotal acts)
Short answer: maybe Work/Life balance? Probably a reasonable combination of “meaty/relevant, without being too controversial an example”.
Longer answer: I’m not actually sure. While thinking about the answer, I notice part of the thing is that Expansive Thinking, Niche Finding and Work-Life balance each introduce somewhat different frames, and maybe another issue was I got sort of frame-fatigued by the time I got to Work/Life/Balance.
I found myself sort of bouncing off here, despite being interested in the topic and having a sense that “there’s something to this thread.”
I think right now my main takeaway is “think a bit more about boundaries on the margin, in a wider variety of places” (which is maybe all you wanted), but I think I got that more from in-person conversations with you than from this post. In this post there’s a lot of individual examples that give a sense of the breadth of how to apply the “boundaries” frame, but each one felt like I could use more details connecting it to the previous post. (Pedagogically I’d have found it helpful if the post was “T-shaped”, where it goes deep on one example connecting it to the game theory concepts in the previous post, and then explore a breadth of other examples which are easier to latch onto by virtue of connecting them to the initial deep example)
I agree that each of the domains listed here has some interesting stuff to think through. (though I’m not sure I need the “boundary” frame to think Something Is Up with each example as a place where EAs/rationalists might be making a mistake)
Thank you for doing this. This sequence seems like a good resource for community organizers.
Neat, I like “first thing you do is learn to fall” comparison.
(Note that I haven’t said anything about nuclear, because I think managing nuclear power plants is actually just an easy problem. That is, I think nearly all the safety strategies surrounding nuclear power are symbolic, unnecessary, and motivated by unfounded paranoia.)
I think managing a nuclear power plant is relatively easy… but, like, I’d expect managing a regular ol’ power plant (or, many other industries) to be pretty easy, and my impression is that random screwups still manage to happen. So it still seems notable to me if nuclear plants avoid the same degree of incidents that other random utility companies have.
I agree many of the safety strategies surrounding nuclear power are symbolic. But I’ve recently updated away from “yay nuclear power” because while it seems to me the current generation of stuff is just pretty safe and is being sort of unreasonably strangled by regulation… I dunno that I actually expect civilizational competence to continue having it be safe, given how I’ve seen the rest of civilizational competence playing otu. (I’m not very informed here though)
Becoming capable of building such a test is essentially the entire field of AI alignment. (yes, we don’t have the ability to build such a test and that’s bad, but the difficulty lives in the territory. MIRI’s previously stated goal were specifically to become less confused)