I’m an admin of this site; I work full-time on trying to help people on LessWrong refine the art of human rationality.
Longer bio: www.lesswrong.com/posts/aG74jJkiPccqdkK3c/the-lesswrong-team-page-under-construction#Ben_Pace___Benito
Ben Pace(Ben)
Karma: 20,102
Just for record-keeping, here is the OWID global death tracker (from google), with the vertical line at the point when the comment was written.
Brief review: I think this post represents a realization many people around here have made, and says it clearly. I think it’s fine to keep it as a record that people used to be blasé about the ease of secrecy, and later learned that it was much more complex than they thought. I think I’m at +1.
My quick two-line review is something like: this post (and its sequel) is an artifact from someone with an interesting perspective on the world looking at the whole problem and trying to communicate their practical perspective. I don’t really share this perspective, but it is looking at enough of the real things, and differently enough to the other perspectives I hear, that I am personally glad to have engaged with it. +4.
Most people have not put tens of thousands of deliberate hours of practice into their writing skills so do not have the clarity to be able to say what they think shortly, and this lack of skill is typically why their writing is long. Eliezer has worked hard to be able to write clearly, and also to build a rare skill of being able to expose more of the cognition behind a thought as he writes longer, which is in many important domains more valuable to do than just stating the output of the cognition.
I’m saying: Eliezer’s has built the skill to say his thoughts precisely and clearly; but he has also built the next-level skill of being able to expose the cognition behind a thought, and this is the sort of valuable length that he hopes to have in his writing.
“Search versus design” explores the basic way we build and trust systems in the world. A few notes:
My favorite part is the definitions about an abstraction layer being an artifact combined with a helpful story about it. It helps me see the world as a series of abstraction layers. We’re not actually close to true reality, we are very much living within abstraction layers — the simple stories we are able to tell about the artefacts we build. A world built by AIs will be far less comprehensible than the world we live in today. (Much more like biology is, except made by something that is much smarter and faster than us instead of stupider and slower.)
The post puts in the time to bring into the conversation a lot of other work that attempts to help build simple stories about the AI artefacts that we are building, which I appreciate.
The post is pretty simply written, for me, and I understand all the examples and arguments.
It also attempts to (briefly) describe a novel direction of future work for solving the problem of building untrustworthy systems with selection, and that’s exciting.
For looking at the alignment problem clearly and with a subtly different frame than other discussions, one that resonates for me, and that points to new frames for a solution, I am voting this post +9.
Indeed the books are not yet in-stock in Amazon UK.
I am also quite interested to read this sequence.
I note that I also like “Zeynep’s Razor”. Seems about as punchy as “Zeynep’s Law” without slightly weakening the use of the term ‘law’.
(I did not write a curation notice in time, but that doesn’t mean I don’t get to share why I wanted to curate this post! So I will do that here.)
Typically when I read a post by Paul, it feels like a single ingredient in a recipe, but one where I don’t know what meal the recipe is for. This report felt like one of the first times I was served a full meal, and I got to see how all the prior ingredients come together.
Alternative framing: Normally Paul’s posts feel like the argument step “J → K” and I’m left wondering how we got to J, and where we’ll go from K. This felt like one of the first times I got to go from A all the way to (say) P. I can see how the pieces fit together, and I have an interest in and a better perspective on where it might go on from P later.
There are many more positive things to say about this post. I am very excited by the way the post takes a relatively simple problem and shows, in trying to solve it, a great deal of the depth of the alignment problem. The smart vault example, story and art is very clear and fun. Explaining the methodology along with the steps of implementing it works really well to show how the methodology works. I love seeing how things like Iterated Amplification fit into the bigger solution. I find it thrilling every time the authors are like “let us make this wildly optimistic assumption, because even then we have a deadly counterargument”. I feel like for the first time I got to understand what seems weird and strange and interesting about some of Paul’s ideas, even ones that have been discussed before, because I saw them in the larger context, as thoughts that I myself would be very unlikely to think in that context. Etc.
Paul gives a 25% chance that he and Mark will see major progress which qualitatively changes their picture within a year, and seeing this post, the methodology, and all of the creative argumentative steps so far, I share this optimism, within the methodology that is being used here.
Curated. This might be my favorite thing I’ve read about GPT-3. Ideas about how bigger models might just be finding the same knowledge more effectively, or how it’s sensible for longer but then makes the same old-dumb mistakes at the end, really help me understand what GPT-3 is and what it isn’t. I wish I had time to write a more thoughtful curation notice, but I’ll just repeat I’m very glad you wrote this post, I’d be much more conceptually malnourished about modern AI without it.
Woop great post and great thread!
For me, I only do the former post when I want to really nail something and put loads of work into it (e.g. my common knowledge post).
I do the latter kind when I’ve just thought about a thing for a while and I feel like I got somewhere good. I don’t aim to write a perfect piece on it, I aim to write like I would explain my thinking in conversation. I typically can write such posts in ~2hrs (e.g. my environment post), and that seems worth publishing to me, and then time to move on with my thoughts.
hahaha
I found this post quite helpful, thank you. I think there’s a 40% chance I would find a sequel very illuminating again.
25% of the value I reckon? You wouldn’t recreate the art or layout or additions content. Would be nice if you did it, I’d certainly learn a new fact based on number of downloads if that was measurable.
Glad to hear that was helpful. I do recommend reading the rest of the sequence.
I think the ones I had the strongest negative reactions to were “Do Less Things and Be Smaller” and “Start Again”. I had a feeling of “I have no idea how to succeed if I have to do these things”, and now I’m like “succeeding at anything is generally easier with these things”.
Fixed.
Elephant seal is a picture of an elephant seal. It has a mysterious Mona Lisa smile that I can’t pin down, that shows glee, intent, focus, forward-looking-ness, and satisfaction. It’s fat and funny-looking. It looks very happy lying on the sand. I give this post a +4.
(This review is taken from my post Ben Pace’s Controversial Picks for the 2020 Review.)
This post feels like a fantasy description of a better society, one that I would internally label “wish-fulfilment”. And yet it is history! So it makes me more hopeful about the world. And thus I find it beautiful.