That’s not for me to answer, sorry.
Gretta Duleba
Karma: 2,623
1) MIRI never stopped doing technical alignment research, actually! It’s just a much smaller part of the portfolio than it used to be. The majority of the focus is now on comms and policy. We do still need both, though; we don’t want to halt AI research forever, so we’re going to need a technical alignment solution.
2) I no longer work at MIRI at all, I left (amicably!) near the beginning of the year. I’m independent, and eagerly awaiting word on my SFF application.
Yes, that’s right. I managed/grew the comms team for about a year and a half, and then I supported Eliezer through the process of writing, editing, and launching If Anyone Builds It. When the book launch was over I found myself wildly underutilized and wishing for much more to do. As a technical and autistic nerd, I was always better suited to working on mathy problems than on communicating with neurotypicals anyway. It’s good to be back on the tech side of things.
Me: “I finished my post!”
Eliezer: “What’s it about?”
Me: explains
Eliezer: “Oh, I’ve got some more concepts for you!”
Me: “Oh no.”
Eliezer: “The inside of a brick (from the Feynman story), incorrect meta-ethics, medianworld, threat, leftism, miracle, eugenics, pornography, sin, Chaotic Neutral, paradox, unicorn, magic, the least non-interesting number, …”
Me: “Please stop?”
Eliezer, grinning: “Non-nouns!”
Me: “Ahhhhh”
A Test Suite for Concepts
Gretta Duleba’s Shortform
On January 1st, 2026, I started focusing full time on technical alignment research.
Well, to be more accurate, I started laying the groundwork for doing technical alignment research. Although I have a solid and deep technical background, mostly in computer science and software engineering, my background is not very wide and was missing a bunch of important pieces. So the year so far has involved a lot more studying than thinking new thoughts at the frontier.
I’m working with John Wentworth on natural abstractions. It’s my full time job to understand what he’s been up to, but it’s just a side gig for him to explain it to me. A few hours of explanations from him gives me ~a week of material to chew on.
As I’ve been building all these background models, I’ve discovered that John has a lot of information and thought-structures built up in his head that he’s never properly written down or explained. One of my tasks this year will be to write those down properly and post them here.
I mostly think of those write-ups as prelude to the real work, but you never know. Sometimes when you take the time to write down your background models, you realize that there was a missing or slightly off-kilter piece, and fixing that up leads to new insights. We’ll see.
Maybe you already saw it, but see also the discussion here: https://www.lesswrong.com/posts/FY697dJJv9Fq3PaTd/hpmor-the-probably-untold-lore
This is very helpful, thank you!
Data you could have observed but didn’t
He’s already on it.
Kegan 2 and Kegan 4 often look pretty similar to each other when all you have is a quick glance from the outside. I will grant you that nothing John said in his post or his comment would persuade you he’s running 4, though!
Yup. As I said right up front, one of the main reasons the data is sus is that it’s self-reported.
My advice is both obvious and pretty hard to execute on. Have great, well-reasoned insights and write well.
Man, sometimes I don’t even know what data to offer you, because I would not have guessed that “five hours” would be an update. In fact, I think five hours is a very small amount of time to invest by most people’s lights.
Really glad my post helped. You’re welcome.
I wrote a response post: https://www.lesswrong.com/posts/ytzrakjgcvCfLCCZp/contra-wentworth-on-physical-attractiveness-for-men
h/t to @Seth Herd and @Shoshannah Tekofsky, both of whom made excellent comments here that informed my response.
Contra Wentworth on Physical Attractiveness for Men
Causal inference diary: skiing causes snow
I had exactly the same thought. I didn’t write the same comment, though, because:
1) He did say the genome is part of what makes a squirrel a good object, and
2) The genome is a very natural dividing line between “part of this squirrel” and “not part of this squirrel,” and natural dividing lines are useful!
A “good object” is in the eye of the beholder—indeed, you spend most of your comment beholding and talking about how you and your brain chunk up the world around you, and that all makes sense. But “in the perception of a particular human” is not the only way to judge what a good object is.
Yeah there’s no doubt in my mind that the concept representation, concept relationships/nesting, etc. is going to have a lot to do with the agent’s actual exposure to the environment (both due to what lived experience they have and what sensory organs they have) and also to the relevance of parts of the environment to their causal model/goals/utility function. So different agents will end up with different concepts, differently fleshed out, because of all of those things.
AND YET we still hope that some concepts are just so useful that ~all agents will pick them up.
Luckily this is an empirical claim that we will eventually figure out how to test.