Announcing: OpenAI’s Alignment Research Blog
The OpenAI Alignment Research Blog launched today at 11 am PT! With 1 introductory post, and 2 technical posts.
Blog: https://alignment.openai.com/
Thread on X: https://x.com/j_asminewang/status/1995569301714325935
Speaking purely personally: when I joined the Alignment team at OpenAI in January, I saw there was more safety research than I’d expected. Not to mention interesting thinking on the future of alignment. But that research & thinking didn’t really have a place to go, considering it’s often too short or informal for the main OpenAI blog, and most OpenAI researchers aren’t on LessWrong. I’m hoping the blog is a more informal, lower-friction home than the main blog, and this new avenue of publishing encourages sharing and transparency.
I generally think blogging is a good way to communicate intellectual progress, so see this as a good development!
Some thoughts on your first blogpost:
My reaction: Wait, what, why? I guess it’s nice to be as direct, but it feels sad that this is written as the bottom line.
To be clear, I agree that given this being OpenAI’s stance it’s good to say it plainly! But I was hoping that at least the safety team would have the position that “we will try to determine whether there is any way to build RSI safely, and will strongly advocate for not doing so if we think it cannot be done safely”.
Like, a thing that feels particularly sad here is that I was assuming that figuring out whether this can be done safely, or studying that question, is one of the key responsibilities of the safety team. This is an update that it isn’t, which is sad (and IMO creates some responsibility for members of the safety team to express concern about that publicly, but IDK, OpenAI seems like in a messy state with regards to that kind of stuff).
Thank you for pointing this out! While OpenAI have been public about our plans to build an AI scientists, it is of course crucial that we do this safely, and if it is not possible to do it safely, we should not do it at all.
We have written about this before:
but we should have mentioned this in the hello world post too. We now updated it with a link to this paragraph.
It looks like OpenAI is following Anthropic’s lead, which is great!
Google DeepMind’s alignment team also has a blog, but it’s much more targeted towards laypeople, mostly shares papers as they come out rather than sharing informal research, and IMO isn’t as nice compared to a dedicated website or even a Substack. It might be worth considering doing something like this at GDM, subject to tradeoffs on researchers’ time and Google’s internal publication restrictions.
We mostly just post more informal stuff directly to LessWrong / Alignment Forum (see e.g. our interp updates).
Having a separate website doesn’t seem that useful to readers. I generally see the value proposition of a separate website as attaching the company’s branding to the post, which helps the company build a better reputation. It can also help boost the reach of an individual piece of research, but this is a symmetric weapon, and so applying it to informal research seems like a cost to me, not a benefit. Is there some other value you see?
(Incidentally, I would not say that our blog is targeted towards laypeople. I would say that it’s targeted towards researchers in the safety community who have a small amount of time to spend and so aren’t going to read a full paper. E.g. this post spends a single sentence explaining what scheming is and then goes on to discuss research about it; that would absolutely not work in a piece targeting laypeople.)
Oh yep, I forgot about the interp updates and other informal posts from GDM researchers—these seem to occupy the same niche I was thinking the blog could have. The idea seemed worth suggesting, for the simple reason that Anthropic and OpenAI had both done it, but in retrospect I buy your argument that the main benefit of a blog is to the company rather than the readers, and I retract my suggestion.
My impression with the Medium blog is based pretty much entirely on the one paper I was involved with (MONA), where I remember hearing something like “we’re going to write a separate Medium post and a LessWrong post, and the Medium post will assume (relatively) less knowledge of the field.” I’m not sure if my impression was correct, or if things have changed since then.
Ah yeah, I think with that one the audiences were “researchers heavily involved in AGI Safety” (LessWrong) and “ML researchers with some interest in reward hacking / safety” (Medium blog)
@Naomi Bashkansky Would it make sense for OpenAI to crosspost their blog posts to LW/AF? I’d personally be more likely to see them if they did.
Sure! I’ll make sure they get cross-posted.
Agreed!
Any chance of an RSS feed?