MIRI’s technical agenda: an annotated bibliography, and other updates

[So8res]

This week, I’m pleased to present Aligning Superintelligence with Human Interests: An Annotated Bibliography. This annotated bibliography complements our technical agenda and the six supporting papers that I’ve released over the past few months. Once you’ve read the supporting papers, this annotated bibliography will help you figure out what to read next on any given topic, in order to get to the cutting edge.

This annotated bibliography concludes my series of updates on MIRI’s technical agenda. To review, I’ve presented a series of eight papers which sketch out MIRI’s research strategy and overview a number of active research areas. Those papers are:

• Aligning Superintelligence with Human Interests: A Technical Research Agenda

• Formalizing Two Problems of Realistic World Models

• Toward Idealized Decision Theory

• Questions of Reasoning Under Logical Uncertainty

• Vingean Reflection: Reliable Reasoning for Self-Modifying Agents

• Corrigibility

• The Value Learning Problem

• Aligning Superintelligence with Human Interests: An Annotated Bibliography

The entire suite can be found in one place on our new technical agenda page. You’re encouraged to link people to that page if they’re wondering what sort of technical research MIRI is up to.

I’ve also updated the research guide to synchronize it with the technical agenda. The new annotated bibliography serves a similar purpose to the research guide, but where the research guide is geared more towards people starting from scratch, the annotated bibliography is geared towards people who already have the background knowledge and want to know which papers to read in which order without any fluff.

Enjoy!

[Ben Pace]

Thank you very much! This is exactly what I wanted for when I begin my CS course at Oxford in October.

[ahbwramc]

I’m curious, has this recent series of papers garnered much interest from the wider (or “mainstream”) AI community? It seems like MIRI has made a lot of progress over the past few years in getting a lot of very smart people to take their ideas seriously (and in cultivating a more respectable, “serious” image). I was wondering if similar progress had been made in creating inroads into academia.

[So8res]

Seven of the papers (every one except the annotated bibliography) are referenced in the FLI research priorities document attached to the open letter which received a whole lot of recent publicity :-) Beyond that, it’s a bit too early to guess how these particular papers will affect academia more broadly, but recent progress looks promising.

[ahbwramc]

As a MIRI donor, glad to here it! Good luck to you guys, you’re doing important work to say the least.