Announcing the AI Alignment Prize

Stronger than hu­man ar­tifi­cial in­tel­li­gence would be dan­ger­ous to hu­man­ity. It is vi­tal any such in­tel­li­gence’s goals are al­igned with hu­man­ity’s goals. Max­i­miz­ing the chance that this hap­pens is a difficult, im­por­tant and un­der-stud­ied prob­lem.

To en­courage more and bet­ter work on this im­por­tant prob­lem, we (Zvi Mow­show­itz and Vladimir Slep­nev) are an­nounc­ing a $5000 prize for pub­li­cly posted work ad­vanc­ing un­der­stand­ing of AI al­ign­ment, funded by Paul Chris­ti­ano.

This prize will be awarded based on en­tries gath­ered over the next two months. If the prize is suc­cess­ful, we will award fur­ther prizes in the fu­ture.

The prize is not backed by or af­fili­ated with any or­ga­ni­za­tion.


Your en­try must be pub­lished on­line for the first time be­tween Novem­ber 3 and De­cem­ber 31, 2017, and con­tain novel ideas about AI al­ign­ment. En­tries have no min­i­mum or max­i­mum size. Im­por­tant ideas can be short!

Your en­try must be writ­ten by you, and sub­mit­ted be­fore 9pm Pa­cific Time on De­cem­ber 31, 2017. Sub­mit your en­tries ei­ther as links in the com­ments to this post, or by email to ap­ply@ai-al­ign­ We may provide feed­back on early en­tries to al­low im­prove­ment.

We will award $5000 to be­tween one and five win­ners. The first place win­ner will get at least $2500. The sec­ond place win­ner will get at least $1000. Other win­ners will get at least $500.

En­tries will be judged sub­jec­tively. Fi­nal judg­ment will be by Paul Chris­ti­ano. Prizes will be awarded on or be­fore Jan­uary 15, 2018.

What kind of work are we look­ing for?

AI Align­ment fo­cuses on ways to en­sure that fu­ture smarter than hu­man in­tel­li­gence will have goals al­igned with the goals of hu­man­ity. Many ap­proaches to AI Align­ment de­serve at­ten­tion. This in­cludes tech­ni­cal and philo­soph­i­cal top­ics, as well as strate­gic re­search about re­lated so­cial, eco­nomic or poli­ti­cal is­sues. A non-ex­haus­tive list of tech­ni­cal and other top­ics can be found here.

We are not in­ter­ested in re­search deal­ing with the dan­gers of ex­ist­ing ma­chine learn­ing sys­tems com­monly called AI that do not have smarter than hu­man in­tel­li­gence. Th­ese con­cerns are also un­der­stud­ied, but are not the sub­ject of this prize ex­cept in the con­text of fu­ture smarter than hu­man in­tel­li­gence. We are also not in­ter­ested in gen­eral AI re­search. We care about AI al­ign­ment, which may or may not also ad­vance the cause of gen­eral AI re­search.

(Ad­den­dum: the re­sults of the prize and the rules for the next round have now been an­nounced.)