Learning to get things right first time

Th­ese are quick notes on an idea for an in­di­rect strat­egy to in­crease the like­li­hood of so­ciety ac­quiring ro­bustly safe and benefi­cial AI.


  • Most challenges we can ap­proach with trial-and-er­ror, so many of our habits and so­cial struc­tures are set up to en­courage this. There are some challenges where we may not get this op­por­tu­nity, and it could be very helpful to know what meth­ods help you to tackle a com­plex challenge that you need to get right first time.

  • Giv­ing an ar­tifi­cial in­tel­li­gence good val­ues may be a par­tic­u­larly im­por­tant challenge, and one where we need to be cor­rect first time. (Distinct from cre­at­ing sys­tems that act in­tel­li­gently at all, which can be done by trial and er­ror.)

  • Build­ing stronger so­cietal knowl­edge about how to ap­proach such prob­lems may make us more ro­bustly pre­pared for such challenges. Hav­ing more pro­gram­mers in the AI field fa­mil­iar with the tech­niques is likely to be par­tic­u­larly im­por­tant.

Idea: Develop meth­ods for train­ing peo­ple to write code with­out bugs.

  • Try­ing to teach the skill of get­ting things right first time.

  • Writ­ing or edit­ing code that has to be bug-free with­out any test­ing is a fairly easy challenge to set up, and has sev­eral of the right kind of prop­er­ties. There are some par­allels be­tween value speci­fi­ca­tion and pro­gram­ming.

  • Set-up puts peo­ple in sce­nar­ios where they only get one chance—no op­por­tu­nity to test part/​all of the code, just analyse closely be­fore sub­mit­ting.

    • In­ter­ested in per­sonal habits as well as so­cial norms or pro­ce­dures that help this.

      • Daniel Dewey points to stan­dards for code on the space shut­tle as a good ex­am­ple of get­ting high re­li­a­bil­ity code ed­its.

How to im­ple­ment:

  • Ideal: Offer this train­ing to staff at soft­ware com­pa­nies, for profit.

    • Although it’s teach­ing a skill un­der ar­tifi­cial hard­ship, it seems plau­si­ble that it could teach enough good habits and lines of think­ing to no­tice­ably in­crease pro­duc­tivity, so peo­ple would be will­ing to pay for this.

    • Be­cause such train­ing could cre­ate so­cial value in the short run, this might give a good op­por­tu­nity to launch as a busi­ness that is si­mul­ta­neously do­ing valuable di­rect work.

    • Similarly, there might be a mar­ket for a con­sul­tancy that helped or­gani­sa­tions to get gen­eral tasks right the first time, if we knew how to teach that skill.

  • More fund­ing-in­ten­sive, less labour in­ten­sive: run com­pe­ti­tions with cash prizes

    • Try to es­tab­lish it as some­thing like a com­pet­i­tive sport for teams.

    • Out­source the work of de­ter­min­ing good meth­ods to the con­tes­tants.

This is all quite pre­limi­nary and I’d love to get more thoughts on it. I offer up this idea be­cause I think it would be valuable but not my com­par­a­tive ad­van­tage. If any­one is in­ter­ested in a pro­ject in this di­rec­tion, I’m very happy to talk about it.