Alignment Research Field Guide

This field guide was writ­ten by the MIRI team with MIRIx groups in mind, though the ad­vice may be rele­vant to oth­ers work­ing on AI al­ign­ment re­search.

Pream­ble I: De­ci­sion Theory

Hello! You may no­tice that you are read­ing a doc­u­ment.

This fact comes with cer­tain im­pli­ca­tions. For in­stance, why are you read­ing this? Will you finish it? What de­ci­sions will you come to as a re­sult? What will you do next?

No­tice that, what­ever you end up do­ing, it’s likely that there are dozens or even hun­dreds of other peo­ple, quite similar to you and in quite similar po­si­tions, who will fol­low rea­son­ing which strongly re­sem­bles yours, and make choices which cor­re­spond­ingly match.

Given that, it’s our recom­men­da­tion that you make your next few de­ci­sions by ask­ing the ques­tion “What policy, if fol­lowed by all agents similar to me, would re­sult in the most good, and what does that policy sug­gest in my par­tic­u­lar case?” It’s less of a ques­tion of try­ing to de­cide for all agents suffi­ciently-similar-to-you (which might cause you to make the wrong choice out of guilt or pres­sure) and more some­thing like “if I were in charge of all agents in my refer­ence class, how would I treat in­stances of that class with my spe­cific char­ac­ter­is­tics?

If that kind of think­ing leads you to read fur­ther, great. If it leads you to set up a MIRIx chap­ter, even bet­ter. In the mean­time, we will pro­ceed as if the only peo­ple read­ing this doc­u­ment are those who jus­tifi­ably ex­pect to find it rea­son­ably use­ful.

Pream­ble II: Sur­face Area

Imag­ine that you have been tasked with mov­ing a cube of solid iron that is one me­ter on a side. Given that such a cube weighs ~16000 pounds, and that an av­er­age hu­man can lift ~100 pounds, a naïve es­ti­ma­tion tells you that you can solve this prob­lem with ~150 will­ing friends.

But of course, a me­ter cube can fit at most some­thing like 10 peo­ple around it. It doesn’t mat­ter if you have the the­o­ret­i­cal power to move the cube if you can’t bring that power to bear in an effec­tive man­ner. The prob­lem is con­strained by its sur­face area.

MIRIx chap­ters are one of the best ways to in­crease the sur­face area of peo­ple think­ing about and work­ing on the tech­ni­cal prob­lem of AI al­ign­ment. And just as it would be a bad idea to de­cree “the 10 peo­ple who hap­pen to cur­rently be clos­est to the metal cube are the only ones al­lowed to think about how to think about this prob­lem”, we don’t want MIRI to be­come the bot­tle­neck or au­thor­ity on what kinds of think­ing can and should be done in the realm of em­bed­ded agency and other rele­vant fields of re­search.

The hope is that you and oth­ers like you will help ac­tu­ally solve the prob­lem, not just fol­low di­rec­tions or read what’s already been writ­ten. This doc­u­ment is de­signed to sup­port peo­ple who are in­ter­ested in do­ing real ground­break­ing re­search them­selves.


  1. You and your research

  2. Lo­gis­tics of get­ting started

  3. Models of so­cial dynamics

  4. Other use­ful thoughts and questions

1. You and your research

We some­times hear ques­tions of the form “Even a sum­mer in­tern­ship feels too short to make mean­ingful progress on real prob­lems. How can any­one ex­pect to meet and do real re­search in a sin­gle af­ter­noon?”

There’s a Zeno-es­que sense in which you can’t make re­search progress in a mil­lion years if you can’t also do it in five min­utes. It’s easy to fall into a trap of (ei­ther im­plic­itly or ex­plic­itly) con­cep­tu­al­iz­ing “re­search” as “first study­ing and learn­ing what’s already been figured out, and then at­tempt­ing to push the bound­aries and con­tribute new con­tent.”

The prob­lem with this frame (ac­cord­ing to us) is that it leads peo­ple to op­ti­mize for ab­sorb­ing in­for­ma­tion, rather than seek­ing it in­stru­men­tally, as a pre­cur­sor to un­der­stand­ing. (Be mind­ful of what you’re op­ti­miz­ing in your re­search!)

There’s always go­ing to be more pre-ex­ist­ing, learn­able con­tent out there. It’s hard to pre­dict, in ad­vance, how much you need to know be­fore you’re qual­ified to do your own origi­nal think­ing and see­ing, and it’s easy to Dun­ning-Kruger or im­pos­tor-syn­drome your­self into end­less hes­i­ta­tion or an over-re­li­ance on ex­ist­ing au­thor­ity.

In­stead, we recom­mend throw­ing out the whole ques­tion of au­thor­ity. Just fol­low the threads that feel al­ive and in­ter­est­ing. Don’t think of re­search as “study, then con­tribute.” Fo­cus on your own un­der­stand­ing, and let the ques­tions them­selves de­ter­mine how of­ten you need to go back and read pa­pers or study proofs.

Ap­proach­ing re­search with that at­ti­tude makes the ques­tion “How can mean­ingful re­search be done in an af­ter­noon?” dis­solve. Mean­ingful progress seems very difficult if you try to mea­sure your­self by ob­jec­tive ex­ter­nal met­rics. It is much eas­ier when your own taste drives you for­ward.

No pro­ce­dure for do­ing re­search will fit for ev­ery­one. How­ever, what fol­lows are steps which you can try ei­ther on your own or in a group set­ting (such as MIRIx) in or­der to prac­tice the kind of cu­ri­os­ity-driven re­search just de­scribed.

1. Write a list of ques­tions.

  • If you are do­ing this as a group, put the list on a white­board or other place where ev­ery­one can see.

  • Fo­cus on what you don’t know how to do, or what you feel con­fused about.

  • If no ques­tions come to mind, say to your­self (or the group), “ex­cel­lent, I must know how to solve the whole prob­lem” and try to give de­tails of the solu­tion un­til you get stuck.

  • It’s also OK for things on the list to be ideas you’d like to de­velop fur­ther, or thoughts you’d like the group to cri­tique, rather than ques­tions.

2. Choose one of the ques­tions to fo­cus on, based on what feels most in­ter­est­ing.

  • If you are in a group of more than three peo­ple, con­sider split­ting the group up. Each group can dis­cuss its own ques­tion, or have par­allel dis­cus­sions on the same over­all ques­tion. Agree on a time to come back to­gether and dis­cuss what you thought about.

  • It can be good to keep the whole list of ques­tions some­where visi­ble, so that you have a re­minder of other in­ter­est­ing top­ics to switch to if thoughts pe­ter out on the ques­tion origi­nally cho­sen.

3. Clar­ify your cu­ri­os­ity. What is de­sired? What do you think might be pos­si­ble?

  • In a group, usu­ally the per­son who pro­posed a topic will have some things to say in or­der to get ev­ery­one on the same page.

  • Work­ing on your own, it can be use­ful to just start writ­ing down ev­ery­thing you think you know, and what you think you don’t know. Write down any­thing po­ten­tially rele­vant which comes to mind. Don’t worry ini­tially about whether your claims are true or whether your ques­tions are mean­ingful. Then, go back and try to make sense of it. Try to for­mal­ize your claims and ques­tions un­til they turn into some­thing which is definitely ei­ther true or false.

4. Keep clar­ify­ing.

  • Keep stat­ing sub-ques­tions and mak­ing claims which may or may not be true, start­ing in­for­mally and work­ing to­wards for­mal rigor.

  • No­tice where your cu­ri­os­ity waxes and wanes, and avoid du­tiful com­plete­ness. Look for the sim­plest pos­si­ble cases that you are still con­fused about, and try to work through them.

  • Allow your­self to get side­tracked. Allow your­self to play. So long as ev­ery­one in the dis­cus­sion is cu­ri­ous and en­gaged, it’s work­ing to build un­der­stand­ing. Be open to get­ting nerd-sniped by “ir­rele­vant” math ques­tions; they may even­tu­ally turn out to be more rele­vant than they seem. You’re build­ing your own ca­pa­bil­ity, even if it isn’t di­rectly use­ful to the prob­lem you’re work­ing on.

  • If you do ar­rive at a con­crete math­e­mat­i­cal re­sult which cap­tures some­thing in­ter­est­ing, or even a con­crete math­e­mat­i­cal ques­tion, write it up prop­erly. A good write-up of­ten adds a lot to your own un­der­stand­ing, be­sides the value of com­mu­ni­cat­ing your ideas to oth­ers.

This re­sem­bles how much of the progress at MIRI hap­pens. It’s very differ­ent from the at­trac­tor of “just read lots of pa­pers,” and it’s very differ­ent from the at­trac­tor of “try to figure out top-down what the field as a whole needs.”

An easy mis­take is to think of your­self as try­ing to con­tribute to the world’s col­lec­tive knowl­edge, and thereby ne­glect­ing to pri­ori­tize your own knowl­edge and un­der­stand­ing. “Just read pa­pers” may sound like it’s pri­ori­tiz­ing your own knowl­edge, but it of­ten re­flects a mind­set that’s tac­itly as­sum­ing that oth­ers know ex­actly what you need to know. “Op­ti­mize for your own un­der­stand­ing” is a mind­set with a faster feed­back loop.

There’s noth­ing in­her­ently wrong with read­ing pa­pers—even if it’s just be­cause they’re in the field and you want a broad overview of the field. But through­out, you should be try­ing to form a pic­ture of what you per­son­ally do and don’t know how to do, and what you’d need to know how to do in or­der to solve the prob­lem. That’s hard, and maybe you’re sure that the first five ideas you write down will be wrong. Still, write them down any­way, and try to get them to work, so you can see what hap­pens and dis­cover what goes wrong.

We don’t want a hun­dred bright minds all ask­ing the ex­act same ques­tions, and tak­ing the ex­act same set of as­sump­tions. We want a field full of ex­plor­ers, not ex­ploiters. Put an­other way, the best way to be­come a re­searcher is to prac­tice the skill of in­de­pen­dent thought right from the be­gin­ning, rather than ex­er­cis­ing your “sit back and ab­sorb in­for­ma­tion for its own sake” mus­cles.

So don’t ask “What are the open ques­tions in this field?” Ask: “What are my ques­tions in this field?”

2. Lo­gis­tics of get­ting started

Let’s say you’ve tried some things that re­sem­ble the above, you en­joyed them, and you want to move for­ward on start­ing your own MIRIx chap­ter.

Our first recom­men­da­tion is that you find ONE or TWO other peo­ple (not three+), and try do­ing re­search to­gether once or a few times. There’s more de­tail be­low in the so­cial dy­nam­ics sec­tion about how ex­actly that might look, but the idea is that you want to es­tab­lish a tone and flow with a small num­ber of peo­ple first. Ne­go­ti­at­ing a di­rec­tion for the group tends to be much harder if you start with a larger num­ber of peo­ple.

Another im­por­tant choice which can be difficult to ne­go­ti­ate with a large num­ber of peo­ple is sched­ule. Find­ing a time and place which is good for ev­ery­one can be­come in­tractable, and chang­ing it meet­ing to meet­ing to try to make it work for ev­ery­one can be de-mo­ti­vat­ing. Choose a sched­ule which is good for the found­ing core of the group. What day of the week is good for you? How of­ten do you want to meet? How long do you want meet­ings to be? We recom­mend meet­ings be monthly, weekly, or ev­ery other week. Meet­ing length can be any­where from an hour to a whole day, de­pend­ing on what makes sense for you.

Once you find a part­ner or two that you gen­uinely en­joy mak­ing progress with, your next step is to plan and ad­ver­tise for a first large meetup (where “large” means some­thing like “three to six new peo­ple” and definitely doesn’t mean “twenty or thirty at­ten­dees”).

Try to find a venue that is pri­vate and sound-iso­lated, has flat sur­faces and com­fortable seat­ing, and has white­boards on the walls. Univer­si­ties of­ten have spaces like this, as do pub­lic libraries, but some­one’s liv­ing room is fine if you can min­i­mize the num­ber of in­tru­sions and in­ter­rup­tions. If you can’t find a space with white­boards, look for easels and easel pads, and in ei­ther case be sure to bring your own mark­ers. Also bring along spare pa­per, pens, and clip­boards, and as­sign some­one to make sure that there are snacks and drinks.

(A note about snacks and drinks: peo­ple al­most always un­der­es­ti­mate the im­por­tance of the qual­ity and quan­tity of food, an­chor­ing on some­thing like “I dunno, maybe just spend ten bucks on some chips or some­thing?” In­stead, ask your­self: what dol­lar value would I put on a 15% in­crease in the group’s abil­ity to think, over­all mood, and ul­ti­mate satis­fac­tion with the event? That’s how much you should con­sider spend­ing (/​ ask­ing MIRI to spend) on snacks, es­pe­cially for the first meet­ing. Don’t buy only junk food. It may give you more en­ergy tem­porar­ily, but it will make you worse at think­ing later. So, es­pe­cially for longer meet­ings, healthy snacks are crit­i­cal. Longer meet­ings should also in­clude a meal, per­haps at a nearby restau­rant. This also serves as a good break.)

At that first large meet­ing, you’ll want to start by for­mally elect­ing a pres­i­dent. This is an im­por­tant piece of com­mon-knowl­edge cul­ture—many times, the pres­i­dent won’t do much, but it’s ex­tremely use­ful to have a sin­gle per­son with the moral au­thor­ity to set agen­das, choose be­tween var­i­ous good op­tions, and keep the group on track. You may also end up elect­ing a sec­re­tary/​record-keeper, or pos­si­bly a co­or­di­na­tor to han­dle venue and food, or other offices (or you could do this af­ter a few meet­ings).

Next, you’ll want to model the pro­cess that has already been work­ing for you. Per­haps this means shar­ing a list of pre-ex­ist­ing ques­tions, and see­ing which cap­ture the in­ter­est of your par­ti­ci­pants. Per­haps it means dis­cussing the broader thrust of your re­search thus far be­fore brain­storm­ing some top­ics. Re­gard­less, you’ll want to get down to ac­tual think­ing, writ­ing, prov­ing, and dis­cussing as soon as you can. Break­ing into smaller groups is of­ten helpful if more than four peo­ple are at a meet­ing. If you do this, sched­ule a time to come back and share ideas.

Try to in­clude breaks in your struc­ture to keep ev­ery­one fresh. It can be difficult to re­mem­ber to take a break when things get go­ing, so it’s worth set­ting the in­ten­tion ahead of time. Short breaks ev­ery hour in which peo­ple get up and walk around are very helpful.

It can be helpful to keep a pub­lic list (on a white­board or shared Google doc) of ques­tions you have, needed con­cepts, and promis­ing ideas. This is an easy source of new top­ics if a con­ver­sa­tion runs dry.

One pos­si­ble struc­ture in­cor­po­rat­ing the above ad­vice and the re­search pro­ce­dure from the pre­vi­ous sec­tion:

  1. At the be­gin­ning of each meet­ing, ev­ery­one lists ques­tions/​top­ics/​con­fu­sions, which are writ­ten on a pub­lic list.

  2. Peo­ple make bids to start groups on top­ics they’re ex­cited about, and split off.

  3. Groups talk for 45 min­utes.

  4. Every­one re-gath­ers, and dis­cusses what hap­pened in the smaller groups for a few min­utes.

  5. Five or ten minute break, de­pend­ing on how peo­ple are feel­ing.

  6. New ques­tions/​ideas are added to the board, and the pro­cess re­peats as de­sired. (If you plan to do sev­eral cy­cles, also in­clude a longer break such as a meal some­where.)

At the end of the meet­ing, sched­ule the next event. You may have set­tled on a rough sched­ule which works for the core of the group, but you’ll still be ad­just­ing it meet­ing-to-meet­ing to ac­count for holi­days and other ab­sences. Con­firm­ing the next meet­ing time with ev­ery­one pre­sent is also im­por­tant for at­ten­dance, even if the meet­ing times are set in stone. Make sure to es­tab­lish at the out­set that you’re not go­ing to try to op­ti­mize for ev­ery­one’s availa­bil­ity at once; it’s good to have mee­tups that peo­ple feel okay skip­ping from time to time, as long as there’s some­thing like 70-90% con­sis­tency in the group. If one or two peo­ple can’t make it to the sec­ond meet­ing, be sure to get in­for­ma­tion from them so that you can pri­ori­tize their sched­ules a lit­tle more when plan­ning the third.

3. Models of so­cial dynamics

What fol­lows are some half-baked, ad-hoc mod­els of what makes for a good re­search group, or a good col­lab­o­ra­tive en­ter­prise in gen­eral. You should con­sider all of the fol­low­ing to be true in spirit but false in de­tail, and should try to de­rive your own value rather than treat­ing these as ac­tual sug­ges­tions to fol­low.

3A. Trans­mit­ters and receivers

We’ve found in our own re­search that con­ver­sa­tions tend to go bet­ter when they are pri­mar­ily be­tween two peo­ple. This is not to say that you shouldn’t have three or more peo­ple in­volved in the con­ver­sa­tion, but in any given five-minute span of time, there should mostly be just two peo­ple talk­ing—one who is cur­rently try­ing to con­vey some­thing, and an­other who is try­ing to un­der­stand (and whose un­der­stand­ing the first is speci­fi­cally op­ti­miz­ing for; dis­cussing a topic at a level such that four or five differ­ent peo­ple can all fol­low ev­ery­thing is usu­ally worse on net).

Call these two roles the “trans­mit­ter” and the “re­ceiver.” Things you might trans­mit:

  • A spe­cific ques­tion or confusion

  • A model or chain of reasoning

  • A piece of rele­vant back­ground in­for­ma­tion that needs to be deeply un­der­stood in or­der for the con­ver­sa­tion to proceed

Things the re­ceiver might do:

  • Mir­ror back to the trans­mit­ter what the trans­mit­ter just said, in differ­ent words. This lets the trans­mit­ter check where the trans­mis­sion has suc­ceeded or failed.

  • Take notes on a white­board, or at­tempt to draw di­a­grams, and have the trans­mit­ter ver­ify or cor­rect them. Do this as for­mally as you can. Try to write down state­ments in logic and turn in­for­mal ar­gu­ments into proofs. Type the­ory is good for this kind of re­ceiv­ing; just writ­ing down pre­cise data types cor­re­spond­ing to what’s be­ing dis­cussed can be very helpful.

  • Re­sist the im­pulse to round off what the trans­mit­ter is say­ing to some­thing you already un­der­stand. A good way to guard against this: at­tempt to find at least two in­ter­pre­ta­tions, and ask ques­tions which differ­en­ti­ate be­tween them.

  • Hold tight to the as­sump­tion that the thing the trans­mit­ter is try­ing to con­vey is in­ter­est­ing. Avoid “critic” mode that will tend to make it harder for the trans­mit­ter to think and ex­press freely. Even if there is a fatal flaw in what the trans­mit­ter is ex­plic­itly say­ing, your job is to help them dig up the spark of in­tu­ition which made them go down that path, so that they can turn it into a use­ful idea if pos­si­ble.

  • Stay closely in touch with con­fu­sion, and speak up where things don’t seem to make sense. Ask clar­ify­ing ques­tions. Your job as re­ceiver isn’t to just nod along or make the trans­mit­ter feel un­der­stood. Be gen­tle when nec­es­sary to help the trans­mit­ter get in touch with what they’re try­ing to con­vey; but once they’re in touch, your job is to re­ally get it out of them, in de­tail!

  • If the trans­mit­ter’s idea seems quite clear, the re­ceiver can start red-team­ing it, which means look­ing for at­tacks to make the ap­proach fail. Be­ing the critic when an idea isn’t prop­erly out yet blocks things up, but once there’s a firm pro­posal which seems to make sense, it’s open sea­son.

  • Look for im­pli­ca­tions of what the trans­mit­ter is say­ing. (“Ah, so then X!”; or, “Would that mean that X?”, etc.) This serves at least three pur­poses. First, it lets the trans­mit­ter know that you see why their idea would be to­tally awe­some if it worked. After all, you’re do­ing all these use­ful things with the idea. This helps keep things go­ing. Se­cond, it tests whether you see what they’re get­ting at. Third, a to­tally ab­surd im­pli­ca­tion can sug­gest that you’re down a wrong track and should back up to see where you took a wrong turn.

  • White-hat trol­ling or gad­fly-ing. Some­times there’s not much do­ing with the trans­mit­ter (or there’s no ac­tive trans­mit­ter; no one hav­ing ideas). Play the role of a mischievous Socrates. Ask ques­tions about seem­ingly ba­sic things and try to show why noth­ing any­body thinks makes any sense. Or, defend an ab­surd po­si­tion. (A troll may some­times seem like a trans­mit­ter, but is ac­tu­ally a re­ceiver.)

The trans­mit­ter should feel as free as pos­si­ble to just make claims, in­clud­ing “to­tally fake” claims, as long as they are keep­ing in touch with their in­tu­itions; try to es­tab­lish a norm where you can ask re­ceivers to col­lab­o­rate with you in un­cov­er­ing the ker­nel of truth in what you’re say­ing rather than shoot­ing down half-formed ideas be­cause they’re still half-wrong. No mat­ter how non­judge­men­tal the re­ceivers are, it may help the trans­mit­ter to say things like “ev­ery­thing I’m about to say is to­tally wrong, but” ev­ery so of­ten.

The trans­mit­ter should also re­main in touch with their in­tu­ition and cu­ri­os­ity, steer­ing the con­ver­sa­tion to what they think is most in­ter­est­ing rather than try­ing to perform or en­ter­tain. The trans­mit­ter is un­der no obli­ga­tion to an­swer the re­ceiver’s ques­tions; feel free to say “that’s not what I want to think about right now.”

The key idea is that the re­ceiver is helping mid­wife what the trans­mit­ter is say­ing. In that mo­ment, it is the trans­mit­ter’s think­ing that should take pri­or­ity, and the re­ceiver is act­ing as a sound­ing board, a liv­ing in­tu­ition pump, and a source of con­fu­sion and (minor) chaos.

Mean­while, any third par­ties in the au­di­ence should be try­ing to serve as fa­cil­i­ta­tors/​trans­la­tors. They should be watch­ing both the trans­mit­ter and the re­ceiver and seek­ing to model what’s go­ing on for those peo­ple. Where are they miss­ing each other, and talk­ing past each other? Where are they run­ning up against con­fir­ma­tion bias, or the dou­ble illu­sion of trans­parency? Where are they both agree­ing that some­thing makes sense with­out ac­tu­ally un­der­stand­ing it?

The au­di­ence mem­bers should speak up from time to time (prob­a­bly less than 10% of the to­tal words) to in­ject rele­vant thoughts or mod­els or ques­tions. Some­times, such an in­ter­jec­tion will be the cause of a role switch, with an au­di­ence mem­ber tak­ing on a new role as ei­ther trans­mit­ter or re­ceiver, and one of the other par­ties ro­tat­ing out.

3B. High stan­dards for membership

It’s awk­ward to not-in­vite some­one or to turn them away af­ter one or two meet­ings, but it’s even more awk­ward to wreck your en­tire MIRIx chap­ter be­cause you were too shy or too un­cer­tain to pro­tect it.

Have a clear dis­tinc­tion be­tween “wel­come to come to a meet­ing” and “is now a full part of the group.” Make sure that there is a known de­ci­sion-maker or set of de­ci­sion-mak­ers, and em­power them to make calls by fiat, with­out hav­ing to jus­tify or ex­plain. (If you don’t trust their judg­ment with­out ex­pla­na­tion, don’t have them be part of the de­ci­sion-mak­ing.) Trust your own in­stincts; if you don’t feel like some­one is a good match for the vibe you have go­ing, then don’t in­vite them in. Con­sider re­quiring mul­ti­ple recom­men­da­tions, or hav­ing an in­ter­view pro­cess. Th­ese may seem un­nec­es­sary, but it can be difficult to turn peo­ple away, and a for­mal pro­cess makes it feel more fair.

Also con­sider hav­ing for­mal eth­i­cal guidelines, or a group pledge or set of com­mit­ments, which peo­ple sign at the mo­ment that they fully join. Make sure that any stan­dards you set are ones you are will­ing to ac­tu­ally en­force (e.g. “you must come to half of all meet­ings” or “con­tent dis­cussed here is con­fi­den­tial un­less oth­er­wise stated”).

3C. Es­ca­lat­ing asks and rewards

Con­sider the model of a mar­tial arts academy. When you first ar­rive, the in­struc­tors ask a few small things of you (e.g. kick this tar­get, yell out loudly when you do so). Soon, they re­ward you for these things with a belt and some sta­tus.

At that point, the asks es­ca­late. Per­haps now, as a yel­low belt, you are put in charge of watch­ing some white belts for a few min­utes, and cor­rect­ing their form. In re­turn, they are told to bow to you and call you “sir” or “ma’am.”

As time goes on, the asks in­crease, and the re­wards in­crease com­men­su­rately. This cy­cle fosters com­mit­ment and in­vest­ment—it’s a pro­cess of slowly prov­ing to the in­di­vi­d­ual “if I put some­thing into this sys­tem, I will get some­thing out of it, and the more I put in, the more I’ll get out.” Even­tu­ally, you will re­ceive a black belt, and pos­si­bly be asked to join as a paid in­struc­tor or found your own branch of the school.

There is a similar dy­namic in most groups and or­ga­ni­za­tions. Groups which ask lit­tle or noth­ing of their mem­bers do not re­ceive loy­alty in re­turn. In­di­vi­d­u­als feel bought-in to a group to the ex­tent that that group al­lows them to tell pos­i­tive or epic sto­ries about them­selves.

The same will be true of your MIRIx chap­ter. Con­sider hav­ing some small, early asks that are the same for most new­com­ers (e.g., read such-and-such pa­per, or give a ten-minute talk on a topic of in­ter­est at your third meet­ing). Try to build a pipeline of greater asks and re­wards over time (e.g., on your fifth-ish meet­ing, we’d like you to take charge of set­ting the agenda and di­vid­ing up the groups).

3D. Struc­ture and elbow room

Re­lated to the pre­vi­ous, it’s im­por­tant that you bal­ance top-down and bot­tom-up struc­ture in your MIRIx. If there’s no clear sense of “how we do things,” then new­com­ers will flounder and have a bad time. You want there to be a pre-ex­ist­ing struc­ture that peo­ple can eval­u­ate, to de­ter­mine whether or not they feel like they fit into it. You want the “what’s this like?” of your group to be clearly visi­ble, right from the get-go, so that both peo­ple who are well-suited to it and peo­ple who aren’t can (for the most part) ac­cu­rately self-as­sort.

At the same time, you don’t want that struc­ture to feel limit­ing or con­fin­ing in the long run. Just as mar­tial artists even­tu­ally earn the right to de­ter­mine some of their own train­ing and the abil­ity to con­tribute to the agenda-set­ting and cur­ricu­lum of newer stu­dents, so too do you want the “pie” of your MIRIx to grow as time goes on. Other­wise, peo­ple will grow frus­trated by their in­abil­ity to bring the ful­l­ness of their own in­ter­ests and pri­ori­ties, and will leave to find a bet­ter con­text for their own growth and re­search.

3E. So­cial norms

That which is nor­mal and ac­cepted is that which goes un­challenged. If there is be­hav­ior that you want to dis­cour­age, you need to make sure not only that you challenge it when it oc­curs, but also that you openly, vo­cally, and pub­li­cly sup­port oth­ers who are challeng­ing it. It is the job of the group to en­sure that some­one who is fol­low­ing the rules/​try­ing to do it right is never alone when they are in con­flict with some­one who isn’t.

Con­sider in ad­vance, and be ex­plicit about, things like the ac­cept­abil­ity of in­ter­rup­tions or off-topic dis­cus­sion. Cul­ti­vate a cul­ture of dis­agree­ment, but be de­liber­ate about build­ing in po­lite­ness and sup­port so that dis­agree­ment is net-pos­i­tive and doesn’t turn into abuse or dele­gi­t­imiza­tion. Pro­tect what­ever de­ci­sion-mak­ing struc­tures you de­cide to put in place, and be con­sis­tent about what con­sti­tutes each per­son’s do­main and what marks the end of dis­cus­sion.

4. Other thoughts and questions

  1. Try to have a mix of top­ics or ac­tivi­ties, so that ev­ery meet­ing doesn’t fol­low the ex­act same pat­tern. Read pa­pers, give pre­sen­ta­tions, hold dis­cus­sions, write for­mal proofs or es­says, etc. Try to have fewer than 50% of your meet­ings cen­ter on read­ing and/​or dis­cussing pre-ex­ist­ing ma­te­rial. (Ideally, fewer than 33%.)

  2. Con­sider set­ting long-term agen­das, i.e., six months or a year of meet­ings that stay near a par­tic­u­lar swath of the ter­ri­tory and al­low for the group to build up a body of knowl­edge and progress.

  3. If set­ting a long-term agenda, build in wig­gle room for things that aren’t part of that swath (e.g., ev­ery third meet­ing is de­liber­ately not con­sis­tent with the over­all arc).

  4. Con­sider as­sign­ing some­one to take min­utes and col­late them in a per­ma­nent place, such that you can look back over the arc of a given sea­son or year. Con­sider whether or not it feels valuable to go over min­utes from the pre­vi­ous meet­ing at the start of each meet­ing.

  5. At the end of a meet­ing, as­sign some­one to col­lect and email out ques­tions that peo­ple in­tend to mull over, or thoughts that will lead into the next meet­ing. Con­sider de­cid­ing in ad­vance who will be lead­ing what at the next meet­ing, so they have a rea­son to pre­pare and to show up.

  6. En­sure that you have up-to-date con­tact in­for­ma­tion for all full mem­bers and as­so­ci­ated/​in­ter­ested par­ties. Think in ad­vance about whether you want to do email mes­sages, FB groups, in­di­vi­d­ual texts, etc.

  7. Con­sider what re­la­tion­ship you want to have with other MIRIx groups, such as shar­ing min­utes or ques­tions or oc­ca­sion­ally send­ing or re­ceiv­ing am­bas­sadors. Take agen­tic ac­tion in caus­ing such things to hap­pen, if you want them—re­mem­ber that you’re part of a class, and if you want it but never take steps to bring it about, this is prob­a­bly true of lots of other peo­ple as well.

  8. Con­sider whether you want to run events for the gen­eral pub­lic or po­ten­tial new re­cruits (e.g., in math or CS de­part­ments). Con­sider whether you want to try more am­bi­tious pro­jects, like the Hu­man-Aligned AI Sum­mer School, and reach out to peo­ple with knowl­edge and re­sources to do it well rather than rein­vent­ing the wheel.

  9. If your MIRIx chap­ter is in an aca­demic set­ting, be sure to figure out what sort of pipeline you want to form, so that you have un­der­class­men who are in­vested and ready to take over when the older stu­dents grad­u­ate. If not in an aca­demic set­ting, con­sider how you want to go about re­cruit­ing new mem­bers. Note that a large in­flux of new mem­bers is rarely use­ful, and com­pounds the cul­ture prob­lem; it’s bet­ter by far to add new peo­ple one or two at a time, with plenty of time to ac­cul­turate.

  10. Re­mem­ber that the qual­ity of the re­search and dis­cus­sions and the MIRIx chap­ter as a whole is de­pen­dent on the ac­tions of in­di­vi­d­u­als, and how those ac­tions com­bine. Be sure to im­press this upon ev­ery mem­ber—your MIRIx is only as good as each of you in­di­vi­d­u­ally chooses to make it.


You’ve nearly reached the end of the doc­u­ment! Hope­fully, this con­tained non-zero use­ful in­for­ma­tion, as well as a healthy amount of food-for-thought. Be­fore you go, we recom­mend that you take 30 sec­onds or so to pon­der each of the fol­low­ing ques­tions:

  • Why did we choose to write this doc­u­ment? What were we ex­pect­ing from it, and what caused us to se­lect this par­tic­u­lar for­mat and con­tent, out of all of the pos­si­bil­ities?

  • Where are you still hun­gry or frus­trated or dis­satis­fied? What’s miss­ing from this doc­u­ment, that we failed to ad­dress? How did you come to be aware of this/​these thing/​s that we missed?

  • What sort of doc­u­ment would you write? How would you know if it was a good idea to write one, or not? How would you de­cide what to put into it?

  • How the hell does progress even get made?

Happy hunt­ing,

- The MIRI re­search team