riceissa

• riceissa 9 Oct 2019 21:14 UTC

  3 points

  on: Oc­cam’s Ra­zor May Be Suffi­cient to In­fer the Prefer­ences of Ir­ra­tional Agents: A re­ply to Arm­strong & Mindermann

  I thought about this more and re-read the A&M pa­per, and I now have a differ­ent line of think­ing com­pared to my pre­vi­ous com­ments.

  I still think A&M’s No Free Lunch the­o­rem goes through, but now I think A&M are prov­ing the wrong the­o­rem. A&M try to find the sim­plest (plan­ner, re­ward) de­com­po­si­tion that is com­pat­i­ble with the hu­man policy, but it seems like we in­stead ad­di­tion­ally want com­pat­i­bil­ity with all the ev­i­dence we have ob­served, in­clud­ing sen­sory data of hu­mans say­ing things like “if I was more ra­tio­nal, I would be ex­er­cis­ing right now in­stead of watch­ing TV” and “no re­ally, my re­ward func­tion is not empty”. The im­por­tant point is that such sen­sory data gives us in­for­ma­tion not just about the hu­man policy, but also about the de­com­po­si­tion. Forc­ing com­pat­i­bil­ity with this sen­sory data seems to rule out de­gen­er­ate pairs. This makes me feel like Oc­cam’s Ra­zor would work for in­fer­ring prefer­ences up to a cer­tain point (i.e. as long as the situ­a­tions are all “in-dis­tri­bu­tion”).

  If we are try­ing to find the (plan­ner, re­ward) de­com­po­si­tion of non-hu­man minds: I think if we were ran­domly handed a mind from all of mind de­sign space, then A&M’s No Free Lunch the­o­rem would ap­ply, be­cause the sim­plest ex­pla­na­tion re­ally is that the mind has a de­gen­er­ate de­com­po­si­tion. But if we were ran­domly handed an alien mind from our uni­verse, then we would be able to use all the facts we have learned about our uni­verse, in­clud­ing how the aliens likely evolved, any state­ments they seem to be mak­ing about what they value, and so on.

  Does this line of think­ing also ap­ply to the case of sci­ence? I think not, be­cause we wouldn’t be able to use our ob­ser­va­tions to get in­for­ma­tion about the de­com­po­si­tion. Un­like the case of val­ues, the nat­u­ral world isn’t mak­ing state­ments like “ac­tu­ally, the laws are empty and all the com­plex­ity is in the ini­tial con­di­tions”. I still don’t think the No Free Lunch the­o­rem works for sci­ence ei­ther, be­cause of my pre­vi­ous com­ments.

• riceissa 9 Oct 2019 7:50 UTC

  5 points

  on: List of re­solved con­fu­sions about IDA

  Seems odd to have the ideal­is­tic goal get to be the stan­dard name, and the dime-a-dozen failure mode be a longer name that is more con­fus­ing.

  I agree this is con­fus­ing.

  Is there a rea­son why the stan­dard terms are not be­ing used to re­fer to the stan­dard, short-term re­sults?

  As far as I know, Paul hasn’t ex­plained his choice in de­tail. One rea­son he does men­tion, in this com­ment, is that in the con­text of strat­egy-steal­ing, prefer­ences like “help me stay in con­trol and be well-in­formed” do not make sense when in­ter­preted as prefer­ences-as-elic­ited, since the cur­rent user has no way to know if they are in con­trol or well-in­formed.

  In the post Wei con­trasts “cur­rent” and “ac­tual” prefer­ences. “Stated” vs “re­flec­tive” prefer­ences also seem like nice al­ter­na­tives too.

  I think cur­rent=elic­ited=stated, but ac­tual≈re­flec­tive (be­cause there is the pos­si­bil­ity that un­der­go­ing re­flec­tion isn’t a good way to find out our ac­tual prefer­ences, or as Paul says ‘There’s a hy­poth­e­sis that “what I’d say af­ter some par­tic­u­lar ideal­ized pro­cess of re­flec­tion” is a rea­son­able way to cap­ture “ac­tual pre­fer­ences,” but I think that’s up for de­bate—e.g. it could fail if me-on-re­flec­tion is self­ish and has val­ues op­posed to cur­rent-me, and cer­tainly it could fail for any par­tic­u­lar pro­cess of re­flec­tion and so it might just hap­pen to be the case that there is no pro­cess of re­flec­tion that satis­fies it.’)

• riceissa 9 Oct 2019 6:41 UTC

  1 point

  on: List of re­solved con­fu­sions about IDA

  I think Paul calls that “prefer­ences-as-elic­ited”, so if we’re talk­ing about act-based agents, it would be “short-term prefer­ences-as-elic­ited” (see this com­ment).

• riceissa 9 Oct 2019 5:16 UTC

  4 points

  on: List of re­solved con­fu­sions about IDA

  My un­der­stand­ing is that Paul never meant to in­tro­duce the term “nar­row prefer­ences” (i.e. “nar­row” is not an ad­jec­tive that ap­plies to prefer­ences), and the fact that he talked about nar­row prefer­ences in the act-based agents post was an ac­ci­dent/​some­thing he no longer en­dorses.

  In­stead, when Paul says “nar­row”, he’s talk­ing not about prefer­ences but about nar­row vs am­bi­tious value learn­ing. This is what Paul means when he says “I’ve only ever used [the term “nar­row”] in the con­text of value learn­ing, in or­der to make this par­tic­u­lar dis­tinc­tion be­tween two differ­ent goals you might have when do­ing value learn­ing.”

  See also this com­ment and the am­bi­tious vs nar­row value learn­ing post.

• riceissa 9 Oct 2019 4:40 UTC

  1 point

  on: Oc­cam’s Ra­zor May Be Suffi­cient to In­fer the Prefer­ences of Ir­ra­tional Agents: A re­ply to Arm­strong & Mindermann

  Thanks for the ex­pla­na­tion, I think I un­der­stand this bet­ter now.

  My re­sponse to your sec­ond point: I wasn’t sure how the se­quence pre­dic­tion ap­proach to in­duc­tion (like Solomonoff in­duc­tion) deals with coun­ter­fac­tu­als, so I looked it up, and it looks like we can con­vert the coun­ter­fac­tual ques­tion into a se­quence pre­dic­tion ques­tion by ap­pend­ing the coun­ter­fac­tual to all the data we have seen so far. So in the nu­clear launch codes ex­am­ple, we would feed the se­quence pre­dic­tor with a video of the launch codes be­ing posted to the in­ter­net, and then ask it to pre­dict what se­quence it ex­pects to see next. (See the top of page 9 of this PDF and also ex­am­ple 5.2.2 in Li and Vi­tanyi for more de­tails and fur­ther ex­am­ples.) This doesn’t re­quire a de­com­po­si­tion into laws and con­di­tions; rather it seems to re­quire that the events E be a func­tion that can take in bits and print out more bits (or a prob­a­bil­ity dis­tri­bu­tion over bits). But this doesn’t seem like a prob­lem, since in the val­ues case the policy π is also a func­tion. (Maybe my real point is that I don’t un­der­stand why you are as­sum­ing E has to be a se­quence of events?) [ETA: ac­tu­ally, maybe E can be just a se­quence of events, but if we’re talk­ing about com­plex­ity, there would be some pro­gram that gen­er­ates E, so I am sug­gest­ing we use that pro­gram in­stead of L and C for coun­ter­fac­tual rea­son­ing.]

  My re­sponse to your first point: I am far from an ex­pert here, but my guess is that an Oc­cam’s Ra­zor ad­vo­cate would bite the bul­let and say this is fine, since ei­ther (1) the de­gen­er­ate pre­dic­tors will have high com­plex­ity so will be dom­i­nated by sim­pler pre­dic­tors, or (2) we are just as likely to be liv­ing in a “de­gen­er­ate” world as we are to be liv­ing in the kind of “pre­dictable” world that we think we are liv­ing in.

• riceissa 7 Oct 2019 22:00 UTC

  5 points

  on: Oc­cam’s Ra­zor May Be Suffi­cient to In­fer the Prefer­ences of Ir­ra­tional Agents: A re­ply to Arm­strong & Mindermann

  I’m not con­fi­dent I’ve un­der­stood this post, but it seems to me that the differ­ence be­tween the val­ues case and the em­piri­cal case is that in the val­ues case, we want to do bet­ter than hu­mans at achiev­ing hu­man val­ues (this is the “am­bi­tious” in “am­bi­tious value learn­ing”) whereas in the em­piri­cal case, we are fine with just pre­dict­ing what the uni­verse does (we aren’t try­ing to pre­dict the uni­verse even bet­ter than the uni­verse it­self). In the for­mal­ism, in π = P(R) we are af­ter R (rather than π), but in E = L(C) we are af­ter E (rather than L or C), so in the lat­ter case it doesn’t mat­ter if we get a de­gen­er­ate pair (be­cause it will still pre­dict the fu­ture events well). Similarly, in the val­ues case, if all we wanted was to imi­tate hu­mans, then it seems like get­ting a de­gen­er­ate pair would be fine (it would act just as hu­man as the “in­tended” pair).

  If we use Oc­cam’s Ra­zor alone to find law-con­di­tion pairs that fit all the world’s events, we’ll set­tle on one of the de­gen­er­ate ones (or some­thing else en­tirely) rather than a rea­son­able one. This could be very dan­ger­ous if we are e.g. build­ing an AI to do sci­ence for us and an­swer coun­ter­fac­tual ques­tions like “If we had posted the nu­clear launch codes on the In­ter­net, would any nukes have been launched?”

  I don’t un­der­stand how this con­clu­sion fol­lows (un­less it’s about the ma­lign prior, which seems not rele­vant here). Could you give more de­tails on why an­swer­ing coun­ter­fac­tual ques­tions like this would be dan­ger­ous?

  What links here?

  • riceissa's comment on Oc­cam’s Ra­zor May Be Suffi­cient to In­fer the Prefer­ences of Ir­ra­tional Agents: A re­ply to Arm­strong & Mindermann by Daniel Kokotajlo (9 Oct 2019 21:14 UTC; 3 points)

• riceissa 6 Oct 2019 23:08 UTC

  4 points

  on: What do the baby eaters tell us about ethics?

  Eliezer has writ­ten a se­quence on meta-ethics. I won­der if you’re aware of it? (If you are, my next ques­tion is why you don’t con­sider it an an­swer to your ques­tion.)

  Another thought I’ve had since I read the story is that it seems like a lot of hu­man-hu­man in­ter­ac­tions are re­ally hu­man-babyeater in­ter­ac­tions.

  I think Un­der-ac­knowl­edged Value Differ­ences makes the same point.

• riceissa 2 Oct 2019 23:12 UTC

  1 point

  on: LW Team Up­dates—Oc­to­ber 2019

  On LessWrong’s GraphiQL, I no­ticed that hov­er­ing over key­words no longer pro­vides doc­u­men­ta­tion help. (See here for what the hover-over used to look like.) Would it be pos­si­ble to turn this back on?

• riceissa 2 Oct 2019 23:08 UTC

  4 points

  on: LW Team Up­dates—Oc­to­ber 2019

  On textboxes for com­ments/​posts, I no­ticed that the doc­u­men­ta­tion is for the rich text ed­i­tor re­gard­less of what op­tion I have set in prefer­ences (I have mark­down). To be clear, mark­down for­mat­ting works; it’s just that the doc­u­men­ta­tion tells me to e.g. use Cmd-4 for LaTeX rather than dol­lar signs. I tried switch­ing to rich text and back to mark­down, but the prob­lem per­sists.

• riceissa 2 Oct 2019 22:55 UTC

  1 point

  on: Three ways that “Suffi­ciently op­ti­mized agents ap­pear co­her­ent” can be false

  I be­lieve there’s rea­son to think that Eliezer never in­tended “Suffi­ciently op­ti­mized agents ap­pear co­her­ent” to have an air­tight ar­gu­ment and be uni­ver­sally true.

  On the Ar­bital ver­sion of the page (but not the GreaterWrong ver­sion you linked to) [ETA: I just re­al­ized that you did link to the Ar­bital ver­sion, but I was view­ing it on GW] one can see that Eliezer as­signed 85% prob­a­bil­ity to the claim (though it’s not clear if the un­cer­tainty is more like “I tried to make an air­tight uni­ver­sal ar­gu­ment, but it might be wrong” or more like “I tried to show that this will hap­pen in most cases, but there are also cases where I don’t think it will hap­pen”).

• riceissa 2 Oct 2019 21:57 UTC

  2 points

  on: World State is the Wrong Level of Ab­strac­tion for Impact

  I ap­pre­ci­ate this clar­ifi­ca­tion, but when I wrote my com­ment, I hadn’t read the origi­nal AUP post or the pa­per, since I as­sumed this se­quence was sup­posed to ex­plain AUP start­ing from scratch (so I didn’t have the idea of aux­iliary set when I wrote my com­ment).

• riceissa 2 Oct 2019 0:21 UTC

  5 points

  on: World State is the Wrong Level of Ab­strac­tion for Impact

  It seems like one down­side of im­pact in the AU sense is that in or­der to figure out whether an ac­tion has high im­pact, the AI needs to have a de­tailed un­der­stand­ing of hu­man val­ues and the on­tol­ogy used by hu­mans. (This is in con­trast to the state-based mea­sures of im­pact, where calcu­lat­ing the im­pact of a state change seems eas­ier.) Without such an un­der­stand­ing, the AI seems to ei­ther do noth­ing (in or­der to pre­vent it­self from caus­ing bad kinds of high im­pact) or make a bunch of mis­takes. So my feel­ing is that in or­der to ac­tu­ally im­ple­ment an AI that does not cause bad kinds of high im­pact, we would need to make progress on value learn­ing (but once we’ve made progress on value learn­ing, it’s not clear to me what AU the­ory adds in terms of in­creased safety).

• riceissa 1 Oct 2019 20:26 UTC

  4 points

  on: What fund­ing sources ex­ist for tech­ni­cal AI safety re­search?

  Po­ten­tial ad­di­tions: Paul Chris­ti­ano, Fu­ture of Life In­sti­tute, EA Grants.

• riceissa 1 Oct 2019 9:31 UTC

  3 points

  on: AI Safety “Suc­cess Sto­ries”

  Cor­rigible Contender

  A semi-au­tonomous AGI that does not have long-term prefer­ences of its own but acts ac­cord­ing to (its un­der­stand­ing of) the short-term prefer­ences of some hu­man or group of humans

  In light of re­cent dis­cus­sion, it seems like this part should be clar­ified to say “ac­tual prefer­ences” or “short-term prefer­ences-on-re­flec­tion”.

  Also in the table, for Cor­rigible Con­tender should the re­li­ance on hu­man safety be changed from “High” to “Medium”? (My feel­ing is that since the AI isn’t rely­ing on the cur­rent hu­mans’ elic­ited prefer­ences, the re­li­ance on hu­man safety would be some­where be­tween that of Sovereign Sin­gle­ton and Pivotal Tool.)

  (I’m mak­ing these sug­ges­tions mainly be­cause I ex­pect peo­ple will con­tinue to re­fer to this post in the fu­ture.)

• riceissa 1 Oct 2019 6:23 UTC

  5 points

  on: List of re­solved con­fu­sions about IDA

  I still feel con­fused about “dis­till ≈ RL”. In RL+Imi­ta­tion (which I as­sume is also talk­ing about dis­til­la­tion, and which was writ­ten af­ter Semi-su­per­vised re­in­force­ment learn­ing), Paul says things like “In the same way that we can rea­son about AI con­trol by tak­ing as given a pow­er­ful RL sys­tem or pow­er­ful gen­er­a­tive mod­el­ing, we could take as given a pow­er­ful solu­tion to RL+imi­ta­tion. I think that this is prob­a­bly a bet­ter as­sump­tion to work with” and “Go­ing for­ward, I’ll prefer­en­tially de­sign AI con­trol schemes us­ing imi­ta­tion+RL rather than imi­ta­tion, epi­sodic RL, or some other as­sump­tion”.

  Was there a later place where Paul went back to just RL? Or is RL+Imi­ta­tion about some­thing other than dis­til­la­tion? Or is the imi­ta­tion part such a small con­tri­bu­tion that writ­ing “dis­till ≈ RL” is still ac­cu­rate?

  ETA: From the FAQ for Paul’s agenda:

  1.2.2: OK, so given this am­plified al­igned agent, how do you get the dis­til­led agent?

  Train a new agent via some com­bi­na­tion of imi­ta­tion learn­ing (pre­dict­ing the ac­tions of the am­plified al­igned agent), semi-su­per­vised re­in­force­ment learn­ing (where the am­plified al­igned agent helps spec­ify the re­ward), and tech­niques for op­ti­miz­ing ro­bust­ness (e.g. cre­at­ing red teams that gen­er­ate sce­nar­ios that in­cen­tivize sub­ver­sion).

  and:

  The imi­ta­tion learn­ing is more about get­ting this new agent off the ground than about en­sur­ing al­ign­ment. The bulk of the al­ign­ment guaran­tee comes from the semi-su­per­vised re­in­force­ment learn­ing, where we train it to work on a wide range of tasks and an­swer ques­tions about its cog­ni­tion.

• riceissa 30 Sep 2019 22:49 UTC

  9 points

  on: List of re­solved con­fu­sions about IDA

  I used to think that af­ter the ini­tial dis­til­la­tion step, the AI would be ba­si­cally hu­man-level. Now I un­der­stand that af­ter the ini­tial dis­til­la­tion step, the AI will be su­per­hu­man in some re­spects and sub­hu­man in oth­ers, but wouldn’t be “ba­si­cally hu­man” in any sense. Source

• riceissa 29 Sep 2019 23:22 UTC

  LW: 5 AF: 3

  AF

  on: Utility ≠ Reward

  To me, it seems like the two dis­tinc­tions are differ­ent. There seem to be three lev­els to dis­t­in­guish:

  1. The re­ward (in the re­in­force­ment learn­ing sense) or the base ob­jec­tive (ex­am­ple: in­clu­sive ge­netic fit­ness for hu­mans)

  2. A mechanism in the brain that dis­penses plea­sure or pro­vides a proxy for the re­ward (ex­am­ple: plea­sure in hu­mans)

  3. The ac­tual goal/​util­ity that the agent ends up pur­su­ing (ex­am­ple: a re­flec­tive equil­ibrium for some hu­man’s val­ues, which might have noth­ing to do with plea­sure or in­clu­sive ge­netic fit­ness)

  The base ob­jec­tive vs mesa-ob­jec­tive dis­tinc­tion seems to be about (1) vs a com­bi­na­tion of (2) and (3). The re­ward max­i­mizer vs util­ity max­i­mizer dis­tinc­tion seems to be about (2) vs (3), or maybe (1) vs (3).

  Depend­ing on the agent that is con­sid­ered, only some of these lev­els may be pre­sent:

  • A “dumb” RL-trained agent that en­gages in re­ward gam­ing. Only level (1), and there is no mesa-op­ti­mizer.

  • A “dumb” RL-trained agent that en­gages in re­ward tam­per­ing. Only level (1), and there is no mesa-op­ti­mizer.

  • A pa­per­clip max­i­mizer built from scratch. Only level (3), and there is no mesa-op­ti­mizer.

  • A rel­a­tively “dumb” mesa-op­ti­mizer trained us­ing RL might have just (1) (the base ob­jec­tive) and (2) (the mesa-ob­jec­tive). This kind of agent would be in­cen­tivized to tam­per with its plea­sure cir­cuitry (in the sense of (2)), but wouldn’t be in­cen­tivized to tam­per with its RL-re­ward cir­cuitry. (Ex­am­ple: rats wire­head to give them­selves MAX_PLEASURE, but don’t self-mod­ify to de­lude them­selves into think­ing they have left many de­scen­dants.)

  • If the train­ing pro­ce­dure some­how coughs up a mesa-op­ti­mizer that doesn’t have a “plea­sure cen­ter” in its brain (I don’t know how this would hap­pen, but it seems log­i­cally pos­si­ble), there would just be (1) (the base ob­jec­tive) and (3) (the mesa-ob­jec­tive). This kind of agent wouldn’t try to tam­per with its util­ity func­tion (in the sense of (3)), nor would it try to tam­per with its RL-re­ward/​base-ob­jec­tive to de­lude it­self into think­ing it has high re­wards.

  ETA: Here is a table that shows these dis­tinc­tions vary­ing in­de­pen­dently:

  	Utility max­i­mizer	Re­ward max­i­mizer
  Op­ti­mizes for base ob­jec­tive (i.e. mesa-op­ti­mizer ab­sent)	Paper­clip max­i­mizer	“Dumb” RL-trained agent
  Op­ti­mizes for mesa-ob­jec­tive (i.e. mesa-op­ti­mizer pre­sent)	Hu­man in re­flec­tive equil­ibrium	Rats

• riceissa 28 Sep 2019 0:38 UTC

  LW: 22 AF: 6

  AF

  on: The strat­egy-steal­ing assumption

  Like Wei Dai, I am also find­ing this dis­cus­sion pretty con­fus­ing. To sum­ma­rize my state of con­fu­sion, I came up with the fol­low­ing list of ways in which prefer­ences can be short or long:

  1. time hori­zon and time dis­count­ing: how far in the fu­ture is the prefer­ence about? More gen­er­ally, how much weight do we place on the pre­sent vs the fu­ture?

  2. act-based (“short”) vs goal-based (“long”): us­ing the hu­man’s (or more gen­er­ally, the hu­man-plus-AI-as­sis­tants’; see (6) be­low) es­ti­mate of the value of the next ac­tion (act-based) or do­ing more open-ended op­ti­miza­tion of the fu­ture based on some goal, e.g. us­ing a util­ity func­tion (goal-based)

  3. amount of re­flec­tion the hu­man has un­der­gone: “short” would be the cur­rent hu­man (I think this is what you call “prefer­ences-as-elic­ited”), and this would get “longer” as we give the hu­man more time to think, with some­thing like CEV/​Long Reflec­tion/​Great De­liber­a­tion be­ing the “longest” in this sense (I think this is what you call “prefer­ence-on-ideal­ized-re­flec­tion”). This sense fur­ther breaks down into whether the hu­man it­self is ac­tu­ally do­ing the re­flec­tion, or if the AI is in­stead pre­dict­ing what the hu­man would think af­ter re­flec­tion.

  4. how far the search hap­pens: “short” would be a limited search (that lacks in­sight/​doesn’t see in­ter­est­ing con­se­quences) and “long” would be a search that has in­sight/​sees in­ter­est­ing con­se­quences. This is a dis­tinc­tion you made in a dis­cus­sion with Eliezer a while back. This dis­tinc­tion also isn’t strictly about prefer­ences, but rather about how one would achieve those prefer­ences.

  5. de dicto (“short”) vs de re (“long”): This is a dis­tinc­tion you made in this post. I think this is the same dis­tinc­tion as (2) or (3), but I’m not sure which. (But if my in­ter­pre­ta­tion of you be­low is cor­rect, I guess this must be the same as (2) or else a com­pletely differ­ent dis­tinc­tion.)

  6. un­der­stand­able (“short”) vs evaluable (“long”): A course of ac­tion is un­der­stand­able if the hu­man (with­out any AI as­sis­tants) can un­der­stand the ra­tio­nale be­hind it; a course of ac­tion is evaluable if there is some pro­ce­dure the hu­man can im­ple­ment to eval­u­ate the ra­tio­nale us­ing AI as­sis­tants. I guess there is also a “not even evaluable” op­tion here that is even “longer”. (Thanks to Wei Dai for bring­ing up this dis­tinc­tion, al­though I may have mi­s­un­der­stood the ac­tual dis­tinc­tion.)

  My in­ter­pre­ta­tion is that when you say “short-term prefer­ences-on-re­flec­tion”, you mean short in sense (1), ex­cept when the AI needs to gather re­sources, in which case ei­ther the hu­man or the AI will need to do more long-term plan­ning; short in sense (2); long in sense (3), with the AI pre­dict­ing what the hu­man would think af­ter re­flec­tion; long in sense (4); short in sense (5); long in sense (6). Does this sound right to you? If not, I think it would help me a lot if you could “fill in the list” with which of short or long you choose for each point.

  As­sum­ing my in­ter­pre­ta­tion is cor­rect, my con­fu­sion is that you say we shouldn’t ex­pect a situ­a­tion where “the user-on-re­flec­tion might be happy with the level of cor­rigi­bil­ity, but the user them­selves might be un­happy” (I take you to be talk­ing about sense (3) from above). It seems like the user-on-re­flec­tion and the cur­rent user would dis­agree about many things (that is the whole point of re­flec­tion), so if the AI acts in ac­cor­dance with the in­ten­tions of the user-on-re­flec­tion, the cur­rent user is likely to end up un­happy.

• riceissa 25 Sep 2019 0:25 UTC

  4 points

  on: Two clar­ifi­ca­tions about “Strate­gic Back­ground”

  That post says “We plan to say more in the fu­ture about the crite­ria for strate­gi­cally ad­e­quate pro­jects in 7a” and also “A num­ber of the points above re­quire fur­ther ex­pla­na­tion and mo­ti­va­tion, and we’ll be pro­vid­ing more de­tails on our view of the strate­gic land­scape in the near fu­ture”. As far as I can tell, MIRI hasn’t pub­lished any fur­ther ex­pla­na­tion of this strate­gic plan (I ex­pected there to be some­thing in the 2018 up­date but that post talks about other things). Is MIRI still plan­ning to say more about its strate­gic plan in the near fu­ture, and if so, is there a con­crete timeframe (e.g. “in a few months”, “in a year”, “in two years”) for pub­lish­ing such an ex­pla­na­tion?

• riceissa 24 Sep 2019 9:27 UTC

  5 points

  on: What are the differ­ences be­tween all the iter­a­tive/​re­cur­sive ap­proaches to AI al­ign­ment?

  Thanks. It looks like all the re­al­is­tic ex­am­ples I had of weak HCH are ac­tu­ally ex­am­ples of strong HCH af­ter all, so I’m look­ing for some ex­am­ples of weak HCH to help my un­der­stand­ing. I can see how weak HCH would com­pute the an­swer to a “nat­u­rally lin­ear re­cur­sive” prob­lem (like com­put­ing fac­to­ri­als) but how would weak HCH an­swer a ques­tion like “Should I get laser eye surgery?” (to take an ex­am­ple from here). The nat­u­ral way to de­com­pose a prob­lem like this seems to use branch­ing.

  Also, I just looked again at Alex Zhu’s FAQ for Paul’s agenda, and Alex’s ex­pla­na­tion of weak HCH (in sec­tion 2.2.1) seems to im­ply that it is do­ing tree re­cur­sion (e.g. “I some­times pic­ture this as an in­finite tree of hu­mans-in-boxes, who can break down ques­tions and pass them to other hu­mans-in-boxes”). It seems like ei­ther you or Alex must be mis­taken here, but I have no idea which.

  What links here?

  • [AN #66]: De­com­pos­ing ro­bust­ness into ca­pa­bil­ity ro­bust­ness and al­ign­ment robustness by rohinmshah (30 Sep 2019 18:00 UTC; 11 points)