riceissa(Issa Rice)

• riceissa 4 Apr 2020 1:23 UTC

  3 points

  on: Take­aways from safety by de­fault interviews

  What is the plan go­ing for­ward for in­ter­views? Are you plan­ning to in­ter­view peo­ple who are more pes­simistic?

• riceissa 1 Apr 2020 7:30 UTC

  1 point

  in reply to: Tommy Crow's comment on: Cat­e­go­riza­tion of Meta-Eth­i­cal The­o­ries (a flowchart)

  In the first cat­e­go­riza­tion scheme, I’m also not ex­actly sure what nihilism is refer­ring to. Do you know? Is it just refer­ring to Er­ror The­ory (and maybe in­co­her­en­tism)?

  Yes, Hue­mer writes: “Nihilism (a.k.a. ‘the er­ror the­ory’) holds that eval­u­a­tive state­ments are gen­er­ally false.”

  Usu­ally non-cog­ni­tivism would fall within nihilism, no?

  I’m not sure how the term “nihilism” is typ­i­cally used in philo­soph­i­cal writ­ing, but if we take nihilism=er­ror the­ory then it looks like non-cog­ni­tivism wouldn’t fall within nihilism (just like non-cog­ni­tivism doesn’t fall within er­ror the­ory in your flowchart).

  I ac­tu­ally don’t think ei­ther of these di­a­grams place Nihilism cor­rectly.

  For the first di­a­gram, Hue­mer writes “if we say ‘good’ pur­ports to re­fer to a prop­erty, some things have that prop­erty, and the prop­erty does not de­pend on ob­servers, then we have moral re­al­ism.” So for Hue­mer, nihilism fails the mid­dle con­di­tion, so is clas­sified as anti-re­al­ist. For the sec­ond di­a­gram, see the quote be­low about du­al­ism vs monism.

  I’m not su­per well ac­quainted with the monism/​du­al­ism dis­tinc­tion, but in the com­mon con­cep­tion don’t they both gen­er­ally as­sume that moral­ity is real, at least in some semi-ro­bust sense?

  Hue­mer writes:

  Here, du­al­ism is the idea that there are two fun­da­men­tally differ­ent kinds of facts (or prop­er­ties) in the world: eval­u­a­tive facts (prop­er­ties) and non-eval­u­a­tive facts (prop­er­ties). Only the in­tu­ition­ists em­brace this.

  Every­one else is a monist: they say there is only one fun­da­men­tal kind of fact in the world, and it is the non-eval­u­a­tive kind; there aren’t any value facts over and above the other facts. This im­plies that ei­ther there are no value facts at all (elimi­na­tivism), or value facts are en­tirely ex­pli­ca­ble in terms of non-eval­u­a­tive facts (re­duc­tion­ism).

• riceissa 1 Apr 2020 6:42 UTC

  LW: 5 AF: 3

  AF

  in reply to: jessicata's comment on: How spe­cial are hu­man brains among an­i­mal brains?

  It seems like “agri­cul­tural rev­olu­tion” is used to mean both the be­gin­ning of agri­cul­ture (“First Agri­cul­tural Revolu­tion”) and the 18th cen­tury agri­cul­tural rev­olu­tion (“Se­cond Agri­cul­tural Revolu­tion”).

• riceissa 30 Mar 2020 20:19 UTC

  3 points

  on: Cat­e­go­riza­tion of Meta-Eth­i­cal The­o­ries (a flowchart)

  Michael Hue­mer gives two tax­onomies of metaeth­i­cal views in sec­tion 1.4 of his book Eth­i­cal In­tu­ition­ism:

  As the pre­ced­ing sec­tion sug­gests, metaeth­i­cal the­o­ries are tra­di­tion­ally di­vided first into re­al­ist and anti-re­al­ist views, and then into two forms of re­al­ism and three forms of anti-re­al­ism:

             Nat­u­ral­ism
            /​
     Real­ism
    /​       \
   /​         In­tu­ition­ism
  /​
  \
   \              Sub­jec­tivism
    \            /​
     Anti-Real­ism—Non-Cog­ni­tivism
                 \
                  Nihilism
  

  This is not the most illu­mi­nat­ing way of clas­sify­ing po­si­tions. It im­plies that the most fun­da­men­tal di­vi­sion in metaethics is be­tween re­al­ists and anti-re­al­ists over the ques­tion of ob­jec­tivity. The dis­pute be­tween nat­u­ral­ism and in­tu­ition­ism is then seen as rel­a­tively minor, with the nat­u­ral­ists be­ing much closer to the in­tu­ition­ists than they are, say, to the sub­jec­tivists. That isn’t how I see things. As I see it, the most fun­da­men­tal di­vi­sion in metaethics is be­tween the in­tu­ition­ists, on the one hand, and ev­ery­one else, on the other. I would clas­sify the po­si­tions as fol­lows:

     Dual­ism—In­tu­ition­ism
    /​
   /​                      Sub­jec­tivism
  /​                      /​
  \          Re­duc­tion­ism
   \        /​            \
    \      /​              Nat­u­ral­ism
     Mon­ism
           \               Non-Cog­ni­tivism
            \             /​
             Elimi­na­tivism
                          \
                           Nihilism
  

• riceissa 20 Mar 2020 1:19 UTC

  1 point

  in reply to: lc's comment on: Open & Wel­come Thread—March 2020

  Do you have prior po­si­tions on re­la­tion­ships that you don’t want to get cor­rupted through the dat­ing pro­cess, or some­thing else?

  I think that’s one way of putting it. I’m fine with my prior po­si­tions on re­la­tion­ships chang­ing be­cause of bet­ter in­tro­spec­tion (aided by dat­ing), but not fine with my prior po­si­tions chang­ing be­cause they are get­ting cor­rupted.

  In­tel­li­gence be­yond your cone of tol­er­ance is usu­ally a trait that peo­ple pur­sue be­cause they think it’s “eth­i­cal”

  I’m not sure I un­der­stand what you mean. Could you try re-stat­ing this in differ­ent words?

• riceissa 20 Mar 2020 0:04 UTC

  1 point

  on: Open & Wel­come Thread—March 2020

  A ques­tion about ro­man­tic re­la­tion­ships: Let’s say cur­rently I think that a girl needs to have a cer­tain level of smart­ness in or­der for me to date her long-term/​marry her. Sup­pose I then start dat­ing a girl and de­cide that ac­tu­ally, be­ing smart isn’t as im­por­tant as I thought be­cause the girl makes up for it in other ways (e.g. be­ing very pretty/​pleas­ant/​sub­mis­sive). I think this kind of change of mind is le­gi­t­i­mate in some cases (e.g. be­cause I got bet­ter at figur­ing out what I value in a woman) and ille­gi­t­i­mate in other cases (e.g. be­cause the girl I’m dat­ing man­aged to se­duce me and mess up my in­tro­spec­tion). My ques­tion is, is this dis­tinc­tion real, and if so, is there any way for me to tell which situ­a­tion I am in (le­gi­t­i­mate vs ille­gi­t­i­mate change of mind) once I’ve already be­gun dat­ing the girl?

  This prob­lem arises be­cause I think dat­ing is im­por­tant for in­tro­spect­ing about what I want, i.e. there is a point af­ter which I can no longer ob­tain new in­for­ma­tion about my prefer­ences via think­ing alone. The prob­lem is that dat­ing is also po­ten­tially a val­ues-cor­rupt­ing pro­cess, i.e. dat­ing some­one who doesn’t meet cer­tain crite­ria I think I might have means that I can get trapped in a re­la­tion­ship.

  I’m also cu­ri­ous to hear if peo­ple think this isn’t a big prob­lem (and if so, why).

• riceissa 16 Mar 2020 23:52 UTC

  LW: 3 AF: 2

  AF

  in reply to: romeostevensit's comment on: What are some ex­er­cises for build­ing/​gen­er­at­ing in­tu­itions about key dis­agree­ments in AI al­ign­ment?

  I have only a very vague idea of what you mean. Could you give an ex­am­ple of how one would do this?

• riceissa 9 Mar 2020 23:09 UTC

  LW: 1 AF: 1

  AF

  in reply to: johnswentworth's comment on: Name of Prob­lem?

  I think that makes sense, thanks.

• riceissa 9 Mar 2020 22:30 UTC

  LW: 3 AF: 2

  AF

  in reply to: johnswentworth's comment on: Name of Prob­lem?

  Just to make sure I un­der­stand, the first few ex­pan­sions of the sec­ond one are:

  • f(n)

  • f(n+1)

  • f((n+1) + 1)

  • f(((n+1) + 1) + 1)

  • f((((n+1) + 1) + 1) + 1)

  Is that right? If so, wouldn’t the in­finite ex­pan­sion look like f((((...) + 1) + 1) + 1) in­stead of what you wrote?

• riceissa 8 Mar 2020 7:43 UTC

  LW: 3 AF: 2

  AF

  in reply to: TurnTrout's comment on: Co­her­ence ar­gu­ments do not im­ply goal-di­rected behavior

  I read the post and parts of the pa­per. Here is my un­der­stand­ing: con­di­tions similar to those in The­o­rem 2 above don’t ex­ist, be­cause Alex’s pa­per doesn’t take an ar­bi­trary util­ity func­tion and prove in­stru­men­tal con­ver­gence; in­stead, the idea is to set the re­wards for the MDP ran­domly (by sam­pling i.i.d. from some dis­tri­bu­tion) and then show that in most cases, the agent seeks “power” (states which al­low the agent to ob­tain high re­wards in the fu­ture). So it avoids the twitch­ing robot not by say­ing that it can’t make use of ad­di­tional re­sources, but by say­ing that the twitch­ing robot has an atyp­i­cal re­ward func­tion. So even though there aren’t con­di­tions similar to those in The­o­rem 2, there are still con­di­tions analo­gous to them (in the struc­ture of the ar­gu­ment “ex­pected util­ity/​re­ward max­i­miza­tion + X im­plies catas­tro­phe”), namely X = “the re­ward func­tion is typ­i­cal”. Does that sound right?

  Writ­ing this com­ment re­minded me of Oliver’s com­ment where X = “agent wasn’t speci­fi­cally op­ti­mized away from goal-di­rect­ed­ness”.

• riceissa 7 Mar 2020 23:33 UTC

  LW: 2 AF: 1

  AF

  in reply to: Matthew Barnett's comment on: Co­her­ence ar­gu­ments do not im­ply goal-di­rected behavior

  Can you say more about Alex Turner’s for­mal­ism? For ex­am­ple, are there con­di­tions in his pa­per or post similar to the con­di­tions I named for The­o­rem 2 above? If so, what do they say and where can I find them in the pa­per or post? If not, how does the pa­per avoid the twitch­ing robot from seek­ing con­ver­gent in­stru­men­tal goals?

• riceissa 7 Mar 2020 21:25 UTC

  LW: 5 AF: 3

  AF

  on: Co­her­ence ar­gu­ments do not im­ply goal-di­rected behavior

  One ad­di­tional source that I found helpful to look at is the pa­per “For­mal­iz­ing Con­ver­gent In­stru­men­tal Goals” by Tsvi Ben­son-Tilsen and Nate Soares, which tries to for­mal­ize Omo­hun­dro’s in­stru­men­tal con­ver­gence idea us­ing math. I read the pa­per quickly and skipped the proofs, so I might have mi­s­un­der­stood some­thing, but here is my cur­rent in­ter­pre­ta­tion.

  The key as­sump­tions seem to ap­pear in the state­ment of The­o­rem 2; these as­sump­tions state that us­ing ad­di­tional re­sources will al­low the agent to im­ple­ment a strat­egy that gives it strictly higher util­ity (com­pared to the util­ity it could achieve if it didn’t make use of the ad­di­tional re­sources). There­fore, any op­ti­mal strat­egy will make use of those ad­di­tional re­sources (kil­ling hu­mans in the pro­cess). In the Bit Uni­verse ex­am­ple given in the pa­per, if the agent doesn’t ter­mi­nally care what hap­pens in some par­tic­u­lar re­gion $h$ (I guess they chose this let­ter be­cause it’s sup­posed to rep­re­sent where hu­mans are), but $h$ con­tains re­sources that can be burned to in­crease util­ity in other re­gions, the agent will burn those re­sources.

  Both Ro­hin’s and Jes­sica’s twitch­ing robot ex­am­ples seem to vi­o­late these as­sump­tions (if we were to trans­late them into the for­mal­ism used in the pa­per), be­cause the robot can­not make use of ad­di­tional re­sources to ob­tain a higher util­ity.

  For me, the up­shot of look­ing at this pa­per is some­thing like:

  • MIRI peo­ple don’t seem to be ar­gu­ing that ex­pected util­ity max­i­miza­tion alone im­plies catas­tro­phe.

  • There are some ad­di­tional con­di­tions that, when taken to­gether with ex­pected util­ity max­i­miza­tion, seem to give a pretty good ar­gu­ment for catas­tro­phe.

  • Th­ese ad­di­tional con­di­tions don’t seem to have been ar­gued for (or at least, this spe­cific pa­per just as­sumes them).

• riceissa 7 Mar 2020 5:06 UTC

  1 point

  on: What does Solomonoff in­duc­tion say about brain du­pli­ca­tion/​con­scious­ness?

  Lan­rian’s men­tion of UDASSA made me search for dis­cus­sions of UDASSA again, and in the pro­cess I found Hal Fin­ney’s 2005 post “Ob­server-Mo­ment Mea­sure from Uni­verse Mea­sure”, which seems to be de­scribing UDASSA (though it doesn’t men­tion UDASSA by name); it’s the clear­est dis­cus­sion I’ve seen so far, and goes into de­tail about how the part that “reads off” the cam­era in­puts from the phys­i­cal world works.

  I also found this post by Wei Dai, which seems to be where UDASSA was first pro­posed.

• riceissa 6 Mar 2020 0:32 UTC

  3 points

  in reply to: Daniel Kokotajlo's comment on: What does Solomonoff in­duc­tion say about brain du­pli­ca­tion/​con­scious­ness?

  My ver­sion: Solomonoff In­duc­tion is solip­sis­tic phe­nom­e­nal ideal­ism.

  I don’t un­der­stand what this means (even search­ing “phe­nom­e­nal ideal­ism” yields very few re­sults on google, and none that look es­pe­cially rele­vant). Have you writ­ten up your ver­sion any­where, or do you have a link to ex­plain what solip­sis­tic phe­nom­e­nal ideal­ism or phe­nom­e­nal ideal­ism mean? (I un­der­stand solip­sism and ideal­ism already; I just don’t know how they com­bine and what work the “phe­nom­e­nal” part is do­ing.)

• riceissa 6 Mar 2020 0:31 UTC

  2 points

  in reply to: jessicata's comment on: What does Solomonoff in­duc­tion say about brain du­pli­ca­tion/​con­scious­ness?

  Thanks, that’s definitely re­lated. I had ac­tu­ally read that post when it was first pub­lished, but didn’t quite un­der­stand it. Reread­ing the post, I feel like I un­der­stand it much bet­ter now, and I ap­pre­ci­ate hav­ing the con­nec­tion pointed out.

• riceissa 6 Mar 2020 0:29 UTC

  1 point

  in reply to: Donald Hobson's comment on: What does Solomonoff in­duc­tion say about brain du­pli­ca­tion/​con­scious­ness?

  I might have mi­s­un­der­stood your com­ment, but it sounds like you’re say­ing that Solomonoff in­duc­tion isn’t nat­u­ral­ized/​em­bed­ded, and that this is a prob­lem (sort of like in this post). If so, I’m fine with that, and the point of my ques­tion was more like, “given this flawed-but-in­ter­est­ing model (Solomonoff in­duc­tion), what does it say about this ques­tion that I’m in­ter­ested in (con­scious­ness)?”

• riceissa 6 Mar 2020 0:03 UTC

  1 point

  in reply to: Pattern's comment on: What does Solomonoff in­duc­tion say about brain du­pli­ca­tion/​con­scious­ness?

  I’m not sure I un­der­stand. The bit se­quence that Solomonoff in­duc­tion re­ceives (af­ter the point where the cam­era is du­pli­cated) will ei­ther con­tain the cam­era in­puts for just one cam­era, or it will con­tain cam­era in­puts for both cam­eras. (There are also other pos­si­bil­ities, like maybe the in­puts will just be blank.) I ex­plained why I think it will just be the cam­era in­puts for one cam­era rather than two (namely, track­ing the lo­ca­tions of two cam­eras re­quires a longer pro­gram). Do you have an ex­pla­na­tion of why “both, sep­a­rately” is more likely? (I’m as­sum­ing that “both, sep­a­rately” is the same thing as the bit se­quence con­tain­ing cam­era in­puts for both cam­eras. If not, please clar­ify what you mean by “both, sep­a­rately”.)

• riceissa 2 Mar 2020 2:34 UTC

  3 points

  on: De­caf vs. reg­u­lar coffee self-experiment

  I’ve no­ticed that for me, caf­feine with­drawal re­ally be­gins (and is worst) on the sec­ond day I stop drink­ing coffee. In your ex­per­i­ment, if the coin flips went some­thing like reg­u­lar, de­caf, reg­u­lar, de­caf, …, then I don’t think I would no­tice a huge differ­ence be­tween the reg­u­lar and de­caf days (de­spite there be­ing a very no­tice­able differ­ence be­tween drink­ing coffee af­ter ab­sti­nence, caf­feine with­drawal, and a reg­u­lar sober/​caf­feinated day).

  Here is a ran­dom ar­ti­cle which says “Typ­i­cally, on­set of [caf­feine with­drawal] symp­toms oc­curred 12–24 h af­ter ab­sti­nence, with peak in­ten­sity at 20–51 h, and for a du­ra­tion of 2–9 days.” (I haven’t looked at this ar­ti­cle in de­tail, so I don’t know how good the sci­ence is.)

  My sug­ges­tion would be to use larger “blocks” of days (e.g. 3-day blocks) so that caf­feine with­drawal/​in­tro­duc­tion be­comes more ob­vi­ous. Maybe the eas­iest would be to drink the same grounds for a week (flip­ping a coin once to de­ter­mine which to start with).

• riceissa 25 Feb 2020 6:47 UTC

  2 points

  in reply to: Rob Bensinger's comment on: Two clar­ifi­ca­tions about “Strate­gic Back­ground”

  Thanks! I have some re­main­ing ques­tions:

  • The post says “On our cur­rent view of the tech­nolog­i­cal land­scape, there are a num­ber of plau­si­ble fu­ture tech­nolo­gies that could be lev­er­aged to end the acute risk pe­riod.” I’m won­der­ing what these other plau­si­ble fu­ture tech­nolo­gies are. (I’m guess­ing things like whole brain em­u­la­tion and in­tel­li­gence en­hance­ment count, but are there any oth­ers?)

  • One of the foot­notes says “There are other paths to good out­comes that we view as lower-prob­a­bil­ity, but still suffi­ciently high-prob­a­bil­ity that the global com­mu­nity should al­lo­cate marginal re­sources to their pur­suit.” What do some of these other paths look like?

  • I’m con­fused about the differ­ences be­tween “min­i­mal al­igned AGI” and “task AGI”. (As far as I know, this post is the only place MIRI has used the term “min­i­mal al­igned AGI”, so I have very lit­tle to go on.) Is “min­i­mal al­igned AGI” the larger class, and “task AGI” the spe­cific kind of min­i­mal al­igned AGI that MIRI has de­cided is most promis­ing? Or is the plan to first build a min­i­mal al­igned AGI, which then builds a task AGI, which then performs a pivotal task/​helps build a Sovereign?

    • If the lat­ter, then it seems like MIRI has gone from a one-step view (“build a Sovereign”), to a two-step view (“build a task-di­rected AGI first, then go for Sovereign”), to a three-step view (“build a min­i­mal al­igned AGI, then task AGI, then Sovereign”). I’m not sure why “three” is the right num­ber of stages (why not two or four?), and I don’t think MIRI has ex­plained this. In fact, I don’t think MIRI has even ex­plained why it switched to the two-step view in the first place. (Wei Dai made this point here.)

• riceissa 24 Feb 2020 5:37 UTC

  2 points

  in reply to: Luke A Somers's comment on: Ar­gu­ments about fast takeoff

  It’s from the linked post un­der the sec­tion “Univer­sal­ity thresh­olds”.