G Gordon Worley III

• G Gordon Worley III 15 Jan 2020 23:29 UTC

  LW: 2 AF: 1

  AF

  on: Mal­ign gen­er­al­iza­tion with­out in­ter­nal search

  How­ever, it’s worth not­ing that say­ing the agent is mis­taken about the state of the world is re­ally an an­thro­po­mor­phiza­tion. It was ac­tu­ally perfectly cor­rect in in­fer­ring where the red part of the world was—we just didn’t want it to go to that part of the world. We model the agent as be­ing ‘mis­taken’ about where the land­ing pad is, but it works equally well to model the agent as hav­ing goals that are counter to ours.

  That we can flip our per­spec­tive like this sug­gests to me that think­ing of the agent as hav­ing differ­ent goals is likely still an­thro­po­mor­phic or at least tele­olog­i­cal rea­son­ing that re­sults from us mod­el­ing this agent has hav­ing dis­po­si­tions it doesn’t ac­tu­ally have.

  I’m not sure what to offer as an al­ter­na­tive since we’re not talk­ing about a cat­e­gory where I feel grounded enough to see clearly what might be re­ally go­ing on, much less offer a more use­ful ab­strac­tion that avoids this prob­lem, but I think it’s worth con­sid­er­ing that there’s a deeper con­fu­sion here that this ex­poses but doesn’t re­solve.

• G Gordon Worley III 14 Jan 2020 22:18 UTC

  3 points

  on: Book re­view: Re­think­ing Consciousness

  Now put the two to­gether, and you get an “at­ten­tion schema”, an in­ter­nal model of at­ten­tion (i.e., of the ac­tivity of the GNW). The at­ten­tion schema is sup­pos­edly key to the mys­tery of con­scious­ness.

  The idea of an at­ten­tion schema helps make sense of a thing talked about in med­i­ta­tion. In zen we talk some­times about it via the metaphor of the mind like a mir­ror such that it sees it­self re­flect­ing in it­self. In The Mind Illu­mi­nated it’s referred to as metacog­ni­tive aware­ness. The point is that the pro­cess by which the mind op­er­ates can be ob­served by it­self even as it op­er­ates, and and per­haps the at­ten­tion schema is an im­por­tant part of what it means to do that, speci­fi­cally caus­ing the at­ten­tion schema to be able to model it­self.

• G Gordon Worley III 14 Jan 2020 20:23 UTC

  3 points

  on: What is the re­la­tion­ship be­tween Prefer­ence Learn­ing and Value Learn­ing?

  The short an­swer is that yes, they are re­lated and ba­si­cally about the same thing. How­ever the ap­proaches of re­searchers vary a lot.

  Rele­vant con­sid­er­a­tions that come to mind:

  • The ex­tent to which val­ues/​prefer­ences are legible

  • The ex­tent to which they are discoverable

  • The ex­tent to which they are hid­den variables

  • The ex­tent to which they are normative

  • How im­por­tant im­me­di­ate im­ple­mentabil­ity is

  • How im­por­tant ex­treme op­ti­miza­tion is

  • How im­por­tant safety con­cerns are

  The re­sult is that I think there is some­thing of a di­vide be­tween safety-fo­cused re­searchers and ca­pa­bil­ities-fo­cused re­searchers in this area due to differ­ent as­sump­tions and that makes each oth­ers work not very in­ter­est­ing/​rele­vant to the other cluster.

• G Gordon Worley III 14 Jan 2020 20:14 UTC

  4 points

  on: Ascetic aesthetic

  So I think you are right about the way aes­thet­ics power eth­i­cal rea­son­ing, and I think aes­thet­ics is just a way­point on the causal mechanism of gen­er­at­ing eth­i­cal judge­ments, be­cause aes­thet­ics are ul­ti­mately about what we value (how we com­pare things for var­i­ous pur­poses), and what we value is a func­tion of valence. So to the ex­tent I agree it’s to the ex­tent that I see ethics and aes­thet­ics as ap­pli­ca­tions of valence to differ­ent do­mains.

• G Gordon Worley III 14 Jan 2020 20:00 UTC

  8 points

  on: Re­pos­sess­ing Degrees

  Another pos­si­bly use­ful data point in this dis­cus­sion: I spent 6 years work­ing on my PhD be­fore drop­ping out. On my re­sume I men­tion that I worked on a PhD but didn’t re­ceive it (to ex­plain what I was do­ing for all those years of my life). Mul­ti­ple things hap­pen as a re­sult of this:

  • some peo­ple don’t read care­fully and just think I have a PhD

  • some peo­ple are ex­cited that I dropped out of one (that’s a com­mon Bay Area re­ac­tion)

  • some peo­ple ig­nore ev­ery­thing un­der “Ed­u­ca­tion” and only look at work experience

  I sus­pect that hav­ing a re­pos­sessed PhD would re­ceive similar sorts of re­ac­tions to hav­ing dropped out be­fore grad­u­a­tion.

• G Gordon Worley III 14 Jan 2020 1:29 UTC

  7 points

  in reply to: jimrandomh's comment on: Re­pos­sess­ing Degrees

  On the other hand it would work in cases where a de­gree is nec­es­sary to li­cen­sure. For ex­am­ple, de­gree re­pos­ses­sion would be effec­tive against pro­fes­sions who need a de­gree as part of their cer­tifi­ca­tion to prac­tice. If it is not the case already, this could be made the case for doc­tors, nurses, lawyers, ac­coun­tants, pro­fes­sional en­g­ineers, etc. such that there would be real im­pacts of loss of de­gree even if you could still tell peo­ple you had it be­cause you wouldn’t be al­lowed to prac­tice your pro­fes­sion in the same ca­pac­ity as be­fore with­out your li­cense.

  (Whether or not li­cens­ing is a good policy is a sep­a­rate ques­tion from the con­sid­er­a­tion of how the mechanism of de­gree re­pos­ses­sion might work.)

• G Gordon Worley III 10 Jan 2020 2:12 UTC

  2 points

  on: Ra­tion­al­ist Scrip­tures?

  Ar­guably the LessWrong Wiki is some­thing like your de­sired com­piled list of ra­tio­nal­ist wis­dom, even if it is old and out of date.

• G Gordon Worley III 9 Jan 2020 21:20 UTC

  LW: 2 AF: 1

  AF

  in reply to: Stuart_Armstrong's comment on: Prefer­ence syn­the­sis illus­trated: Star Wars

  I’d de­scribe that as a statis­ti­cal reg­u­lar­ity over statis­ti­cal reg­u­lar­i­ties over prefer­ences.

• G Gordon Worley III 9 Jan 2020 19:27 UTC

  LW: 2 AF: 1

  AF

  on: Prefer­ence syn­the­sis illus­trated: Star Wars

  But the “meta-prefer­ences” are a bit more wor­ry­ing. Are they gen­uine meta-prefer­ences? Espe­cially since the sec­ond one is one that was more sub­con­scious, and the third one looks more like a stan­dard prefer­ence than a meta-prefer­ence. If the cat­e­gory of meta-prefer­ence is not clear, then that part of the re­search agenda needs to be im­proved.

  I think one of the challenges is that, to me at least, it’s still un­clear if we re­ally have any­thing like meta-prefer­ences that be­have in sys­tem­atic ways. That is, is there a sys­tem­atic way in which our highly con­di­tional prefer­ences (which, in a very real sense, ex­ist only mo­men­tar­ily at a par­tic­u­lar de­ci­sion point situ­ated within the causal his­tory of the uni­verse) com­bine such that we can say more than that there are some statis­ti­cal reg­u­lar­i­ties to our prefer­ences. Our prefer­ences may man­age to have some co­her­ent statis­ti­cal fea­tures about which we can make some stochas­ti­cally con­sis­tent state­ments, but I think this falls short of what we are usu­ally hop­ing for in terms of meta-prefer­ences, and cer­tainly seems to fall short in terms of how I un­der­stand you to be think­ing about them (though maybe I mi­s­un­der­stand you: I think of you of think­ing of meta-prefer­ences as some­thing that can ul­ti­mately be made to have nice math­e­mat­i­cal prop­er­ties, like some ver­sion of ra­tio­nal­ity, that would al­low them to be op­ti­mized against with­out weird things hap­pen­ing).

• G Gordon Worley III 9 Jan 2020 19:04 UTC

  LW: 2 AF: 1

  AF

  in reply to: shminux's comment on: (Dou­ble-)In­verse Embed­ded Agency Problem

  I’m ac­tu­ally not re­ally sure. We have some vague no­tion that, for ex­am­ple, my prefer­ence for eat­ing pizza shouldn’t re­sult in at­tempts at un­bounded pizza eat­ing max­i­miza­tion, and I would prob­a­bly be un­happy from my cur­rent val­ues if a max­i­miz­ing agent saw I liked pizza the best of all foods and then pro­ceeded to feed me only pizza for­ever, even if it mod­ified me such that I would max­i­mally en­joy the pizza each time and not get bored of it.

  Think­ing more in terms of re­gres­sional Good­hart­ing, maybe some­thing like not de­vi­at­ing from the true tar­get be­cause of op­ti­miz­ing for the mea­sure of it. Con­sider the clas­sic rat ex­ter­mi­na­tion ex­am­ple of Good­hart­ing. We already know col­lect­ing rat tails as ev­i­dence of ex­ter­mi­na­tion is a func­tion that leads to weird effects. Does there ex­ist a func­tion that mea­sures rat ex­ter­mi­na­tions that, when op­ti­mized for, pro­duces the in­tended effect (ex­ter­mi­na­tion of rats) with­out do­ing any­thing “weird”, e.g. gen­er­at­ing un­in­tended side-effects, max­i­miz­ing rat re­pro­duc­tion so we can ex­ter­mi­nate more of them, just straight­for­wardly leads to the ex­tinc­tion of rats and noth­ing else.

• G Gordon Worley III 8 Jan 2020 22:33 UTC

  LW: 2 AF: 1

  AF

  on: (Dou­ble-)In­verse Embed­ded Agency Problem

  Think­ing about my fo­cus on a the­ory of hu­man val­ues for AI al­ign­ment, the prob­lem is quite hard when we ask for a way to pre­cisely spec­ify val­ues. I might state the prob­lem as some­thing like find­ing “a the­ory of hu­man val­ues ac­cu­rate and pre­cise enough that its pre­dic­tions don’t come apart un­der ex­treme op­ti­miza­tion”. To bor­row Is­nasene’s no­ta­tion, here X = “a the­ory of hu­man val­ues ac­cu­rate and pre­cise enough” and Y = “its pre­dic­tions don’t come apart un­der ex­treme op­ti­miza­tion”.

  So what is an in­verse prob­lem with X’ and Y’? A Y’ might be some­thing like “func­tions that be­have as ex­pected un­der ex­treme op­ti­miza­tion”, where “be­have as ex­pected” is some­thing like no Good­hart effects. We could even just be more nar­row and make Y’ = “func­tions that don’t ex­ist Good­hart effects un­der ex­treme op­ti­miza­tion”. Then the X’ would be some­thing like a gen­er­al­ized de­scrip­tion of the classes of func­tions that satisfy Y’.

  Do­ing the dou­ble in­verse, we would try to find X from X’ by look­ing at what prop­er­ties hold for this class of func­tions that don’t suffer from Good­hart­ing, and use them to help us iden­tify what would be needed to cre­ate an ad­e­quate the­ory of hu­man val­ues.

• G Gordon Worley III 8 Jan 2020 22:13 UTC

  5 points

  on: Strip­ping Away the Protections

  Re mazes ev­ery­where, we should of course also pay at­ten­tion to what ex­tent our own com­mu­nity is an im­moral maze. Cf. a similar anal­y­sis from nearly a year ago in re­sponse to a Robin Han­son post.

• G Gordon Worley III 8 Jan 2020 21:02 UTC

  3 points

  in reply to: RedMan's comment on: How should I dis­pose of a dan­ger­ous idea?

  I think a se­ri­ous com­pli­ca­tion arises in this sce­nario:

  • I dis­cover a dan­ger­ous idea at age 20, and get a re­verse patent on it as de­scribed here.

  • At age 60 I learn I am ter­mi­nally ill and don’t be­lieve there is any mechanism by which my ex­is­tence can be car­ried for­ward (e.g. cry­on­ics). I am given 1 year to live.

  • I make the dan­ger­ous idea pub­lic and col­lect and large sum for hav­ing kept quiet for years so I can en­joy my last year of life, even if the world doesn’t con­tinue much be­yond that be­cause it’s de­stroyed by my dan­ger­ous idea.

• G Gordon Worley III 8 Jan 2020 0:52 UTC

  4 points

  on: Mo­ral­ity vs re­lated concepts

  To the ex­tent that ax­iol­ogy is about val­ues (what is good/​bad), it is about prefer­ences (what would one rather do), and is thus tied to de­ci­sion the­ory in that it offers the place from which num­bers gets as­signed to differ­ent de­ci­sions even if it doesn’t say how to choose among them. I as­sume most peo­ple are fa­mil­iar with prefer­ences and it may or may not be very rele­vant for your work as it’s already pretty similar to is­sues in moral­ity that re­quires choos­ing be­tween op­tions, but thought it worth men­tion­ing.

• G Gordon Worley III 6 Jan 2020 21:25 UTC

  2 points

  in reply to: Raemon's comment on: Clump­ing Sols­tice Sin­ga­longs in Groups of 2-4

  If you are rea­son­ably skil­led at singing along with such, there is ba­si­cally no such thing has hit­ting wrong notes (un­less the song is de­liber­ately con­fus­ing). In­stead, you end up hit­ting har­mony notes. Ba­si­cally, each note con­strains the note that come af­ter­wards and there are only a few “valid” op­tions.

  [...]

  And then by the mid­dle of Sols­tice, when you get to things like Brighter Than To­day (a mod­er­ately com­pli­cated song), it’s ac­tu­ally an achiev­able ask to sing along if you haven’t heard it be­fore.

  So I no­ticed this year that in par­tic­u­lar songs like “Brighter Than To­day” are mu­si­cally com­pli­cated in ways that made them harder to sing along to than some of the oth­ers. Maybe this prob­lem is less of an is­sue in, for ex­am­ple, the smaller space where a stronger feed­back loop to­wards good singing is cre­ated, but my ex­pe­rience of it was that peo­ple were strug­gling with the melody, hit­ting notes at the right time, and gen­er­ally not stum­bling over the words.

  For ex­am­ple, I ex­pe­rience “Brighter Than To­day” has hav­ing some weird timing change ups in the cho­rus where syl­la­bles sud­denly and un­ex­pect­edly get short­ened to stay within me­ter. I heard this most strongly around the sec­ond line of the cho­rus, “al­though the night is cold”, where it seems like peo­ple start trip­ping over them­selves to keep up. I no­ticed other places in this song and other songs hav­ing similar effects. Maybe some­thing else is go­ing on mu­si­cally, but that’s how my semi-trained ear per­ceives it.

  I think I ig­nored this in the past be­cause of some com­bi­na­tion of the songs be­ing new, ev­ery­one be­ing new to Sols­tice, there be­ing differ­ent lev­els of em­pha­sis on sin­ga­long in the past, and fac­tors like those you men­tioned that made the singing of the songs at differ­ent venues differ­ent. But this year we are not so new to Sols­tice, the songs are not so new, and there was a stronger effort to make sin­ga­long an im­por­tant com­po­nent of the event.

  It made me yearn for more sim­pler and fa­mil­iar songs that fit with the theme gen­er­ally and less songs that were crafted to fit Sols­tice tightly but that are more com­plex and less fa­mil­iar.

• G Gordon Worley III 6 Jan 2020 20:51 UTC

  4 points

  on: Prop­a­gat­ing Facts into Aesthetics

  I’m mak­ing a fairly strong claim (weakly held) that “is it beau­tiful or ugly?” is at least one of the im­por­tant ques­tions to be ask­ing, in ad­di­tion to “is cap­i­tal­ism good/​bad” and “does rais­ing min­i­mum wage help or harm work­ers?”. Not be­cause it’s how a flawless AI would think about it, but be­cause it’s how hu­mans seem to of­ten think about it.

  What is an aes­thetic?

  An aes­thetic is a mish­mash of val­ues, strate­gies, and on­tolo­gies that re­in­force each other.

  I sus­pect it’s be­cause these are re­ally all the same thing, unified by a com­mon men­tal mechanism. “Aes­thetic” is as good a name for the nat­u­ral cat­e­gory as any.

• G Gordon Worley III 6 Jan 2020 20:03 UTC

  2 points

  in reply to: Wei_Dai's comment on: The Uni­verse Doesn’t Have to Play Nice

  A lot of places where it seems like the uni­verse isn’t play­ing nice might just be a re­flec­tion of us not be­ing smart enough or not hav­ing thought long enough.

  I guess there can be some dis­agree­ment on what con­sti­tutes “a lot”, but it seems to me that some of these are not sub­ject to this be­cause they are proofs of limi­ta­tions that can only be got­ten around by ei­ther prag­matic means (as­sum­ing away the prob­lem by limit­ing your­self the cases where it doesn’t arise) or by re­lax­ing the strength of what the mechanism claims. Of the ex­am­ples listed, the prob­lem of per­cep­tion (Chris calls it the prob­lem of skep­ti­cism), Gödel’s the­o­rem, the prob­lem of in­duc­tion, the prob­lem of the prior (Bayesi­anism), and qualia (the prob­lem of per­cep­tion again) all seem fun­da­men­tal in the sense that they are prob­lems that arise from within the ab­strac­tion/​sys­tem, so they can only be got­ten around by prag­ma­tism or tran­scen­dence/​re­lax­ation, and can­not pos­si­bly be­come more tractable by think­ing about them more, even if we come up with bet­ter ways to deal with them prag­mat­i­cally or to bet­ter tran­scend them by us­ing differ­ent ab­strac­tions. In this sense the uni­verse will con­tinue to look like it doesn’t play nice from within those ab­strac­tions for­ever and always be­cause it is a fea­ture of the way those ab­strac­tions re­late to re­al­ity.

• G Gordon Worley III 6 Jan 2020 19:16 UTC

  2 points

  on: Clump­ing Sols­tice Sin­ga­longs in Groups of 2-4

  While I re­ally like the idea of mul­ti­ple songs clumped to­gether and hav­ing them be sin­ga­longs, I feel like many of the songs at, for ex­am­ple, the 2019 Bay Area Sols­tice, were not easy to sing along to. Aside from a hand­ful that are ei­ther well known pop­u­lar songs or folks songs de­signed to be sung well the first time through, I feel like a lot of the songs suffer from be­ing difficult to sign right on the first try un­less you already are fairly fa­mil­iar with them, and for many (maybe most) at­ten­dees that is just not the case (or at least I and a lot of peo­ple near me strug­gled with a lot of the songs). So I like this idea, but I think it only works if the songs are bet­ter op­ti­mized for singing along.

• G Gordon Worley III 6 Jan 2020 19:11 UTC

  11 points

  in reply to: bgold's comment on: bgold’s Shortform

  Bab­ble and prune seems related

• G Gordon Worley III 5 Jan 2020 18:47 UTC

  3 points

  on: [Book Re­view] The Trou­ble with Physics

  Does there any­where ex­ist a com­mu­nity of folks you might work on this with who aren’t de­ri­sive of this ap­proach, woe­fully un­in­formed on the speci­fics (I think that’s broadly the is­sue here on LW; we’re not se­lect­ing for physi­cists of the sort you’d like to talk to even if there are a lot of them here), or cranks?

  My ex­pe­rience has been that it’s very helpful to have sup­port­ive folks who can at least a lit­tle bit ap­pre­ci­ate what you are do­ing and offer feed­back, even if you are go­ing su­per deep in some di­rec­tion such that you are the ex­pert in ways that means they will of­ten be mis­taken even if they can help you avoid mak­ing ob­vi­ous mis­takes.