G Gordon Worley III

• G Gordon Worley III 16 Oct 2019 18:04 UTC

  4 points

  on: TAISU 2019 Field Report

  One thing I for­got to in­clude in the post was that I en­joyed the work­shop so much that I in­tend to try to bring it to the Bay Area (likely to be speci­fi­cally held in Berkeley) in 2020 if that doesn’t hap­pen oth­er­wise so that more Bay Area AI safety re­searchers (cur­rent and as­piring) will have an easy op­por­tu­nity to par­ti­ci­pate in it. This doesn’t re­quire a lot of money but it does a lit­tle bit, mostly to pay for Linda’s travel, ac­com­mo­da­tions, time, and en­ergy (I ex­pect I can ar­range for space for lit­tle to no money, as well as cheap ac­com­mo­da­tions for par­ti­ci­pants who would be trav­el­ing).

  If that sounds in­ter­est­ing ei­ther let me know via pri­vate mes­sage or com­ment here. If you’re in­ter­ested in fi­nan­cially con­tribut­ing to putting some­thing like that on (my es­ti­mate is it will cost no more than $5000 to­tal, and prob­a­bly less), also reach out to me via pri­vate mes­sage.

• G Gordon Worley III 16 Oct 2019 17:55 UTC

  6 points

  in reply to: capybaralet's comment on: TAISU 2019 Field Report

  So, first let me give you some rea­sons it was valuable to me, which I think will also be true for other peo­ple:

  • It cre­ated space for re­con­sid­er­ing AI safety from the ground up, which is im­por­tant be­cause I can of­ten be­come trapped by my plans once they have been set in mo­tion.

  • It offered an op­por­tu­nity to learn from and teach oth­ers about AI safety, in­clud­ing those who I wouldn’t think would have some­thing to teach me, usu­ally by say­ing weird things that knocked me out of lo­cal max­ima cre­ated by be­ing rel­a­tively im­mersed in the field, but also by teach­ing me about things I thought I un­der­stood but didn’t re­ally be­cause I hadn’t spent as much time as them spe­cial­iz­ing in some other small part of the AI safety field. (I’d give ex­am­ples ex­cept it’s been long enough that I can’t re­mem­ber the speci­fics.)

  • It let me con­nect with folks who I oth­er­wise would not have con­nected with be­cause they are less ac­tive on LW or not liv­ing in the Bay Area, and this has gen­er­ally proven fruit­ful to me over the years to know other folks in the space in a va­ri­ety of ways such as in­creased will­ing­ness to con­sider each oth­ers re­search and give each other the benefit of the doubt on new and weird ideas, ac­cess to peo­ple who are will­ing and ex­cited to bounce ideas around with you, and feel­ing con­nected to the com­mu­nity of AI safety re­searchers so this isn’t such a lonely pro­ject (this last one be­ing way more im­por­tant than I think many peo­ple rec­og­nize!).

  • It let me quickly get feed­back on ideas from mul­ti­ple peo­ple with differ­ent spe­cial­iza­tions and in­ter­ests that would have oth­er­wise been hard to get if I had to rely on them, say, in­ter­act­ing with my posts on LW or re­spond­ing to my emails.

  In the end though what most mo­ti­vates me to make such a strong claim is how much more valuable it was than I thought it would be. I ex­pected it to be a nice few days get­ting to work and think full time about a thing I care greatly about but, due to a va­ri­ety of life cir­cum­stances, find it hard to de­vote more than ~15 hours a week to, when av­er­aged out over many weeks. In­stead it turned out to be a cat­a­lyst for get­ting me to re­con­sider my re­search as­sump­tions, to re-ex­am­ine my plans, to help oth­ers and learn I had more to offer oth­ers than I thought, and to get me un­stuck on prob­lems I’ve been think­ing about for months with­out much mea­surable progress.

  In terms of op­por­tu­nity costs I would guess that even if you’re already spend­ing the ma­jor­ity of your time work­ing on AI safety and do­ing so in an in-per­son col­lab­o­ra­tive en­vi­ron­ment with other AI safety re­searchers, my guess is you still would find it valuable to at­tend an event like this maybe once a year to help break you out of lo­cal max­ima cre­ated by that bub­ble and re­con­sider your re­search pri­ori­ties by in­ter­act­ing with a broader range of folks in­ter­ested in AI safety.

• G Gordon Worley III 16 Oct 2019 2:12 UTC

  4 points

  on: The Gears of Impact

  The way you pre­sented AU here makes me think of it in terms of “at­tach­ment”, as in the way we tend to get at­tached to out­comes that haven’t hap­pened yet but that we ex­pect to and then can be sur­prised in good and bad ways when the out­comes are bet­ter or worse than we ex­pected. In this way im­pact seems tied in with our ca­pac­ity to ex­pect to see what we ex­pect to see (meta-ex­pec­ta­tions?), e.g. I 100% ex­pect a 40% chance of X and a 60% chance of Y hap­pen­ing. That 100% meta-ex­pec­ta­tion cre­ates a kind of at­tach­ment that doesn’t leave any room for be­ing wrong, and so just see­ing some­thing hap­pen in a way that makes you want to up­date your ob­ject level ex­pec­ta­tions of X and Y af­ter the fact seems to cre­ate a sense of im­pact.

• G Gordon Worley III 16 Oct 2019 2:04 UTC

  2 points

  on: The AI is the model

  The AI doesn’t have a model of what it should do, the AI is the model.

  This of course gen­er­al­izes to say that be­ings don’t have mod­els, they are the mod­els in the sense that the model ex­ists as a post hoc reifi­ca­tion of what the be­ing is.

  I think you are tack­ling an im­port prob­lem of in­tu­itions here. See­ing that the votes on this post sug­gest a mixed re­cep­tion (9 votes be­fore I voted with a to­tal score of 5), and given there are no other com­ments, I’m left to spec­u­late about what peo­ple might dis­like about this post.

  My guesses are:

  • gen­er­ally “boo­ing” things that veer too far into Con­ti­nen­tal philos­o­phy ter­ri­tory (via talk of in­ten­tion­al­ity)

  • dis­agree­ment that in­ten­tion­al­ity mat­ters in AI

  • dis­agree­ment with the em­bod­ied/​em­bed­ded ap­proach that breaks down neat abstractions

  • some other dis­like of how you wrote this

  I want to en­courage you to keep at it. I gen­er­ally agree with you from what you’ve writ­ten in this vein so far, and I con­tinue to sense a gen­eral ap­proval of ap­proaches to AI that treat the ab­strac­tions as ab­solute in ways that will dan­ger­ously break down due to them be­ing leaky, so I view writ­ing like yours here as an im­por­tant cor­rec­tive to chang­ing in­tu­itions such that they bet­ter re­flect the em­bed­ded­ness and in­ter­con­nect­ed­ness of agents.

• G Gordon Worley III 15 Oct 2019 1:37 UTC

  7 points

  on: Towards an em­piri­cal in­ves­ti­ga­tion of in­ner alignment

  I re­ally like the maze ex­am­ple. It’s good enough, thanks to your short ex­pla­na­tion, that when I saw the big maze I was gen­uinely un­sure of the in­tended be­hav­ior, i.e. should the thing find the green ar­row (that mark­ing the goal, and the exit is the proxy) or exit the maze (the ar­row is the proxy).

TAISU 2019 Field Report

• G Gordon Worley III 14 Oct 2019 17:41 UTC

  13 points

  in reply to: DanielFilan's comment on: DanielFilan’s Short­form Feed

  For com­par­i­son, in a uni­verse where EA ex­isted 30 years ago we would have thought it very im­por­tant to have many Rus­sia spe­cial­ists.

• G Gordon Worley III 14 Oct 2019 17:15 UTC

  2 points

  in reply to: abramdemski's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  Ah, I guess I don’t ex­pect it to end up ig­nor­ing the parts of the net­work that can’t learn be­cause I don’t think er­ror min­i­miza­tion, learn­ing, or any­thing else is a top level goal of the net­work. That is, there are only low-level con­trol sys­tems in­ter­act­ing, and parts of the net­work get not ig­nored by their be­ing more pow­er­ful in var­i­ous ways, prob­a­bly by be­ing po­si­tioned such that they are lo­cated in the net­work such that they have more in­fluence on be­hav­ior than other parts of the net­work that perform Bayesian learn­ing. This does mean I ex­pect those parts of the net­work don’t learn or learn in­effi­ciently, but they do that be­cause it’s adap­tive.

  For ex­am­ple, I would guess some­thing in hu­mans like the neo­cor­tex is ca­pa­ble of Bayesian learn­ing, but it only in­fluences the rest of the sys­tem through nar­row chan­nels that pre­vent it from “tak­ing over” and mak­ing hu­mans true pre­dic­tion er­ror min­i­miz­ers, in­stead forc­ing them to do things that satisfy other set points. In buzz words you might say hu­man minds are “com­plex, adap­tive, emer­gent sys­tems” built out of neu­rons with most of the func­tion com­ing bot­tom up from the neu­rons or “from the mid­dle”, if you will, in terms of net­work topol­ogy.

• G Gordon Worley III 14 Oct 2019 17:05 UTC

  2 points

  in reply to: Linda Linsefors's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  From my un­der­stand­ing I’m happy to talk just in terms of set points if that helps avoid con­fu­sion. Things like pre­dic­tions, goals, in­ten­tions, learn­ing, etc. seem to me like ways of talk­ing about con­trol sys­tems with set points and pos­si­ble set point up­date mechanisms that func­tion in par­tic­u­lar ways that we iden­tify with those things. My origi­nal use of “pre­dic­tion” seems to be con­fus­ing to many so I guess I should just stick to “set point” or make more clear what “pre­dic­tion” means here since I as­sume (al­though can’t re­mem­ber) that I picked up us­ing “pre­dic­tion” to mean “set point” from the rele­vant liter­a­ture.

• G Gordon Worley III 11 Oct 2019 20:22 UTC

  5 points

  on: When we sub­stan­tially mod­ify an old post should we edit di­rectly or post a ver­sion 2?

  I gen­er­ally think new posts are a good idea. The old post is what it was when it was writ­ten. If you no longer en­dorse it or think you can write a bet­ter ver­sion, do so and link it from the origi­nal. I think there is value in be­ing able to find and read old ver­sions of things as they were when they were writ­ten with­out the past be­ing ed­ited out of ex­is­tence, let­ting us see the trail of how an idea was shaped and formed and how the au­thor grew in their un­der­stand­ing and abil­ity to ex­press them­selves. Ad­di­tion­ally the com­ments on a post may no longer make sense if you sub­stan­tially edit it.

• G Gordon Worley III 11 Oct 2019 20:15 UTC

  4 points

  in reply to: Linda Linsefors's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  Yeah, this has been a re­ally good com­ment sec­tion for figur­ing out how my in­ter­nal mod­els are not as eas­ily con­veyed to oth­ers as I had hoped. I’ll likely write a fol­low up post try­ing to ex­plain this idea again with some re­vised lan­guage to make the point pos­si­bly clearer and lean more on speci­fics from ex­ist­ing re­search on these mod­els since there seem to be some in­fer­en­tial gaps that I have for­got­ten about such that what feels like the ex­cit­ing new part to me (pre­dic­tion er­ror sig­nal = valence = ground of value) is maybe the least in­ter­est­ing and least im­por­tant as­pect to eval­u­ate for oth­ers who lack the same be­liefs I have about how I think what I’m ges­tur­ing at with “pre­dic­tive cod­ing” and “min­i­miza­tion of pre­dic­tion er­ror” work.

• G Gordon Worley III 11 Oct 2019 20:10 UTC

  3 points

  in reply to: elriggs's comment on: “Mild Hal­lu­ci­na­tion” Test

  Yeah, that’s a de­cent ap­prox­i­ma­tion. Same effect at twilight. Although it’s miss­ing some re­ally im­por­tant as­pect of the ex­pe­rience that’s hard to ex­plain, like it’s just vi­su­ally similar and lacks the al­ive­ness that fills ev­ery­thing when I ex­pe­rience them as glow­ing or bright or self illu­mi­nat­ing.

• G Gordon Worley III 11 Oct 2019 16:50 UTC

  3 points

  in reply to: elriggs's comment on: “Mild Hal­lu­ci­na­tion” Test

  Prob­a­bly not? It’s a thing that started hap­pen­ing to me af­ter ~500 hours of med­i­ta­tion, and it hap­pens in any en­vi­ron­ment. There’s not even a thing to look for ex­actly be­cause it doesn’t feel like a nor­mal vi­sual hal­lu­ci­na­tion. Things look ba­si­cally the same as they did be­fore but it feels like they are bright. Maybe the clos­est thing would be some­thing like what hap­pens when your eyes are ar­tifi­cially di­lated.

• G Gordon Worley III 11 Oct 2019 16:43 UTC

  3 points

  in reply to: Jan Kulveit's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  It looks like I read your post but for­got about it. I’ll have to look at it again.

  I am build­ing this the­ory in a way that I think is highly com­pat­i­ble with Fris­ton, al­though I also don’t have a gears-level un­der­stand­ing of Fris­ton, so I find it eas­ier to think in terms of con­trol sys­tems which ap­pear to offer an equiv­a­lent model to me.

• G Gordon Worley III 11 Oct 2019 16:34 UTC

  4 points

  in reply to: abramdemski's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  I find speak­ing in terms of min­i­miza­tion of pre­dic­tion er­ror use­ful to my own in­tu­itions, but it does in­creas­ingly look like what I’m re­ally think­ing of are just generic home­o­static con­trol sys­tems. I like talk­ing in terms of pre­dic­tion er­ror be­cause I think it makes the trans­la­tion to other similar the­o­ries eas­ier (I’m think­ing other Bayesian brain the­o­ries and Fris­ton’s free en­ergy the­ory), but I think it’s right to think I’m just think­ing about a con­trol sys­tem send­ing sig­nals to hit a set point, even if some of those con­trol sys­tems do learn in a way that looks like Bayesian up­dat­ing or min­i­miza­tion of pre­dic­tion er­ror and oth­ers don’t.

  The sense in which I think of this the­ory as par­si­mo­nious is that I don’t be­lieve there is a sim­pler mechanism that can ex­plain what we see. If we could talk about these phe­nom­ena in terms of con­trol sys­tems with­out us­ing sig­nals about dis­tance from set points I’d pre­fer that, and I think the com­plex­ity we get from hav­ing to build things out of such sim­ple com­po­nents is the right move in terms of par­si­mony rather than hav­ing to pos­tu­late ad­di­tional mechanisms. As long as I can ex­plain things ad­e­quately with­out hav­ing to in­tro­duce more mov­ing parts I’ll con­sider it max­i­mally par­si­mo­nious as far as my cur­rent knowl­edge and needs go.

• G Gordon Worley III 11 Oct 2019 4:23 UTC

  2 points

  in reply to: abramdemski's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  So, similarly, we can’t pre­dict that you or I ver­bally re­port pos­i­tive/​nega­tive/​neu­tral at­tach­ing to per­cepts from the claim that the sen­sory hi­er­ar­chy is com­posed of units which are con­trol­lers. A con­trol­ler has valence in that it has goals and how-it’s-do­ing on those goals, but why should we ex­pect that hu­mans ver­bally re­port the di­rect ex­pe­rience of that? Hu­mans don’t have di­rect con­scious ex­pe­rience of ev­ery­thing go­ing on in neu­ral cir­cuitry.

  Yeah this is s good point and I agree it’s one of the things that I am look­ing for oth­ers to ver­ify with bet­ter brain imag­ing tech­nol­ogy. I find my­self in the po­si­tion of work­ing ahead of what we can com­pletely ver­ify now be­cause I’m will­ing to take the bet that it’s right or at least right enough that how­ever it’s wrong won’t throw out the work I do.

• G Gordon Worley III 11 Oct 2019 1:44 UTC

  2 points

  on: Gears vs Behavior

  Great and sim­ple ex­pla­na­tion of an im­por­tant topic. I just hope I can re­mem­ber to link this post of­ten so peo­ple can find it when they start to ques­tion whether or not they re­ally need to know about the gears of some­thing.

• G Gordon Worley III 10 Oct 2019 22:04 UTC

  2 points

  in reply to: abramdemski's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  We Need To Ex­plain Why Hu­mans Differ­en­ti­ate Goals and Beliefs, Not Just Why We Con­flate Them

  You men­tion that good/​bad seem like nat­u­ral cat­e­gories. I agree that peo­ple of­ten seem to mix up “should” and “prob­a­bly is”, “good” and “nor­mal”, “bad” and “weird”, etc. Th­ese ob­ser­va­tions in them­selves speak in fa­vor of the min­i­mize-pre­dic­tion-er­ror the­ory of val­ues.

  How­ever, we also differ­en­ti­ate these con­cepts at other times. Why is that? Is it some kind of mis­take? Or is the con­fla­tion of the two the mis­take?

  I think the mix-up be­tween the two is partly ex­plained by the effect I men­tioned ear­lier: com­mon prac­tice is op­ti­mized to be good, so there will be a ten­dency for com­mon­al­ity and good­ness to cor­re­late. So, it’s sen­si­ble to cluster them to­gether men­tally, which can re­sult in them get­ting con­fused. There’s likely an­other as­pect as well, which has some­thing to do with so­cial en­force­ment (ie, peo­ple are strate­gi­cally con­flat­ing the two some of the time?) -- but I’m not sure ex­actly how that works.

  This seems like an im­por­tant ques­tion: if all these phe­nom­ena re­ally are ul­ti­mately the same thing and pow­ered by the same mechanisms, why do we make dis­tinc­tions be­tween them and find those dis­tinc­tions use­ful?

  I don’t have an an­swer I’m satis­fied with, but I’ll try to say a few words about what I’m think­ing and see if that moves us along.

  My first ap­prox­i­ma­tion would be that we’re look­ing at things that we ex­pe­rience by differ­ent means and so give them differ­ent names be­cause when we ob­serve them they pre­sent in differ­ent ways. Goals (I as­sume by this you mean the cluster of things we might call de­sires, aver­sions, and gen­er­ally in­ten­tion to­wards ac­tion) prob­a­bly tend to be ob­served by notic­ing the gen­er­a­tion of sig­nals go­ing out that usu­ally gen­er­ate ob­serv­able ac­tions (move­ment, speech, etc.) whereas be­liefs (the cluster of things that in­cludes thoughts and maybe emo­tions) are in­ter­nal and not send­ing out sig­nals to ac­tion be­yond men­tal ac­tion.

  I don’t know enough to be very con­fi­dent in that, though, and think like you that it could be due to nu­mer­ous rea­sons why it might make sense to think of them as sep­a­rate even if they are fun­da­men­tally not very differ­ent.

• G Gordon Worley III 10 Oct 2019 21:42 UTC

  2 points

  in reply to: abramdemski's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  It’s Easy to Over­es­ti­mate The De­gree to which Agents Min­i­mize Pre­dic­tion Error

  I of­ten en­joy va­ri­ety—in food, tele­vi­sion, etc—and ob­serve other hu­mans do­ing so. Naively, it seems like hu­mans some­times pre­fer pre­dictabil­ity and some­times pre­fer va­ri­ety.

  How­ever: any learn­ing agent, al­most no mat­ter its val­ues, will tend to look like it is seek­ing pre­dictabil­ity once it has learned its en­vi­ron­ment well. It is tak­ing ac­tions it has taken be­fore, and steer­ing to­ward the en­vi­ron­men­tal states similar to what it always steers for. So, one could un­der­stand­ably reach the con­clu­sion that it is re­li­a­bil­ity it­self which the agent likes.

  In other words: if I seem to eat the same foods quite of­ten (de­spite claiming to like va­ri­ety), you might con­clude that I like fa­mil­iar­ity when it’s ac­tu­ally just that I like what I like. I’ve found a set of foods which I par­tic­u­larly en­joy (which I can ro­tate be­tween for the sake of va­ri­ety). That doesn’t mean it is fa­mil­iar­ity it­self which I en­joy.

  I’m not deny­ing that mere fa­mil­iar­ity has some pos­i­tive valence for hu­mans; I’m just say­ing that for ar­bi­trary agents, it seems easy to over-es­ti­mate the im­por­tance of fa­mil­iar­ity in their val­ues, so we should be a bit sus­pi­cious about it for hu­mans too. And I’m say­ing that it seems like hu­mans en­joy sur­prises some­times, and there’s evolu­tion­ary/​ma­chine-learn­ing rea­son­ing to ex­plain why this might be the case.

  I’ve replied about sur­prise, its benefits, and its mechanism a cou­ple times now. My the­ory is that sur­prise is by it­self bad but can be made good by hav­ing con­trol sys­tems that ex­pect sur­prise and send a good sig­nal when sur­prise is seen. Depend­ing on how this gets weighted, this cre­ates a net pos­i­tive mixed emo­tion where sur­prise is ex­pe­rienced as some­thing good and serves many use­ful pur­poses.

  I think this mostly dis­solves the other points you bring up that I read as con­tin­gent on think­ing the the­ory doesn’t pre­dict hu­mans would find va­ri­ety and sur­prise good in some cir­cum­stances, but if not please let me know what the re­main­ing con­cerns are in light of this ex­pla­na­tion (or pos­si­bly ob­ject to my ex­pla­na­tion of why we ex­pect sur­prise to some­times be net good).

• G Gordon Worley III 10 Oct 2019 21:37 UTC

  2 points

  in reply to: abramdemski's comment on: Min­i­miza­tion of pre­dic­tion er­ror as a foun­da­tion for hu­man val­ues in AI alignment

  Evolved Agents Prob­a­bly Don’t Min­i­mize Pre­dic­tion Error

  If we look at the field of re­in­force­ment learn­ing, it ap­pears to be gen­er­ally use­ful to add in­trin­sic mo­ti­va­tion for ex­plo­ra­tion to an agent. This is the ex­act op­po­site of pre­dictabil­ity: in one case we add re­ward for en­ter­ing un­pre­dictable states, whereas in the other case we add re­ward for en­ter­ing pre­dictable states. I’ve seen peo­ple try to defend min­i­miz­ing pre­dic­tion er­ror by show­ing that the agent is still mo­ti­vated to learn (in or­der to figure out how to avoid un­pre­dictabil­ity). How­ever, the fact re­mains: it is still mo­ti­vated to learn strictly less than an un­pre­dictabil­ity-lov­ing agent. RL has, in prac­tice, found it use­ful to add re­ward for un­pre­dictabil­ity; this sug­gests that evolu­tion might have done the same, and sug­gests that it would not have done the ex­act op­po­site. Agents op­er­at­ing un­der a pre­dic­tion-er­ror penalty would likely un­der-ex­plore.

  I ended up re­ply­ing to this in a sep­a­rate post since I felt like similar ob­jec­tions kept com­ing up. My short an­swer is: min­i­miza­tion of pre­dic­tion er­ror is min­i­miza­tion of er­ror at pre­dict­ing in­put to a con­trol sys­tem that may not be ar­bi­trar­ily free to change its pre­dic­tion set point. This means that it won’t always be the case that a con­trol sys­tem is globally try­ing to min­i­mize pre­dic­tion er­ror, but in­stead is lo­cally try­ing to min­i­mize pre­dic­tion er­ror, al­though it may not be able to be­come less wrong over time be­cause it can’t change the pre­dic­tion to bet­ter pre­dict the in­put.

  From an evolu­tion­ary per­spec­tive my guess is that true Bayesian up­dat­ing is a fairly re­cent adap­ta­tion, and most min­i­miza­tion of pre­dic­tion er­ror is min­i­miza­tion of er­ror of mostly fixed pre­dic­tion set points that are benefi­cial for sur­vival.