Charlie Steiner

• Charlie Steiner 5 Dec 2019 20:27 UTC

  LW: 2 AF: 1

  AF

  in reply to: capybaralet's comment on: What I talk about when I talk about AI x-risk: 3 core claims I want ma­chine learn­ing re­searchers to ad­dress.

  Well, you men­tioned that a lot of peo­ple were get­ting off the train at point 1. My com­ment can be thought of as giv­ing a much more thor­oughly in­side-view look at point 1, and de­riv­ing other stuff as in­ci­den­tal con­se­quences.

  I’m men­tally work­ing with an anal­ogy to teach­ing peo­ple a new con­tra dance (if you don’t know what con­tra danc­ing is, I’m just talk­ing about some se­quence of dance moves). The teacher of­ten has an ab­stract view of ex­pres­sion and flow that the stu­dents lack, and there’s a temp­ta­tion for the teacher to try to share that view with the stu­dents. But the stu­dents don’t want to ab­strac­tions, what they want is con­crete steps to fol­low, and good dancers will dance the dance just fine with­out ever hear­ing about the teacher’s ab­stract view. Be­fore danc­ing they re­gard the ab­strac­tions as difficult to un­der­stand and dis­tract­ing from the con­crete in­struc­tions; they’ll be much more equipped to un­der­stand and ap­pre­ci­ate them *af­ter* danc­ing the dance.

• Charlie Steiner 3 Dec 2019 17:43 UTC

  LW: 5 AF: 2

  AF

  on: What I talk about when I talk about AI x-risk: 3 core claims I want ma­chine learn­ing re­searchers to ad­dress.

  Huh, I won­der what you think of a differ­ent way of split­ting it up. Some­thing like:

  • It’s a sci­en­tific pos­si­bil­ity to have AI that’s on av­er­age bet­ter than hu­man­ity at the class of tasks “choose ac­tions that achieve a goal in the real world.” Let’s la­bel this by some su­per­la­tive jar­gon like “su­per­in­tel­li­gent AI.” Such a tech­nol­ogy would be hugely im­pact­ful.

  • It would be re­ally bad if a su­per­in­tel­li­gent AI was choos­ing ac­tions to achieve some goal, but this goal wasn’t benefi­cial to hu­mans. There are sev­eral open prob­lems that this means we need to solve be­fore safely turn­ing on any such AI.

  • We know enough that we can do use­ful work on (most of) these open prob­lems right now. Ar­gu­ing for this also im­plies that su­per­in­tel­li­gent AI is close enough (if not in years, then in “num­ber of paradigm shifts”) that this work needs to start get­ting done.

  • We would ex­pect a pri­ori that work on these open prob­lems of benefi­cial goal de­sign should be un­der-pri­ori­tized (pub­lic goods prob­lem, low im­me­di­ate profit, not ob­vi­ous you need it be­fore you re­ally need it). And in­deed that seems to be the case (in­sert NIPS sur­vey here), though there’s work go­ing on at non­prof­its that have differ­ent in­cen­tives. So con­sider think­ing about this area if you’re look­ing for things to re­search.

• Charlie Steiner 21 Nov 2019 8:03 UTC

  4 points

  on: [1911.08265] Mas­ter­ing Atari, Go, Chess and Shogi by Plan­ning with a Learned Model | Arxiv

  Welp, we’re doomed (/​s), as soon as some­one figures out how to get 100 mil­lion tries at tak­ing over the world so we can crush the world-tak­ing-over prob­lem with stochas­tic gra­di­ent de­scent.

• Charlie Steiner 21 Nov 2019 7:53 UTC

  3 points

  on: A fun cal­ibra­tion game: “0-hit Google phrases”

  Here’s some: “an­tipacek”, “pro­gres­sive kil­ler fat”, “hut re­fusal guideline”, “south-stream res­ig­na­tion”, “pam­plem”, “con­science bw”, “fog log dog bog”, “layer iron trol­ley”, “pre­vent pub­li­ca­tion fre­quency”, “scon­spir­a­cyn”.

• Charlie Steiner 20 Nov 2019 18:35 UTC

  LW: 2 AF: 1

  AF

  in reply to: John_Maxwell's comment on: The Good­hart Game

  Pretty sure you un­der­stood it :) But yeah, not only would I like to be able to com­pare two things, I’d like to be able to find the op­ti­mum val­ues of some con­tin­u­ous vari­ables. Though I sup­pose it doesn’t mat­ter as much if you’re try­ing to check /​ eval­u­ate ideas that you ar­rived at by more ab­stract rea­son­ing.

• Charlie Steiner 20 Nov 2019 5:42 UTC

  2 points

  in reply to: G Gordon Worley III's comment on: Cy­ber­netic dreams: Beer’s pond brain

  I’m also look­ing for­ward to up­com­ing posts, but all these ex­am­ples so far sound to me like a mod­ernist’s sub­sti­tute for sym­pa­thetic magic :P

• Charlie Steiner 20 Nov 2019 5:30 UTC

  4 points

  on: Draw­ing on Walls

  Sounds like a sales pitch for white­board wal­l­pa­per :)

• Charlie Steiner 20 Nov 2019 2:32 UTC

  LW: 2 AF: 1

  AF

  on: The Good­hart Game

  The im­prac­ti­cal part about train­ing for good be­hav­ior is that it’s a nested loop—ev­ery train­ing ex­am­ple on how to find good max­ima re­quires train­ing a model that in turn needs its own train­ing ex­am­ples. So it’s des­tined to be be­hind the state of the art, prob­a­bly us­ing state of the art mod­els to gen­er­ate the co­pi­ous re­quired train­ing data.

  The ques­tion, I sup­pose, is whether this is still good enough to learn use­ful gen­eral les­sons. And af­ter think­ing about it, I think the an­swer is that yes, it should be, es­pe­cially for feed-for­ward ar­chi­tec­tures that look like mod­ern ma­chine learn­ing, where you don’t ex­pect qual­i­ta­tive changes in ca­pa­bil­ity as you scale com­pu­ta­tional re­sources.

• Charlie Steiner 19 Nov 2019 4:46 UTC

  3 points

  in reply to: Mary Chernyshenko's comment on: Im­pos­si­ble moral prob­lems and moral authority

  Yes, I hope that my fram­ing of the prob­lem sup­ports this sort of con­clu­sion :P

  An al­ter­nate fram­ing where it still seems im­por­tant would be “moral un­cer­tainty”. Where when we don’t know what to do, it’s be­cause we are lack­ing some facts, maybe even key facts. So I’m sort of sneak­ily ar­gu­ing against that frame.

• Charlie Steiner 18 Nov 2019 10:40 UTC

  4 points

  on: The Power to Draw Better

  Any se­quence that in­volves recom­mend­ing peo­ple work through Draw­ing on the Right Side of the Brain is a se­quence I should read :P

Im­pos­si­ble moral prob­lems and moral authority

• Charlie Steiner 17 Nov 2019 22:24 UTC

  2 points

  in reply to: capybaralet's comment on: Can in­differ­ence meth­ods re­deem per­son-af­fect­ing views?

  You mean, why I ex­pect a per­son-af­fect­ing util­ity func­tion to be differ­ent if eval­u­ated to­day v. to­mor­row?

  Well, sup­pose that to­day I con­sider the ac­tion of cre­at­ing a per­son, and am in­differ­ent to cre­at­ing them. Since this is true for all sorts of peo­ple, I am in­differ­ent to cre­at­ing them one way vs. an­other (e.g. happy vs sad). If they are to be cre­ated in­side my guest bed­room, this means I am in­differ­ent be­tween cer­tain ways the atoms in my guest bed­room could be ar­ranged. Then if this per­son gets cre­ated tonight and is around to­mor­row, I’m no longer in­differ­ent be­tween the ar­range­ment that is them sad and the ar­range­ment that is them happy.

  Yes, you could always re­verse-en­g­ineer a util­ity func­tion over world-his­to­ries that en­com­passes both of these. But this doesn’t nec­es­sar­ily solve the prob­lems that come to mind when I say “change in util­ity func­tions”—for ex­am­ple, I might take bets about the fu­ture that ap­pear lose/​lose when I have to pay them off, or take ac­tions that mod­ify my own ca­pa­bil­ities in ways I later re­gret.

  I dunno—were you think­ing of some spe­cific ap­pli­ca­tion of in­differ­ence that could sidestep some of these prob­lems?

• Charlie Steiner 12 Nov 2019 18:03 UTC

  2 points

  on: Can in­differ­ence meth­ods re­deem per­son-af­fect­ing views?

  Hilary Greaves sounds like a re­ally in­ter­est­ing per­son :)

  So, you could use these meth­ods to con­struct a util­ity func­tion cor­re­spond­ing to the per­son-af­fect­ing view­point from your cur­rent world, but this wouldn’t pro­tect this util­ity func­tion from cri­tique. She brings up the Pareto prin­ci­ple, where this per­son-af­fect­ing util­ity func­tion would be in­differ­ent to some things that were strict im­prove­ments, which seems un­de­sir­able.

  I think the more fun­da­men­tal prob­lem there is in­tran­si­tivity. You might be able to define a util­ity func­tion that cap­tures the per­son-af­fect­ing view to you, but a copy of you one day later (or one world over) would say “hang on, I didn’t agree to that.” They’d make their own util­ity func­tion with pri­ori­ties on differ­ent peo­ple. And so you end up fight­ing with your­self, un­til one of you can self-mod­ify to ac­tu­ally give up the per­son-af­fect­ing view, and just keep this util­ity func­tion cre­ated by their past self.

  A more re­flec­tive self might try to do some­thing clever like bar­gain­ing be­tween all selves they ex­pect to plau­si­bly be (and who will fol­low the same rea­son­ing), and tak­ing ac­tions that benefit those selves, con­fi­dent that their other selves will keep their end of the bar­gain.

  My gen­eral feel­ing about pop­u­la­tion ethics, though, is that it’s aes­thet­ics. This was a re­ally im­por­tant re­al­iza­tion for me, and I think most peo­ple who think about pop­u­la­tion ethics don’t think about the prob­lem the right way. Peo­ple don’t in­her­ently have util­ity, util­ity isn’t a fluid stored in the gall blad­der, it’s some­thing eval­u­ated by a de­ci­sion-maker when they think about pos­si­ble ways for the world to be. This means it’s okay to have a preferred stan­dard of liv­ing for fu­ture peo­ple, to have non­lin­ear terms on pop­u­la­tion and “self­ish” util­ity, etc.

• Charlie Steiner 12 Nov 2019 16:08 UTC

  1 point

  on: An op­ti­mal stop­ping paradox

  If the growth is ex­po­nen­tial, I still don’t think there’s a para­dox—sure, you’re in­cen­tivized to wait for­ever, but I’m already in­cen­tivized to wait for­ever with my real life in­vest­ments. The only thing that stops me from real life in­vest­ing my money for­ever is that some­times I have things (not in­cluded in the toy prob­lem) that I re­ally want to buy with that money.

• Charlie Steiner 11 Nov 2019 17:42 UTC

  3 points

  in reply to: Pattern's comment on: What are hu­man val­ues? - Thoughts and challenges

  So, the dic­tio­nary defi­ni­tion (SEP) would be some­thing like “ob­jec­tively good/​par­si­mo­nious/​effec­tive ways of carv­ing up re­al­ity.”

  There’s also the im­pli­ca­tion that when we use kinds in rea­son­ing, things of the same kind should share most or all im­por­tant prop­er­ties for the task at hand. There’s also sort of the im­pli­ca­tion that hu­mans naively think of the world as made out of nat­u­ral kinds on an on­tolog­i­cally ba­sic level.

  I’m say­ing that even if peo­ple don’t be­lieve in dis­em­bod­ied souls, when they ask “what do I want?” they think they’re get­ting an an­swer back that is ob­jec­tively a good/​par­si­mo­nious/​effec­tive way of talk­ing. That there is some thing, not nec­es­sar­ily a soul but at least a pat­tern, that is be­ing ac­cessed by differ­ent ways of ask­ing “what do I want?”, which can’t give us in­con­sis­tent an­swers be­cause it’s all one thing.

• Charlie Steiner 11 Nov 2019 15:43 UTC

  2 points

  in reply to: Owain_Evans's comment on: Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

  Thanks for the re­ply :)

  Sure, you can get the AI to draw polka-dots by tar­get­ing a fea­ture that likes polka dots, or a Mon­drian by tar­get­ing some fea­tures that like cer­tain ge­ome­tries and col­ors, but now you’re not us­ing style trans­fer at all—the image is the style. More­over, it would be pretty hard to use this to get a Kand­in­sky, be­cause the AI that makes style-paint­ings has no stan­dard by which it would choose things to draw that could be ob­jects but aren’t. You’d need a third and sep­a­rate scheme to make Kand­in­skys, and then I’d just bring up an­other artist not cov­ered yet.

  If you’re not try­ing to cap­ture all hu­man vi­sual art in one model, then this is no big­gie. So now you’re prob­a­bly go­ing “this is fine, why is he go­ing on about this.” So I’ll stop.

  Do you have ex­am­ples in mind when you men­tion “hu­man ex­pe­rience” and “em­bod­i­ment” and “limited agents”

  For “hu­man ex­pe­rience,” yeah, I just means some­thing like com­mu­nica­tive/​evoca­tive con­tent that re­lies on a the­ory of mind to use for com­mu­ni­ca­tion. Maybe you could train an AI on pa­tri­otic paint­ings and then it could pro­duce pa­tri­otic paint­ings, but I think only by work­ing on the­ory of mind would an AI think to pro­duce a pa­tri­otic paint­ing with­out hav­ing seen one be­fore. I’m also re­minded of Karpa­thy’s ex­am­ple of Obama with his foot on the scale.

  For em­bod­i­ment, this means art that blurs the line be­tween vi­sual and phys­i­cal. I was think­ing of how some things aren’t art if they’re nor­mal sized, but if you make them re­ally big, then they’re art. Since all hu­man art is phys­i­cal art, this line can be avoided mostly but not com­pletely.

  For “limited,” I imag­ined some­thing like Den­nett’s ex­am­ple of the peo­ple on the bridge. The artist only has to paint lit­tle blobs, be­cause they know how hu­mans will in­ter­pret them. Com­pared to the ex­am­ple above of us­ing un­der­stand­ing of hu­mans to choose con­tent, this ex­am­ple uses an un­der­stand­ing of hu­mans to choose style.

  Yet even with zero prior train­ing on vi­sual art they can make pretty im­pres­sive images by hu­man lights. I think this was sur­pris­ing to most peo­ple both in and out­side deep learn­ing. I’m cu­ri­ous whether this was sur­pris­ing to you.

  It was im­pres­sive, but I re­mem­ber the old 2015 post the Chris Olah co-au­thored. First off, if you look at the pic­tures, they’re less pretty than the pic­tures that came later. And I re­mem­ber one key sen­tence: “By it­self, that doesn’t work very well, but it does if we im­pose a prior con­straint that the image should have similar statis­tics to nat­u­ral images, such as neigh­bor­ing pix­els need­ing to be cor­re­lated.” My im­pres­sion is that Deep­Dream et al. have been trained to make vi­sual art—by hy­per­pa­ram­e­ter tun­ing (grad stu­dent de­scent).

• Charlie Steiner 10 Nov 2019 17:43 UTC

  2 points

  on: Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

  I like this ex­po­si­tion, but I’m still skep­ti­cal about the idea.

  Since “art” is a hu­man con­cept, it’s nat­u­rally a grab bag of lots of differ­ent mean­ings. It’s plau­si­ble that for some mean­ings of “art,” hu­mans do some­thing similar to search­ing through a space of pa­ram­e­ters for some­thing that strongly ac­ti­vates some tar­get con­cept within the con­straints of a style. But there’s also a lot about art that’s not like that.

  Like art that’s non-rep­re­sen­ta­tional, or oth­er­wise de­nies the sep­a­ra­tion be­tween form and con­tent. Or art that’s heav­ily lin­guis­tic, or so­cial, or re­lies on some sort of think­ing on the part of the au­di­ence. Art that’s very differ­ent for the performer and the au­di­ence, so that it doesn’t make sense to talk about a search pro­cess op­ti­miz­ing for the au­di­ence’s ex­pe­rience, or oth­er­wise doesn’t have a search pro­cess as a par­tic­u­larly sim­ple ex­pla­na­tion. Art that’s so rooted in emo­tion or hu­man ex­pe­rience that we wouldn’t con­sider an ac­count of it com­plete with­out talk­ing about the hu­man ex­pe­rience. Art that only makes sense when con­sid­er­ing hu­mans as em­bod­ied, limited agents.

  So if I con­sider the state­ment “the Deep­Dream al­gorithm is do­ing art,” there is a sense in which this is rea­son­able. But I don’t think that ex­tends to call­ing what Deep­Dream does a model for what hu­mans do when we think about or cre­ate art. We do some­thing not merely more com­pli­cated in the de­tails, but more com­pli­cated in its macros-struc­ture, and hooked into many of the com­pli­ca­tions of hu­man psy­chol­ogy.

• Charlie Steiner 9 Nov 2019 18:42 UTC

  LW: 2 AF: 1

  AF

  in reply to: orthonormal's comment on: The Credit As­sign­ment Problem

  Dropout is like the con­verse of this—you use dropout to as­sess the non-out­dropped el­e­ments. This pro­motes re­siliency to per­tur­ba­tions in the model—whereas if you eval­u­ate things by how bad it is to break them, you could pro­mote frag­ile, in­ter­re­li­ant col­lec­tions of el­e­ments over re­silient el­e­ments.

  I think the root of the is­sue is that this Shap­ley value doesn’t dis­t­in­guish be­tween some­thing be­ing bad to break, and some­thing be­ing good to have more of. If you re­moved all my blood I would die, but that doesn’t mean that I would cur­rently benefit from ad­di­tional blood.

  Any­how, the joke was that as soon as you add a con­tin­u­ous pa­ram­e­ter, you get gra­di­ent de­scent back again.

• Charlie Steiner 9 Nov 2019 14:57 UTC

  2 points

  in reply to: sil ver's comment on: Open & Wel­come Thread—Novem­ber 2019

  0.3 mg mela­tonin an hour be­fore I want to be asleep works, my only trou­ble is ac­tu­ally plan­ning in ad­vance.

• Charlie Steiner 9 Nov 2019 12:39 UTC

  LW: 4 AF: 2

  AF

  on: The Credit As­sign­ment Problem

  You look at the world, and you say: “how can I max­i­mize util­ity?” You look at your be­liefs, and you say: “how can I max­i­mize ac­cu­racy?” That’s not a con­se­quen­tial­ist agent; that’s two differ­ent con­se­quen­tial­ist agents!

  Not… re­ally? “how can I max­i­mize ac­cu­racy?” is a very liberal agen­tifi­ca­tion of a pro­cess that might be more drily thought of as ask­ing “what is ac­cu­rate?” Your stan­dard se­quence pre­dic­tor isn’t search­ing through epistemic pseudo-ac­tions to find which ones best max­i­mize its ex­pected ac­cu­racy, it’s just fol­low­ing a pre-made plan of epistemic ac­tion that hap­pens to in­crease ac­cu­racy.

  Though this does lead to the thought: if you want to put things on equal foot­ing, does this mean you want to de­scribe a rea­soner that searches through epistemic steps/​rules like an agent search­ing through ac­tions/​plans?

  This is more or less how hu­mans already con­ceive of difficult ab­stract rea­son­ing. We don’t solve in­te­grals by gra­di­ent de­scent, we imag­ine do­ing some sort of tree search where the edges are differ­ent ab­stract ma­nipu­la­tions of the in­te­gral. But for ev­ery­day rea­son­ing, like nav­i­gat­ing 3D space, we just use our spe­cial­ized feed-for­ward hard­ware.