Charlie Steiner(Charlie Steiner)

• Charlie Steiner 3 Apr 2020 17:40 UTC

  8 points

  in reply to: maximkazhenkov's comment on: Im­pli­ca­tions of the Dooms­day Ar­gu­ment for x-risk reduction

  To be­lieve that you’re a one in a mil­lion case (e.g. in the first or last mil­lionth of all hu­mans), you need 20 bits of in­for­ma­tion (be­cause 2^20 is about 1000000).

  So on the one hand, 20 bits can be hard to get if the topic is hard to get re­li­able in­for­ma­tion about. But we reg­u­larly get more than 20 bits of in­for­ma­tion about all sorts of ques­tions (read­ing this com­ment has prob­a­bly given you more than 20 bits of in­for­ma­tion). So how hard this should “feel” de­pends heav­ily on how well we can trans­late our ob­ser­va­tional data into in­for­ma­tion about the fu­ture of hu­man­ity.

  Ex­tra note: In the case that there are an in­finite num­ber of hu­mans, this uniform prior ac­tu­ally breaks down (or else naively you’d think you have a 0.0% chance of be­ing any­one at all), so there can be a finite con­tri­bu­tion from the pos­si­bil­ity that there are in­finite peo­ple.

• Charlie Steiner 3 Apr 2020 4:45 UTC

  7 points

  on: Im­pli­ca­tions of the Dooms­day Ar­gu­ment for x-risk reduction

  Peo­ple are bad at in­ter­pret­ing the Dooms­day Ar­gu­ment, be­cause peo­ple are bad at deal­ing with ev­i­dence as Bayesian ev­i­dence, rather than a di­rect state­ment of the cor­rect be­lief.

  The Dooms­day Ar­gu­ment is ev­i­dence that we should up­date on. But it is not a di­rect state­ment of the cor­rect be­lief.

  A parable:

  On a par­allel earth, hu­man­ity is on the de­cline. Some dis­aster has struck, and the once-billions of proud hu­man­ity have been re­duced to a few scat­tered thou­sands. Now the last ex­iles of civ­i­liza­tion hide in sealed habitats that they no longer have the sup­ply chains to re­pair, and they know that soon enough the end will come for them too. But on the other hand, the philoso­phers among them re­mark, at least there’s the Dooms­day Ar­gu­ment, which says that on av­er­age we should ex­pect to be in the mid­dle of hu­man­ity. So if the DA is right, the cur­rent crisis is merely a bot­tle­neck in the mid­dle of hu­man­ity’s time, and ev­ery­thing will prob­a­bly work it­self out any day now. The last philoso­pher dies af­ter breath­ing in con­tam­i­nated air, with the last words “No! The po­si­tion I oc­cupy is… very un­likely!”

  Mo­ral:

  Your eyes and ears also provide you ev­i­dence about the ex­pected span of hu­man­ity.

• Charlie Steiner 1 Apr 2020 6:31 UTC

  2 points

  on: FLI Pod­cast: The Precipice: Ex­is­ten­tial Risk and the Fu­ture of Hu­man­ity with Toby Ord

  Fun pod­cast. The anal­ogy to hu­man plan­ning hori­zons was a very thought-pro­vok­ing one. Though ob­vi­ously, there are forces that ex­plain the way things are; com­pe­ti­tion be­tween differ­ent in­ter­ests is a strong se­lec­tion pres­sure for short-ter­mism.

Meta-prefer­ences two ways: gen­er­a­tor vs. patch

• Charlie Steiner 25 Mar 2020 20:28 UTC

  4 points

  on: [AN #92]: Learn­ing good rep­re­sen­ta­tions with con­trastive pre­dic­tive coding

  Is SIDLE not also a perfectly fine word? I don’t know how this went through peer re­view.

  Any­how, good newslet­ter this week, thanks :)

• Charlie Steiner 25 Mar 2020 18:35 UTC

  LW: 4 AF: 2

  AF

  on: De­con­fus­ing Hu­man Values Re­search Agenda v1

  I al­most agree, but still ended up dis­agree­ing with a lot of your bul­let points. Since read­ing your list was use­ful, I figured it would be worth­while to just make a par­allel list. ✓ for agree­ment, × for dis­agree­ment (• for neu­tral).

  Prob­lem overview

  ✓ I think we’re con­fused about what we re­ally mean when we talk about hu­man val­ues.

  × But our real prob­lem is on the meta-level: we want to un­der­stand value learn­ing so that we can build an AI that learns hu­man val­ues even with­out start­ing with a pre­cise model wait­ing to be filled in.

  _× We can trust AI to dis­cover that struc­ture for us even though we couldn’t ver­ify the re­sult, be­cause the point isn’t get­ting the right an­swer, it’s hav­ing a trust­wor­thy pro­cess.

  _ × We can’t just write down the cor­rect struc­ture any more than we can just write down the cor­rect con­tent. We’re try­ing to trans­late a vague hu­man con­cept into pre­cise in­struc­tions for an AI

  ✓ Agree with ex­ten­sional defi­ni­tion of val­ues, and rele­vance to de­ci­sion-mak­ing.

  • Re­search on the con­tent of hu­man val­ues may be use­ful in­for­ma­tion about what hu­mans con­sider to be hu­man val­ues. I think re­search on the struc­ture of hu­man val­ues is in much the same boat—in­for­ma­tion, not the fi­nal say.

  ✓ Agree about Stu­art’s work be­ing where you’d go to write down a pre­cise set of prefer­ences based on hu­man prefer­ences, and that the prob­lems you men­tion are prob­lems.

  Solu­tion overview

  ✓ Agree with as­sump­tions.

  • I think the ba­sic model leaves out the fact that we’re chang­ing lev­els of de­scrip­tion.

  _ × Merely caus­ing events (in the phys­i­cal level of de­scrip­tion) is not suffi­cient to say we’re act­ing (in the agent level of de­scrip­tion). We need some no­tion of “could have done some­thing else,” which is an ab­strac­tion about agents, not some­thing fun­da­men­tally phys­i­cal.

  _ × Similar quib­bles ap­ply to the other parts—there is no phys­i­cally spe­cial de­ci­sion pro­cess, we can only find one by chang­ing our level of de­scrip­tion of the world to one where we posit such a struc­ture.

  _ × The point: Every­thing in the ba­sic model is a statis­ti­cal reg­u­lar­ity we can ob­serve over the be­hav­ior of a phys­i­cal sys­tem. You need a bit more nu­anced way to place prefer­ences and meta-prefer­ences.

  _ • The sim­ple patch is to just say that there’s some level of de­scrip­tion where the de­ci­sion-gen­er­a­tion pro­cess lives, and prefer­ences live at a higher level of ab­strac­tion than that. There­fore prefer­ences are emer­gent phe­nom­ena from the level of de­scrip­tion the de­ci­sion-gen­er­a­tion pro­cess is on.

  _ _ × But I think if one ap­plies this patch, then it’s a big mis­take to use loaded words like “val­ues” to de­scribe the in­puts (all in­puts?) to the de­ci­sion-gen­er­a­tion pro­cess, which are, af­ter all, at a level of de­scrip­tion be­low the level where we can talk about prefer­ences. I think this con­flicts with the ex­ten­sive defi­ni­tions from ear­lier.

  × If we rec­og­nize that we’re talk­ing about differ­ent lev­els of de­scrip­tion, then prefer­ences are not ei­ther causally af­ter or causally be­fore de­ci­sions-on-the-ba­sic-model-level-of-ab­strac­tion. They’re reg­u­lar pat­terns that we can use to model de­ci­sions at a slightly higher level of ab­strac­tion.

  _ • How to de­scribe self-aware agents at a low level of ab­strac­tion then? Well, time to put on our GEB hats. The low level of ab­strac­tion just has to in­clude a com­pu­ta­tion of the model we would use on the higher level of ab­strac­tion.

  ✓ De­spite all these dis­agree­ments, I think you’ve made a pretty good case that the hu­man brain plau­si­bly com­putes a sin­gle cur­rency (valence) that it uses to rate both most de­ci­sions and most pre­dic­tions.

  _ × But I still don’t agree that this makes valence hu­man val­ues. I mean val­ues in the sense of “the cluster we some­times also point at with words like value, prefer­ence, af­finity, taste, aes­thetic, in­ten­tion, and ax­iol­ogy.” So I don’t think we’re left with a neu­ro­science prob­lem, I still think what we want the AI to learn is on that higher level of ab­strac­tion where prefer­ences live.

  What links here?

  • What is the sub­jec­tive ex­pe­rience of free will for agents? by G Gordon Worley III (2 Apr 2020 15:53 UTC; 10 points)

• Charlie Steiner 23 Mar 2020 23:49 UTC

  4 points

  on: Tag­ging (Click Gear Icon to filter Coron­avirus con­tent)

  Yay (!)

• Charlie Steiner 14 Mar 2020 7:55 UTC

  2 points

  in reply to: John_Maxwell's comment on: Quadratic mod­els and (un)falsified data

  It’s just a mea­sure of how close the data is to the line—like the “in­side view” un­cer­tainty that the model has about the data. In fact, that’s more pre­cisely what it is if this is the chi squared statis­tic (or square root thereof) that you min­i­mized to fit the model. And it’s in nice con­ve­nient units that you can com­pare to other things.

  It’s not quite right, be­cause it uses an im­plicit prior about noise and mod­els that doesn’t match your ac­tual state of in­for­ma­tion. But it’s some­thing that some­one who’s cur­rently re­port­ing R^2 to us can do in 30 sec­onds in Ex­cel.

• Charlie Steiner 13 Mar 2020 6:49 UTC

  6 points

  on: Adap­tive Im­mune Sys­tem Aging

  This was not what I ex­pected to learn to­day :) Alas, poor go­nads, I hardly knew ye.

• Charlie Steiner 12 Mar 2020 23:16 UTC

  5 points

  in reply to: jessicata's comment on: Puz­zles for Physicalists

  Well, I was skim­ming through Word and Ob­ject when I “be­came en­light­ened,” but it may have mostly been a cat­a­lyst. Still recom­mended though?

  I don’t think l was very clear about what prob­lem I was solv­ing, and I don’t think you man­aged to read my mind, so let me try again.

  The prob­lem I was in­ter­ested in was: how does refer­ence work? How can I point at or ver­bally in­di­cate some thingie, and ac­tu­ally be in­di­cat­ing the thingie in ques­tion? And could I pro­gram that into an AI?

  In your post, you con­nect this to in­dex­i­cals, which I’ve in­ter­preted as a ques­tion like “how does refer­ence work? How can I point at or ver­bally in­di­cate some thingie, and ac­tu­ally be in­di­cat­ing that thingie, in a way that you could ex­plain to a micro­scopic physics simu­la­tion?”

  One of the key parts of the solu­tion is that words don’t have in­her­ent “about­ness” at­tached to them. Refer­ence doesn’t make any sense if you just fo­cus on the speaker and try to define the about­ness in their state­ments. It needs to be in­ter­preted as com­mu­ni­ca­tion, which uses some no­tion of a func­tional au­di­ence you’re con­struct­ing a mes­sage for.

  So that ques­tion of “How do I ver­bally in­di­cate the thing and re­ally in­di­cate it?” has to be left unan­swered to the ex­tent that we have false be­liefs about our abil­ity to “re­ally in­di­cate” things. In­stead, I ad­vo­cate break­ing it down into ques­tions about how you model other peo­ple and choose com­mu­nica­tive acts.

  So I am ab­solutely not say­ing we should re­place “is x true?” with “is x a com­mu­nica­tively use­ful act?”. The clos­est thing I’m say­ing would be that we can cash out “what is the refer­ent of sen­tence x?” into “what is the mod­eled au­di­ence get­ting pointed at by the act of say­ing sen­tence x?”.

  I’m not sure how you’re in­ter­pret­ing phys­i­cal­ism here. But if we sin­gle out the no­tion that there should be some kind of “physics short­hand” for hu­man con­cepts and refer­ences—like H2O is for wa­ter, or like the toy model of refer­ence as pass­ing nu­mer­i­cal co­or­di­nates—then yeah, there is no physics short­hand. Where there is some­thing like it, it is hu­mans that have done the work to ac­co­mo­date physics, not vice versa.

• Charlie Steiner 12 Mar 2020 13:43 UTC

  7 points

  on: Puz­zles for Physicalists

  Yeah, I spent a lot of last year strug­gling with the refer­ence thing. In the end I de­cided that refer­ence was not fun­da­men­tal even within the hu­man-cen­tered pic­ture, and that refer­ence was just a spe­cial case of com­mu­ni­ca­tion (in the sense of Quine, Grice, et al.: I do a com­mu­nica­tive act be­cause I model you as mod­el­ing why I do it.)

  Figur­ing this out made me a bit up­set with aca­demic philos­o­phy, be­cause I’d been look­ing through the re­cent liter­a­ture fruitlessly be­fore I found Quine ba­si­cally solv­ing the prob­lem 50 years be­fore. This is the op­po­site of the prob­lem I usu­ally pin on philos­o­phy, that it’s too back­ward-look­ing. In this case, it’s more like the peo­ple talk­ing about refer­ence within the last 20 years are all self-se­lected for not car­ing about Quine much at all.

  Whether or not you find this use­ful may de­pend on a cer­tain men­tal ma­neu­ver of tak­ing some­thing you were ask­ing a ques­tion about, and break­ing it into pieces rather than an­swer­ing the ques­tion. In this case, “How are the se­man­tics of a sen­tence de­ter­mined?” is a ques­tion, but rather than an­swer­ing it I’m ad­vo­cat­ing get­ting rid of this high-level-of-ab­strac­tion word “se­man­tics” by work­ing in a more con­crete level of de­scrip­tion where there are hu­mans with mod­els of each other. And of course I’ve framed this in a very palat­able way, but I think whether this ma­neu­ver feels good or not is a big di­vid­ing line—if you have the un­shake­able feel­ing that I have missed some­thing vi­tal by not an­swer­ing the origi­nal ques­tion, then you fall on the other side of the line—though per­haps one can still be lured over with prac­ti­cal ap­pli­ca­tions.

• Charlie Steiner 12 Mar 2020 13:17 UTC

  4 points

  in reply to: G Gordon Worley III's comment on: [Ar­ti­cle re­view] Ar­tifi­cial In­tel­li­gence, Values, and Alignment

  It sure seems like if he re­ally grokked the philo­soph­i­cal and tech­ni­cal challenge of get­ting a GAI agent to be net benefi­cial, he would write a differ­ent pa­per. That first challenge sort of over­shad­ows the task of di­vid­ing up the post-sin­gu­lar­ity pie.

  But I’m not sure whether the over­shad­ow­ing is merely by be­ing big­ger (in which case this pa­per is still do­ing use­ful work), or if we should ex­pect that solu­tions to the pie-di­vid­ing prob­lems (e.g. weigh­ing egal­i­tar­i­anism vs. util­i­tar­i­anism) will nec­es­sar­ily fall out of the pro­cess that lets the AI learn how to be­have well.

• Charlie Steiner 11 Mar 2020 9:18 UTC

  LW: 4 AF: 2

  AF

  on: Zoom In: An In­tro­duc­tion to Circuits

  I’ll prob­a­bly post a child com­ment af­ter I ac­tu­ally read the ar­ti­cle, but I want to note be­fore I do that I think the power of ResNets are ev­i­dence against these claims. Hav­ing su­per-deep net­works with resi­d­ual con­nec­tions pro­mote a pic­ture that looks much more like a con­tin­u­ous “mas­sag­ing” of the data than a hu­man-friendly de­ci­sion tree.

• Charlie Steiner 9 Mar 2020 6:00 UTC

  4 points

  on: Quadratic mod­els and (un)falsified data

  Thanks!

  Pick­ing a de­scrip­tive statis­tic for these sorts of prob­lems is pretty tricky. But I think we can do bet­ter than R^2, even with­out go­ing all Bayesian-pa­ram­e­ter-es­ti­ma­tion.

  What I mostly care about is just the stan­dard de­vi­a­tion (in ex­cel, STDEV.S() ) of the differ­ence be­tween the data and the model. Then I want to know how this com­pares to other scales in the data (like the av­er­age num­ber of new cases per day).

• Charlie Steiner 27 Feb 2020 18:40 UTC

  LW: 4 AF: 2

  AF

  in reply to: TurnTrout's comment on: At­tain­able Utility Preser­va­tion: Scal­ing to Superhuman

  Right. Some in­tu­ition is nec­es­sary. But a lot of these choices are ad hoc, by which I mean they aren’t strongly con­strained by the re­sult you want from them.

  For ex­am­ple, you have a lin­ear penalty gov­erned by this pa­ram­e­ter lambda, but in prin­ci­ple it could have been any old func­tion—the only strong con­straint is that you want it to mono­ton­i­cally in­crease from a finite num­ber to in­finity. Now, maybe this is fine, or maybe not. But I ba­si­cally don’t have much trust for med­i­ta­tion in this sort of case, and would rather see ex­plicit con­straints that rule out more of the available space.

• Charlie Steiner 27 Feb 2020 18:36 UTC

  LW: 4 AF: 2

  AF

  in reply to: Stuart_Armstrong's comment on: At­tain­able Utility Preser­va­tion: Scal­ing to Superhuman

  Jinx :P

• Charlie Steiner 27 Feb 2020 9:59 UTC

  LW: 7 AF: 4

  AF

  on: At­tain­able Utility Preser­va­tion: Scal­ing to Superhuman

  My very gen­eral con­cern is that strate­gies that max­i­mize $R_{AUP}$ might be very… let’s say cre­ative, and your claims are mostly rely­ing on in­tu­itive ar­gu­ments for why those strate­gies won’t be bad for hu­mans.

  I don’t re­ally buy the claim that if you’ve been able to patch each spe­cific prob­lem, we’ll soon reach a ver­sion with no prob­lems—the ex­act same in­duc­tive ar­gu­ment you men­tion sug­gests that there will just be a se­ries of prob­lems, and patches, and then more prob­lems with the patched ver­sion. Again, I worry that patches are based a lot on in­tu­ition.

  For ex­am­ple, in the lat­est ver­sion, be­cause you’re es­sen­tially di­vid­ing out by the long-term re­ward of tak­ing the best ac­tion now, if the best ac­tion now is re­ally re­ally good, then it be­comes cheap to take mod­er­ately good ac­tions that still in­crease fu­ture re­ward—which means the agent is in­cen­tivized to con­cen­trate the power of ac­tions into spe­cific tim­steps. For ex­am­ple, an agent might be able to set things up so that it can sac­ri­fice its abil­ity to achieve to­tal fu­ture re­ward of ${10}^{10}$ to make it cheap to take an ac­tion that in­creases its fu­ture re­ward by ${10}^8$ . This might looks like sac­ri­fic­ing the abil­ity to colonize dis­tant galax­ies in or­der to gain to­tal con­trol over the Milky Way.

• Charlie Steiner 24 Feb 2020 6:04 UTC

  3 points

  on: How much de­lay do you gen­er­ally have be­tween hav­ing a good new idea and shar­ing that idea pub­li­cly on­line?

  For in­ter­est­ing stuff, two weeks to two months. Usu­ally this is war­ranted, be­cause ideas are cheap but fil­ter­ing and think­ing are hard. The ideal faster time mostly just means that ideally I’d be spend­ing more hours per week on ideas, not that I’d be spend­ing less time per idea.

• Charlie Steiner 23 Feb 2020 7:30 UTC

  LW: 2 AF: 1

  AF

  in reply to: michaelcohen's comment on: Cu­ri­os­ity Killed the Cat and the Asymp­tot­i­cally Op­ti­mal Agent

  After a bit more thought, I’ve learned that it’s hard to avoid end­ing back up with EU max­i­miza­tion—it ba­si­cally hap­pens as soon as you re­quire that strate­gies be good not just on the true en­vi­ron­ment, but on some dis­tri­bu­tion of en­vi­ron­ments that re­flect what we think we’re de­sign­ing an agent for (or the agent’s ini­tial state of knowl­edge about states of the world). And since this is such an effec­tive tool at pe­nal­iz­ing the “just pick the ab­solute best an­swer” strat­egy, it’s hard for me to avoid cir­cling back to it.

  Here’s one pos­si­ble op­tion, though: look for strate­gies that are too sim­ple to en­code the one best an­swer in the first place. If the ab­solute best policy has K-com­plex­ity of 10^3 (achiev­able in the real world by strate­gies be­ing com­pli­cated, or in the multi-armed ban­dit case by just hav­ing 2^1000 pos­si­ble ac­tions) and your agent is only al­lowed to start with 10^2 sym­bols, this might make things in­ter­est­ing.

• Charlie Steiner 23 Feb 2020 7:18 UTC

  4 points

  on: North­west Pas­sage Update

  I like it! But you know, North­west Pas­sage is already writ­ten as a ret­ro­spec­tive.

  Three cen­turies there­after, I take pas­sage over­land
  In the foot­steps of brave Kelso, where his “sea of flow­ers” be­gan
  Watch­ing cities rise be­fore me, then be­hind me sink again
  This tardiest ex­plorer, driv­ing hard across the plain.

  And through the night, be­hind the wheel, the mileage click­ing west
  I think upon Macken­zie, David Thomp­son and the rest
  Who cracked the moun­tain ram­parts and did show a path for me
  To race the roar­ing Fraser to the sea.

  Be­cause the singer is mod­ern, the cho­rus “Ah, for just one time /​ I would take the North­west Pas­sage” is about wish­ing to iden­tify a lonely life with the grandeur of the past. A verse about the loss of the his­tor­i­cal arc­tic would tie right back into this with­out need­ing to change the cho­rus a jot.