interstice

Hal­i­fax SSC Meetup—FEB 8

HALIFAX SSC MEETUP—FEB. 1

SSC Hal­i­fax Meetup—Jan­uary 25

• interstice 17 Jan 2020 4:55 UTC

  14 points

  on: Real­ity-Re­veal­ing and Real­ity-Mask­ing Puzzles

  The post men­tions prob­lems that en­courage peo­ple to hide re­al­ity from them­selves. I think that con­struct­ing a ‘mean­ingful life nar­ra­tive’ is a pretty ubiquitous such prob­lem. For the ma­jor­ity of peo­ple, con­struct­ing a nar­ra­tive where their life has in­trin­sic im­por­tance is go­ing to in­volve a cer­tain amount of self-de­cep­tion.

  Some of the prob­lems that come from the in­ter­ac­tion be­tween these sorts of nar­ra­tives and learn­ing about x-risks have already been men­tioned. To me, how­ever, it looks like some of the AI x-risk memes them­selves are par­tially the re­sult of re­al­ity-mask­ing op­ti­miza­tion with the goal of in­creas­ing the per­ceived mean­ingful­ness of the lives of peo­ple work­ing on AI x-risk. As an ex­am­ple, con­sider the on­go­ing de­bate about whether we should ex­pect the field of AI to mostly solve x-risk on its own. Clearly, if the field can’t be counted upon to avoid the de­struc­tion of hu­man­ity, this greatly in­creases the im­por­tance of out­side re­searchers try­ing to help them. So to satisfy their emo­tional need to feel that their ac­tions have mean­ing, out­side re­searchers have a bias to­wards think­ing that the field is more in­com­pe­tent than it is, and to come up with and prop­a­gate memes jus­tify­ing that con­clu­sion. Peo­ple who are already in in­sider in­sti­tu­tions have the op­po­site bias, so it makes sense that this de­bate di­vides to some ex­tent along these lines.

  From this per­spec­tive, it’s no co­in­ci­dence that in­ter­nal­iz­ing some x-risk memes leads peo­ple to feel that their ac­tions are mean­ingless. Since the memes are par­tially op­ti­mized to in­crease the per­ceived mean­ingful­ness of the ac­tions of a small group of peo­ple, by ne­ces­sity they will de­crease the per­ceived mean­ingful­ness of ev­ery­one else’s ac­tions.

  (Just to be clear, I’m not say­ing that these ideas have no value, that this is be­ing done con­sciously, or that the origi­na­tors of said memes are ‘bad’; this is a pretty uni­ver­sal hu­man be­hav­ior. Nor would I en­dorse bring­ing up these mo­tives in an ob­ject-level con­ver­sa­tion about the is­sues. How­ever, since this post is about re­al­ity-mask­ing prob­lems it seems re­miss not to men­tion.)

• interstice 16 Jan 2020 20:44 UTC

  1 point

  in reply to: paulfchristiano's comment on: Clar­ify­ing The Mal­ig­nity of the Univer­sal Prior: The Lex­i­cal Update

  Thanks, that makes sense. Here is my rephras­ing of the ar­gu­ment:
  
  Let the ‘im­por­tance func­tion’ $Q(X,Y)$ take as in­puts ma­chines $X$and $Y$, and out­put all places where $Y$ is be­ing used as a uni­ver­sal prior, weighted by their effect on $X$-short pro­grams. Sup­pose for the sake of ar­gu­ment that there is some short pro­gram com­put­ing $Q$; this is prob­a­bly the most ‘nat­u­ral’ pro­gram of this form that we could hope for.
  
  Even given such a pro­gram, we’ll still lose to the aliens: in $U^{\prime }$, di­rectly spec­i­fy­ing our im­por­tant de­ci­sions on Earth us­ing $Q$ will re­quire both $U$ and $U^{\prime }$ to be fed into $Q$, cost­ing $K_{U^{\prime }}\left(U\right)+K_{U^{\prime }}\left(U^{\prime }\right)$ bits, then $K_{U^{\prime }}\left(us|Q\right(U,U^{\prime }\left)\right)$ bits to spec­ify us. For the aliens, get­ting them to be mo­ti­vated to con­trol $U$-short pro­grams costs $K_{U^{\prime }}\left(U\right)$ bits, but then they can skip di­rectly to spec­i­fy­ing us given $Q(U,U^{\prime })$, so they save $K_{U^{\prime }}\left(U^{\prime }\right)$ bits over the di­rect ex­pla­na­tion. So the lex­i­cal up­date works.

  (I went wrong in think­ing that the aliens would need to both up­date their no­tion of im­por­tance to match ours *and* lo­cate our world; but if we as­sume the ‘im­por­tance func­tion’ ex­ists then the aliens can just pick out our world us­ing our no­tion of im­por­tance)

Clar­ify­ing The Mal­ig­nity of the Univer­sal Prior: The Lex­i­cal Update

Hal­i­fax SSC Meetup—Satur­day 11/​1/​20

• interstice 5 Jan 2020 6:12 UTC

  7 points

  on: Ma­chine Learn­ing Can’t Han­dle Long-Term Time-Series Data

  To­day’s neu­ral net­works definitely have prob­lems solv­ing more ‘struc­tured’ prob­lems, but I don’t think that ‘neu­ral nets can’t learn long time-se­ries data’ is a good way of fram­ing this. To go through your ex­am­ples:

  This shouldn’t have been a ma­jor is­sue, ex­cept that with each switch it dis­carded past ob­ser­va­tions. Had the car main­tained this his­tory it would have seen that some sort of large ob­ject was pro­gress­ing across the street on a col­li­sion course, and had plenty of time to stop.

  From a brief read­ing of the re­port, this sounds like this con­trol logic is part of the sys­tem sur­round­ing the neu­ral net­work, not the net­work it­self.

  One net­work pre­dicts the odds of win­ning and an­other net­work figures out which move to perform. This turns a time-se­ries prob­lem (what strat­egy to perform) into a two sep­a­rate state­less[1] prob­lems.

  I don’t see how you think this is ‘state­less’. AlphaS­tar’s ar­chi­tec­ture con­tains an LSTM(‘Core’) which is then fed into the value and move net­works, similar to most time se­ries ap­pli­ca­tions of neu­ral net­works.

  Most con­spicu­ously, hu­man be­ings know how to build walls with build­ings. This re­quires a se­quence of steps that don’t gen­er­ate a use­ful re­sult un­til the last of them are com­pleted. A wall is use­less un­til the last build­ing is put into place. AlphaS­tar (the red player in the image be­low) does not know how to build walls.

  But the net­work does learn how to build its econ­omy, which also doesn’t pay off for a very long time. I think the is­sue here is more about a lack of ‘rea­son­ing’ skills than time-scales: the net­work can’t think con­cep­tu­ally, and so doesn’t know that a wall needs to com­pletely block off an area to be use­ful. It just learns a set of as­so­ci­a­tions.

  ML can gen­er­ate clas­si­cal mu­sic just fine but can’t figure out the cho­rus/​verse sys­tem used in rock & roll.

  MustNet was trained from scratch on MIDI data, but it’s still able to gen­er­ate mu­sic with lots of struc­ture on both short and long time scales. GPT2 does the same for text. I’m not sure if MuseNet is able to gen­er­ate cho­rus/​verse struc­tures in par­tic­u­lar, but again this seems more like an is­sue of lack of logic/​con­cepts than time scales(that is, MuseNet can make pieces that ‘sound right’ but has no con­cep­tual un­der­stand­ing of their struc­ture)

  I’ll note that AlphaS­tar, GPT2, and MuseNet all use the Trans­former ar­chi­tec­ture, which seems quite effec­tive for struc­tured time-se­ries data. I think this is be­cause its at­ten­tional mechanism lets it zoom in on the rele­vant parts of past ex­pe­riences.

  I also don’t see how con­nec­tome-spe­cific-waves are sup­posed to help. I think(?) your sug­ges­tion is to store slow-chang­ing data in the largest eigen­vec­tors of the Lapla­cian—but why would this be an im­prove­ment? It’s already the case(by the na­ture of the ma­trix) that the largest eigen­vec­tors of e.g. an RNN’s tran­si­tion ma­trix will tend to store data for longer time pe­ri­ods.

• interstice 1 Jan 2020 23:35 UTC

  12 points

  in reply to: romeostevensit's comment on: romeosteven­sit’s Shortform

  Steroids do fuck a bunch of things up, like fer­til­ity, so they make evolu­tion­ary sense. This sug­gests we should look to po­ten­tially dan­ger­ous or harm­ful al­ter­a­tions to get real IQ boosts. Greg cochran has a post sug­gest­ing gout might be like this.

• interstice 22 Dec 2019 0:16 UTC

  1 point

  in reply to: George's comment on: Un­der­stand­ing Ma­chine Learn­ing (I)

  This seems much too strong, lots of in­ter­est­ing un­solved prob­lems can be cast as i.i.d. Video clas­sifi­ca­tion, for ex­am­ple, can be cast as i.i.d., where the dis­tri­bu­tion is over differ­ent videos, not in­di­vi­d­ual frames.

• interstice 21 Dec 2019 23:47 UTC

  22 points

  in reply to: Scott Alexander's comment on: Free Speech and Triskaideka­pho­bic Calcu­la­tors: A Re­ply to Hub­inger on the Rele­vance of Public On­line Dis­cus­sion to Ex­is­ten­tial Risk

  In the anal­ogy, it’s only pos­si­ble to build a calcu­la­tor that out­puts the right an­swer on non-13 num­bers be­cause you already un­der­stand the true na­ture of ad­di­tion. It might be more difficult if you were con­fused about ad­di­tion, and were try­ing to come up with a gen­eral the­ory by ex­trap­o­lat­ing from known cases—then, think­ing 6 + 7 = 15 could eas­ily send you down the wrong path. In the real world, we’re similarly con­fused about hu­man prefer­ences, mind ar­chi­tec­ture, the na­ture of poli­tics, etc., but some of the in­for­ma­tion we might want to use to build a gen­eral the­ory is taboo. I think that some of these ques­tions are di­rectly rele­vant to AI—e.g. the na­ture of hu­man prefer­ences is rele­vant to build­ing an AI to satisfy those prefer­ences, the na­ture of poli­tics could be rele­vant to rea­son­ing about what the lead-up to AGI will look like, etc.

• interstice 16 Dec 2019 3:54 UTC

  1 point

  in reply to: FactorialCode's comment on: What de­ter­mines the bal­ance be­tween in­tel­li­gence sig­nal­ing and virtue sig­nal­ing?

  Fair point, but I note that the co­op­er­a­tive abil­ity only in­creases fit­ness here be­cause it boosts the in­di­vi­d­u­als’ sta­tus, i.e. they are in a situ­a­tion where sta­tus-jock­ey­ing and co­op­er­a­tive be­hav­ior are al­igned. Of course it’s true that they _are_ of­ten so al­igned.

• interstice 15 Dec 2019 22:39 UTC

  1 point

  in reply to: Logan Zoellner's comment on: Many Tur­ing Machines

  I agree it’s hard to get the ex­act de­tails of the MUH right, but pretty much any ver­sion seems bet­ter to me than ‘only ob­serv­able things ex­ist’ for the rea­sons I ex­plained in my com­ment. And pretty much any ver­sion en­dorses many-wor­lds(of course you can be­lieve many-wor­lds with­out be­liev­ing MUH). Really this is just a de­bate about the mean­ing of the word ‘ex­ist’.

• interstice 12 Dec 2019 7:11 UTC

  4 points

  in reply to: FactorialCode's comment on: What de­ter­mines the bal­ance be­tween in­tel­li­gence sig­nal­ing and virtue sig­nal­ing?

  Abil­ity to co­op­er­ate is im­por­tant, but I think that sta­tus-jock­ey­ing is a more ‘fun­da­men­tal’ ad­van­tage be­cause it gives an ad­van­tage to in­di­vi­d­u­als, not just groups. Any adap­ta­tion that aids groups must first be use­ful enough to in­di­vi­d­u­als to reach fix­a­tion(or near-fix­a­tion) in some groups.

• interstice 10 Dec 2019 20:53 UTC

  6 points

  on: Many Tur­ing Machines

  You’ve es­sen­tially re-in­vented the Math­e­mat­i­cal Uni­verse Hy­poth­e­sis, which many peo­ple around here do in fact be­lieve.

  For some in­tu­ition as to why peo­ple would think that things that can’t ever af­fect our fu­ture ex­pe­riences are ‘real’, imag­ine liv­ing in the dis­tant past and watch­ing your rel­a­tives travel to a dis­tant land, and as­sume that long-dis­tance com­mu­ni­ca­tion such as writ­ing is im­pos­si­ble. You would prob­a­bly still care about them and think they are ‘real’ , even though by your defi­ni­tion they no longer ex­ist to you. Or if you want to quib­ble about the slight chance of see­ing them again, imag­ine be­ing in the fu­ture and watch­ing them get on a space­ship which will travel be­yond your ob­serv­able hori­zon. Again, it still seems like you would still care about them and con­sider them ‘real’.

• interstice 9 Dec 2019 6:39 UTC

  4 points

  in reply to: Wei_Dai's comment on: What de­ter­mines the bal­ance be­tween in­tel­li­gence sig­nal­ing and virtue sig­nal­ing?

  Do you agree that sig­nal­ling in­tel­li­gence is the main ex­pla­na­tion for the evolu­tion of lan­guage? To me, it seems like coal­i­tion-build­ing is a more fun­da­men­tal driv­ing force(af­ter all, be­ing at­tracted to in­tel­li­gence only makes sense if in­tel­li­gence is already valuable in some con­texts, and coal­i­tion poli­tics seems like an es­pe­cially im­por­tant do­main) Miller has also ar­gued that sex­ual sig­nal­ling is a main ex­pla­na­tion of art and mu­sic, which Will Buck­ing­ham has a good cri­tique of here.

• interstice 9 Dec 2019 6:11 UTC

  2 points

  in reply to: Liam Donovan's comment on: Q&A with Shane Legg on risks from AI

  As far as I know no one’s tried to build a unified sys­tem with all of those ca­pac­i­ties, but we do seem to have rudi­men­tary learned ver­sions of each of the ca­pac­i­ties on their own.

• interstice 6 Dec 2019 21:18 UTC

  1 point

  in reply to: FactorialCode's comment on: Re­cent Progress in the The­ory of Neu­ral Networks

  That’s an in­ter­est­ing link. It sound like the re­sults can only be ap­plied to strictly Bayesian meth­ods though, so they couldn’t be ap­plied to neu­ral net­works as they ex­ist now.

• interstice 6 Dec 2019 4:16 UTC

  LW: 3 AF: 2

  AF

  in reply to: FactorialCode's comment on: Re­cent Progress in the The­ory of Neu­ral Networks

  Not sure if I agree re­gard­ing the real-world use­ful­ness. For the non-IID case, PAC-Bayes bounds fail, and to re-in­state them you’d need as­sump­tions about how quickly the dis­tri­bu­tion changes, but then it’s plau­si­ble that you could get high prob­a­bil­ity bounds based on the most re­cent perfor­mance. For small datasets, the PAC-Bayes bounds suffer be­cause they scale as $\sqrt{\frac{KL}{N}}$ . (I may edit the post to be clearer about this)

  Agreed that an­a­lyz­ing how the bounds change un­der differ­ent con­di­tions could be in­sight­ful though. Ul­ti­mately I sus­pect that effec­tive bounds will re­quire pow­er­ful ways to ex­tract ‘the sig­nal from the noise’, and ex­am­in­ing the sig­nal will likely be use­ful for un­der­stand­ing if a model has truly learned what it is sup­posed to.

• interstice 6 Dec 2019 2:15 UTC

  LW: 13 AF: 8

  AF

  in reply to: johnswentworth's comment on: Un­der­stand­ing “Deep Dou­ble Des­cent”

  The neu­ral tan­gent ker­nel guys have a pa­per where they give a heuris­tic ar­gu­ment ex­plain­ing the dou­ble de­scent curve(in num­ber of pa­ram­e­ters) us­ing the NTK.