interstice

• interstice 2 Jun 2020 1:12 UTC

  1 point

  in reply to: Charlie Steiner’s comment on: The Pre­sump­tu­ous Philoso­pher, self-lo­cat­ing in­for­ma­tion, and Solomonoff induction

  Another thing, I don’t think Solomonoff In­duc­tion would give an ad­van­tage of log(n) to the­o­ries with n ob­servers. In the post you men­tion tak­ing the dis­crete in­te­gral of $2^{-log\left(n\right)}=\frac{1}{n}$ to get log scal­ing, but this seems to be based on the plain Kol­mogorov com­plex­ity $C\left(n\right)$, for which $log\left(n\right)$ is ap­prox­i­mately an up­per bound. Solomonoff in­duc­tion uses pre­fix com­plex­ity $K\left(n\right)$, and the dis­crete in­te­gral of $2^{-K\left(n\right)}$ con­verges to a con­stant. This means hav­ing more copies in the uni­verse can give you at most a con­stant ad­van­tage.

  (Based on read­ing some other com­ments it sounds like you might already know this. In any case, it means S.I. is even more anti-PP than im­plied in the post)

• interstice 1 Jun 2020 20:25 UTC

  8 points

  on: The Pre­sump­tu­ous Philoso­pher, self-lo­cat­ing in­for­ma­tion, and Solomonoff induction

  It seems to me that there are (at least) two ways of spec­i­fy­ing ob­servers given a phys­i­cal world-model, and two cor­re­spond­ing ways this would af­fect an­throp­ics in Solomonoff in­duc­tion:

  • You could spec­ify their lo­ca­tion in space-time. In this case, what mat­ters isn’t the num­ber of copies, but rather their den­sity in space, be­cause ob­servers be­ing more sparse in the uni­verse means more bits are needed to pin-point their lo­ca­tion.

  • You could spec­ify what this type of ob­server looks like, run a search for things in the uni­verse match­ing that de­scrip­tion, then pick one off the list. In this case, again what mat­ters is the den­sity of us(ob­servers with the se­quence of ob­ser­va­tions we are try­ing to pre­dict) among all ob­servers of the same type.

  Which of the two meth­ods ends up be­ing the lead­ing con­trib­u­tor to the Solomonoff prior de­pends on the de­tails of the uni­verse and the type of ob­server. But ei­ther way, I think the Pre­sump­tu­ous Philoso­pher’s ar­gu­ment ends up be­ing re­jected: in the ‘search­ing’ case, it seems like differ­ent phys­i­cal the­o­ries shouldn’t af­fect the fre­quency of differ­ent peo­ple in the uni­verse, and in the ‘lo­ca­tion’ case, it seems that any phys­i­cal the­ory com­pat­i­ble with lo­cal ob­ser­va­tions shouldn’t be able to af­fect the den­sity much, be­cause we would per­ceive any copies that were close enough.

Hal­i­fax SSC Meetup—FEB 8

HALIFAX SSC MEETUP—FEB. 1

SSC Hal­i­fax Meetup—Jan­uary 25

• interstice 17 Jan 2020 4:55 UTC

  14 points

  on: Real­ity-Re­veal­ing and Real­ity-Mask­ing Puzzles

  The post men­tions prob­lems that en­courage peo­ple to hide re­al­ity from them­selves. I think that con­struct­ing a ‘mean­ingful life nar­ra­tive’ is a pretty ubiquitous such prob­lem. For the ma­jor­ity of peo­ple, con­struct­ing a nar­ra­tive where their life has in­trin­sic im­por­tance is go­ing to in­volve a cer­tain amount of self-de­cep­tion.

  Some of the prob­lems that come from the in­ter­ac­tion be­tween these sorts of nar­ra­tives and learn­ing about x-risks have already been men­tioned. To me, how­ever, it looks like some of the AI x-risk memes them­selves are par­tially the re­sult of re­al­ity-mask­ing op­ti­miza­tion with the goal of in­creas­ing the per­ceived mean­ingful­ness of the lives of peo­ple work­ing on AI x-risk. As an ex­am­ple, con­sider the on­go­ing de­bate about whether we should ex­pect the field of AI to mostly solve x-risk on its own. Clearly, if the field can’t be counted upon to avoid the de­struc­tion of hu­man­ity, this greatly in­creases the im­por­tance of out­side re­searchers try­ing to help them. So to satisfy their emo­tional need to feel that their ac­tions have mean­ing, out­side re­searchers have a bias to­wards think­ing that the field is more in­com­pe­tent than it is, and to come up with and prop­a­gate memes jus­tify­ing that con­clu­sion. Peo­ple who are already in in­sider in­sti­tu­tions have the op­po­site bias, so it makes sense that this de­bate di­vides to some ex­tent along these lines.

  From this per­spec­tive, it’s no co­in­ci­dence that in­ter­nal­iz­ing some x-risk memes leads peo­ple to feel that their ac­tions are mean­ingless. Since the memes are par­tially op­ti­mized to in­crease the per­ceived mean­ingful­ness of the ac­tions of a small group of peo­ple, by ne­ces­sity they will de­crease the per­ceived mean­ingful­ness of ev­ery­one else’s ac­tions.

  (Just to be clear, I’m not say­ing that these ideas have no value, that this is be­ing done con­sciously, or that the origi­na­tors of said memes are ‘bad’; this is a pretty uni­ver­sal hu­man be­hav­ior. Nor would I en­dorse bring­ing up these mo­tives in an ob­ject-level con­ver­sa­tion about the is­sues. How­ever, since this post is about re­al­ity-mask­ing prob­lems it seems re­miss not to men­tion.)

• interstice 16 Jan 2020 20:44 UTC

  1 point

  in reply to: paulfchristiano’s comment on: Clar­ify­ing The Mal­ig­nity of the Univer­sal Prior: The Lex­i­cal Update

  Thanks, that makes sense. Here is my rephras­ing of the ar­gu­ment:
  
  Let the ‘im­por­tance func­tion’ $Q(X,Y)$ take as in­puts ma­chines $X$and $Y$, and out­put all places where $Y$ is be­ing used as a uni­ver­sal prior, weighted by their effect on $X$-short pro­grams. Sup­pose for the sake of ar­gu­ment that there is some short pro­gram com­put­ing $Q$; this is prob­a­bly the most ‘nat­u­ral’ pro­gram of this form that we could hope for.
  
  Even given such a pro­gram, we’ll still lose to the aliens: in $U^{\prime }$, di­rectly spec­i­fy­ing our im­por­tant de­ci­sions on Earth us­ing $Q$ will re­quire both $U$ and $U^{\prime }$ to be fed into $Q$, cost­ing $K_{U^{\prime }}\left(U\right)+K_{U^{\prime }}\left(U^{\prime }\right)$ bits, then $K_{U^{\prime }}\left(us|Q\right(U,U^{\prime }\left)\right)$ bits to spec­ify us. For the aliens, get­ting them to be mo­ti­vated to con­trol $U$-short pro­grams costs $K_{U^{\prime }}\left(U\right)$ bits, but then they can skip di­rectly to spec­i­fy­ing us given $Q(U,U^{\prime })$, so they save $K_{U^{\prime }}\left(U^{\prime }\right)$ bits over the di­rect ex­pla­na­tion. So the lex­i­cal up­date works.

  (I went wrong in think­ing that the aliens would need to both up­date their no­tion of im­por­tance to match ours *and* lo­cate our world; but if we as­sume the ‘im­por­tance func­tion’ ex­ists then the aliens can just pick out our world us­ing our no­tion of im­por­tance)

Clar­ify­ing The Mal­ig­nity of the Univer­sal Prior: The Lex­i­cal Update

Hal­i­fax SSC Meetup—Satur­day 11/​1/​20

• interstice 5 Jan 2020 6:12 UTC

  9 points

  on: Ma­chine Learn­ing Can’t Han­dle Long-Term Time-Series Data

  To­day’s neu­ral net­works definitely have prob­lems solv­ing more ‘struc­tured’ prob­lems, but I don’t think that ‘neu­ral nets can’t learn long time-se­ries data’ is a good way of fram­ing this. To go through your ex­am­ples:

  This shouldn’t have been a ma­jor is­sue, ex­cept that with each switch it dis­carded past ob­ser­va­tions. Had the car main­tained this his­tory it would have seen that some sort of large ob­ject was pro­gress­ing across the street on a col­li­sion course, and had plenty of time to stop.

  From a brief read­ing of the re­port, this sounds like this con­trol logic is part of the sys­tem sur­round­ing the neu­ral net­work, not the net­work it­self.

  One net­work pre­dicts the odds of win­ning and an­other net­work figures out which move to perform. This turns a time-se­ries prob­lem (what strat­egy to perform) into a two sep­a­rate state­less[1] prob­lems.

  I don’t see how you think this is ‘state­less’. AlphaS­tar’s ar­chi­tec­ture con­tains an LSTM(‘Core’) which is then fed into the value and move net­works, similar to most time se­ries ap­pli­ca­tions of neu­ral net­works.

  Most con­spicu­ously, hu­man be­ings know how to build walls with build­ings. This re­quires a se­quence of steps that don’t gen­er­ate a use­ful re­sult un­til the last of them are com­pleted. A wall is use­less un­til the last build­ing is put into place. AlphaS­tar (the red player in the image be­low) does not know how to build walls.

  But the net­work does learn how to build its econ­omy, which also doesn’t pay off for a very long time. I think the is­sue here is more about a lack of ‘rea­son­ing’ skills than time-scales: the net­work can’t think con­cep­tu­ally, and so doesn’t know that a wall needs to com­pletely block off an area to be use­ful. It just learns a set of as­so­ci­a­tions.

  ML can gen­er­ate clas­si­cal mu­sic just fine but can’t figure out the cho­rus/​verse sys­tem used in rock & roll.

  MustNet was trained from scratch on MIDI data, but it’s still able to gen­er­ate mu­sic with lots of struc­ture on both short and long time scales. GPT2 does the same for text. I’m not sure if MuseNet is able to gen­er­ate cho­rus/​verse struc­tures in par­tic­u­lar, but again this seems more like an is­sue of lack of logic/​con­cepts than time scales(that is, MuseNet can make pieces that ‘sound right’ but has no con­cep­tual un­der­stand­ing of their struc­ture)

  I’ll note that AlphaS­tar, GPT2, and MuseNet all use the Trans­former ar­chi­tec­ture, which seems quite effec­tive for struc­tured time-se­ries data. I think this is be­cause its at­ten­tional mechanism lets it zoom in on the rele­vant parts of past ex­pe­riences.

  I also don’t see how con­nec­tome-spe­cific-waves are sup­posed to help. I think(?) your sug­ges­tion is to store slow-chang­ing data in the largest eigen­vec­tors of the Lapla­cian—but why would this be an im­prove­ment? It’s already the case(by the na­ture of the ma­trix) that the largest eigen­vec­tors of e.g. an RNN’s tran­si­tion ma­trix will tend to store data for longer time pe­ri­ods.

• interstice 1 Jan 2020 23:35 UTC

  12 points

  in reply to: romeostevensit’s comment on: romeosteven­sit’s Shortform

  Steroids do fuck a bunch of things up, like fer­til­ity, so they make evolu­tion­ary sense. This sug­gests we should look to po­ten­tially dan­ger­ous or harm­ful al­ter­a­tions to get real IQ boosts. Greg cochran has a post sug­gest­ing gout might be like this.

• interstice 22 Dec 2019 0:16 UTC

  3 points

  in reply to: George’s comment on: Un­der­stand­ing Ma­chine Learn­ing (I)

  This seems much too strong, lots of in­ter­est­ing un­solved prob­lems can be cast as i.i.d. Video clas­sifi­ca­tion, for ex­am­ple, can be cast as i.i.d., where the dis­tri­bu­tion is over differ­ent videos, not in­di­vi­d­ual frames.

• interstice 21 Dec 2019 23:47 UTC

  22 points

  in reply to: Scott Alexander’s comment on: Free Speech and Triskaideka­pho­bic Calcu­la­tors: A Re­ply to Hub­inger on the Rele­vance of Public On­line Dis­cus­sion to Ex­is­ten­tial Risk

  In the anal­ogy, it’s only pos­si­ble to build a calcu­la­tor that out­puts the right an­swer on non-13 num­bers be­cause you already un­der­stand the true na­ture of ad­di­tion. It might be more difficult if you were con­fused about ad­di­tion, and were try­ing to come up with a gen­eral the­ory by ex­trap­o­lat­ing from known cases—then, think­ing 6 + 7 = 15 could eas­ily send you down the wrong path. In the real world, we’re similarly con­fused about hu­man prefer­ences, mind ar­chi­tec­ture, the na­ture of poli­tics, etc., but some of the in­for­ma­tion we might want to use to build a gen­eral the­ory is taboo. I think that some of these ques­tions are di­rectly rele­vant to AI—e.g. the na­ture of hu­man prefer­ences is rele­vant to build­ing an AI to satisfy those prefer­ences, the na­ture of poli­tics could be rele­vant to rea­son­ing about what the lead-up to AGI will look like, etc.

• interstice 16 Dec 2019 3:54 UTC

  1 point

  in reply to: FactorialCode’s comment on: What de­ter­mines the bal­ance be­tween in­tel­li­gence sig­nal­ing and virtue sig­nal­ing?

  Fair point, but I note that the co­op­er­a­tive abil­ity only in­creases fit­ness here be­cause it boosts the in­di­vi­d­u­als’ sta­tus, i.e. they are in a situ­a­tion where sta­tus-jock­ey­ing and co­op­er­a­tive be­hav­ior are al­igned. Of course it’s true that they _are_ of­ten so al­igned.

• interstice 15 Dec 2019 22:39 UTC

  1 point

  in reply to: Logan Zoellner’s comment on: Many Tur­ing Machines

  I agree it’s hard to get the ex­act de­tails of the MUH right, but pretty much any ver­sion seems bet­ter to me than ‘only ob­serv­able things ex­ist’ for the rea­sons I ex­plained in my com­ment. And pretty much any ver­sion en­dorses many-wor­lds(of course you can be­lieve many-wor­lds with­out be­liev­ing MUH). Really this is just a de­bate about the mean­ing of the word ‘ex­ist’.

• interstice 12 Dec 2019 7:11 UTC

  4 points

  in reply to: FactorialCode’s comment on: What de­ter­mines the bal­ance be­tween in­tel­li­gence sig­nal­ing and virtue sig­nal­ing?

  Abil­ity to co­op­er­ate is im­por­tant, but I think that sta­tus-jock­ey­ing is a more ‘fun­da­men­tal’ ad­van­tage be­cause it gives an ad­van­tage to in­di­vi­d­u­als, not just groups. Any adap­ta­tion that aids groups must first be use­ful enough to in­di­vi­d­u­als to reach fix­a­tion(or near-fix­a­tion) in some groups.

• interstice 10 Dec 2019 20:53 UTC

  6 points

  on: Many Tur­ing Machines

  You’ve es­sen­tially re-in­vented the Math­e­mat­i­cal Uni­verse Hy­poth­e­sis, which many peo­ple around here do in fact be­lieve.

  For some in­tu­ition as to why peo­ple would think that things that can’t ever af­fect our fu­ture ex­pe­riences are ‘real’, imag­ine liv­ing in the dis­tant past and watch­ing your rel­a­tives travel to a dis­tant land, and as­sume that long-dis­tance com­mu­ni­ca­tion such as writ­ing is im­pos­si­ble. You would prob­a­bly still care about them and think they are ‘real’ , even though by your defi­ni­tion they no longer ex­ist to you. Or if you want to quib­ble about the slight chance of see­ing them again, imag­ine be­ing in the fu­ture and watch­ing them get on a space­ship which will travel be­yond your ob­serv­able hori­zon. Again, it still seems like you would still care about them and con­sider them ‘real’.

• interstice 9 Dec 2019 6:39 UTC

  4 points

  in reply to: Wei_Dai’s comment on: What de­ter­mines the bal­ance be­tween in­tel­li­gence sig­nal­ing and virtue sig­nal­ing?

  Do you agree that sig­nal­ling in­tel­li­gence is the main ex­pla­na­tion for the evolu­tion of lan­guage? To me, it seems like coal­i­tion-build­ing is a more fun­da­men­tal driv­ing force(af­ter all, be­ing at­tracted to in­tel­li­gence only makes sense if in­tel­li­gence is already valuable in some con­texts, and coal­i­tion poli­tics seems like an es­pe­cially im­por­tant do­main) Miller has also ar­gued that sex­ual sig­nal­ling is a main ex­pla­na­tion of art and mu­sic, which Will Buck­ing­ham has a good cri­tique of here.

• interstice 9 Dec 2019 6:11 UTC

  2 points

  in reply to: Liam Donovan’s comment on: Q&A with Shane Legg on risks from AI

  As far as I know no one’s tried to build a unified sys­tem with all of those ca­pac­i­ties, but we do seem to have rudi­men­tary learned ver­sions of each of the ca­pac­i­ties on their own.

• interstice 6 Dec 2019 21:18 UTC

  1 point

  in reply to: FactorialCode’s comment on: Re­cent Progress in the The­ory of Neu­ral Networks

  That’s an in­ter­est­ing link. It sound like the re­sults can only be ap­plied to strictly Bayesian meth­ods though, so they couldn’t be ap­plied to neu­ral net­works as they ex­ist now.