interstice

• interstice 12 May 2019 16:06 UTC

  2 points

  on: “UDT2” and “against UD+ASSA”

  In the mad sci­en­tist ex­am­ple, why would your mea­sure for the die land­ing 0 be 0.91? I think Solomonoff In­duc­tion would as­sign prob­a­bil­ity 0.1 to that out­come, be­cause you need an ex­tra $lo{g}_2\left(90\right)$ bits to spec­ify which clone you are. Or is this just meant to illus­trate a prob­lem with ASSA, UD not in­cluded?

Hal­i­fax Meetup—Board Games

• interstice 25 Mar 2019 20:57 UTC

  1 point

  in reply to: capybaralet's comment on: Pre­dic­tors as Agents

  Yeah, if you train the al­gorithm by ran­dom sam­pling, the effect I de­scribed will take place. The same thing will hap­pen if you use an RL al­gorithm to up­date the pa­ram­e­ters in­stead of an un­su­per­vised learn­ing al­gorithm(though it seems willfully per­verse to do so—you’re throw­ing away a lot of the struc­ture of the prob­lem by do­ing this, so train­ing will be much slower)

  I also just found an old com­ment which makes the ex­act same ar­gu­ment I made here. (Though it now seems to me that ar­gu­ment is not nec­es­sar­ily cor­rect!)

• interstice 4 Mar 2019 23:33 UTC

  4 points

  in reply to: John_Maxwell_IV's comment on: Two Small Ex­per­i­ments on GPT-2

  If you liter­ally ran (a pow­ered-up ver­sion of) GPT-2 on “A brilli­ant solu­tion to the AI al­ign­ment prob­lem is...” you would get the sort of thing an av­er­age in­ter­net user would think of as a brilli­ant solu­tion to the AI al­ign­ment prob­lem. Try­ing to do this more use­fully ba­si­cally leads to Paul’s agenda (which is about try­ing to do imi­ta­tion learn­ing of an im­plicit or­ga­ni­za­tion of hu­mans)

Pre­dic­tors as Agents

• interstice 3 Jan 2019 21:22 UTC

  3 points

  in reply to: jessicata's comment on: Pre­dic­tors as Agents

  Reflec­tive Or­a­cles are a bit of a weird case case be­cause their ‘loss’ is more like a ⁰⁄₁ loss than a log loss, so all of the min­ima are ex­actly the same(If we take a sam­ple of 100000 uni­verses to score them, the differ­ence is merely in­cred­ibly small in­stead of 0). I was be­ing a bit glib refer­enc­ing them in the ar­ti­cle; I had in mind some­thing more like a model pa­ram­e­ter­iz­ing a dis­tri­bu­tion over out­puts, whose only in­fluence on the world is via a ran­dom sam­ple from this dis­tri­bu­tion. I think that such mod­els should in gen­eral have fixed points for similar rea­sons, but am not sure. Re­gard­less, these mod­els will, I be­lieve, favour fixed points whose dis­tri­bu­tions are easy to com­pute(But not fixed points with low en­tropy, that is they will pun­ish log­i­cal un­cer­tainty but not in­trin­sic un­cer­tainy). I’m plan­ning to run some ex­per­i­ments with VAEs and post the re­sults later.

• interstice 3 Jan 2019 3:49 UTC

  3 points

  on: Gen­er­al­is­ing CNNs

  You might be in­ter­ested in Trans­former Net­works, which use a learned pat­tern of at­ten­tion to route data be­tween lay­ers. They’re pretty pop­u­lar and have been used in some im­pres­sive ap­pli­ca­tions like this very con­vinc­ing image-syn­the­sis GAN.

  re: whether this is a good re­search di­rec­tion. The fact that neu­ral net­works are highly com­press­ible is very in­ter­est­ing and I too sus­pect that ex­ploit­ing this fact could lead to more pow­er­ful mod­els. How­ever, if your goal is to in­crease the chance that AI has a pos­i­tive im­pact, then it seems like the rele­vant thing is how quickly our un­der­stand­ing of how to al­ign AI sys­tems pro­gresses, rel­a­tive to our un­der­stand­ing of how to build pow­er­ful AI sys­tems. As de­scribed, this idea sounds like it would be more use­ful for the lat­ter.

• interstice 1 Jan 2019 21:19 UTC

  1 point

  in reply to: jessicata's comment on: Pre­dic­tors as Agents

  Is there a rea­son you think a re­flec­tive or­a­cle (or equiv­a­lent) can’t just be se­lected “ar­bi­trar­ily”, and will likely be se­lected to max­i­mize some score?

  The gra­di­ent de­scent is not be­ing done over the re­flec­tive or­a­cles, it’s be­ing done over some gen­eral com­pu­ta­tional model like a neu­ral net. Any highly-perform­ing solu­tion will nec­es­sar­ily look like a fixed-point-find­ing com­pu­ta­tion of some kind, due to the self-refer­en­tial na­ture of the pre­dic­tions. Then, since this fixed-point-fin­der is *in­ter­nal* to the model, it will be op­ti­mized for log loss just like ev­ery­thing else in the model.

  That is, the global op­ti­miza­tion of the model is dis­tinct from what­ever in­ter­nal op­ti­miza­tion the fixed-point-fin­der uses to choose the re­flec­tive or­a­cle. The global op­ti­miza­tion will fa­vor in­ter­nal op­ti­miz­ers that pro­duce fixed-points with good score. So while fixed-point-fin­ders in gen­eral won’t op­ti­mize for any­thing in par­tic­u­lar, the one this model uses will.

• interstice 31 Dec 2018 22:53 UTC

  3 points

  on: An­nounce­ment: AI al­ign­ment prize round 3 win­ners and next round

  I sub­mit Pre­dic­tors as Agents.

• interstice 30 Sep 2018 15:18 UTC

  1 point

  AF

  in reply to: Charlie Steiner's comment on: Reflec­tive AIXI and Anthropics

  If we as­sume Sleep­ing Beauty has lots of in­for­ma­tion, we might ex­pect that the short­est match­ing pro­gram will look like a simu­la­tion of phys­i­cal law plus a “bridg­ing law” that, given this simu­la­tion, tells you what sym­bols get writ­ten to the tape

  I agree. I still think that the prob­a­bil­ities would be closer to ¹⁄₂, ¹⁄₄, ¹⁄₄. The bridg­ing law could look like this: search over the uni­verse for com­pact en­cod­ings of my mem­o­ries so far, then see what is writ­ten next onto this en­cod­ing. In this case, it would take no more bits to spec­ify wak­ing up on Tues­day, be­cause the mem­o­ries are iden­ti­cal, in the same for­mat, and just slightly later tem­po­rally.

  In a nat­u­ral­ized set­ting, it seems like the tricky part would be get­ting the AIXI on Mon­day to care what hap­pens af­ter it goes to sleep. It ‘knows’ that it’s go­ing to lose con­scious­ness(it can see that its cur­rent mem­ory en­cod­ing is go­ing to be over­writ­ten) so its next pre­dic­tion is un­de­ter­mined by its world-model. There is one pro­gram that will give it the re­ward of its suc­ces­sor then ter­mi­nates, as I de­scribed above, but it’s not clear why the AIXI would favour that hy­poth­e­sis. Maybe if it has been in situ­a­tions in­volv­ing mem­ory-wiping be­fore, or has ob­served other RO-AIXI’s in such situ­a­tions.

• interstice 29 Sep 2018 19:19 UTC

  1 point

  in reply to: Ilya Shpitser's comment on: Deep learn­ing—deeper flaws?

  “I can’t make bets on my be­liefs about the Escha­ton, be­cause they are about the Escha­ton.” -- Well, it makes sense. Be­sides, I did offer you a bet tak­ing into ac­count a) that the money may be worth less in my branch b) I don’t think DL + RL AGI is more likely than not, just plau­si­ble. If you’re more than 96% cer­tain there will be no such AI, 20:1 odds are a good deal.

  But any­ways, I would be fine with bet­ting on a nearer-term challenge. How about—in 5 years, a bipedal robot that can run on rough ter­rain, as in this video, us­ing a policy learned from scratch by DL + RL(pos­si­bly in­clud­ing a simu­lated en­vi­ron­ment dur­ing train­ing) 1:1 odds.

• interstice 28 Sep 2018 17:07 UTC

  1 point

  in reply to: Ilya Shpitser's comment on: Deep learn­ing—deeper flaws?

  Hmmm...but if I win the bet then the world may be de­stroyed, or our en­vi­ron­ment could change so much the money will be­come worth­less. Would you take 20:1 odds that there won’t be DL+RL-based HLAI in 25 years?

• interstice 28 Sep 2018 16:59 UTC

  1 point

  AF

  in reply to: Charlie Steiner's comment on: Reflec­tive AIXI and Anthropics

  I still don’t see how you’re get­ting those prob­a­bil­ities. Say it takes 1 bit to de­scribe the out­come of the coin toss, and as­sume it’s easy to find all the copies of your­self(ie your mem­o­ries) in differ­ent wor­lds. Then you need:

  1 bit to spec­ify if the coin landed heads or tails

  If the coin landed tails, you need 1 more bit to spec­ify if it’s Mon­day or Tues­day.

  So AIXI would give these sce­nar­ios P(HM)=0.50, P(TM)=0.25, P(TT)=0.25.

• interstice 27 Sep 2018 22:08 UTC

  1 point

  in reply to: Ilya Shpitser's comment on: Deep learn­ing—deeper flaws?

  Have some­thing in mind?

• interstice 27 Sep 2018 20:34 UTC

  1 point

  AF

  in reply to: Diffractor's comment on: Reflec­tive AIXI and Anthropics

  Well, it COULD be the case that the K-com­plex­ity of the mem­ory-erased AIXI en­vi­ron­ment is lower, even when it learns that this hap­pened. The rea­son for this is that there could be many pos­si­ble past AIXI’s who have their mem­ory erased/​al­tered and end up in the same sub­jec­tive situ­a­tion. Then the mem­ory-era­sure hy­poth­e­sis can use the low­est K-com­plex­ity AIXI who ends up with these mem­o­ries. As the AIXI learns more it can grad­u­ally piece to­gether which of the po­ten­tial past AIXI’s it ac­tu­ally was and the K-com­plex­ity will go back up again.

  EDIT: Oh, I see you were talk­ing about ac­tu­ally hav­ing a RANDOM mem­ory in the sense of a ran­dom se­quence of 1s and 0s. Yeah, but this is no differ­ent than AIXI think­ing that any ran­dom pro­cess is high K-com­plex­ity. In gen­eral, and dis­count­ing merg­ing, the mem­ory-al­ter­ing sub­rou­tine will in­crease the com­plex­ity of the en­vi­ron­ment by a con­stant plus the com­plex­ity of what­ever trans­for­ma­tion you want to ap­ply to the mem­o­ries.

• interstice 27 Sep 2018 19:49 UTC

  2 points

  in reply to: Ilya Shpitser's comment on: Deep learn­ing—deeper flaws?

  Well, the DotA bot pretty much just used PPO,. AlphaZero used MCTS + RL, OpenAI re­cently got a robot hand to do ob­ject ma­nipu­la­tion with PPO and a simu­la­tor(the simu­la­tor was hand-built, but in prin­ci­ple it could be pro­duced by un­su­per­vised learn­ing like in this). Clearly it’s pos­si­ble to get so­phis­ti­cated be­hav­iors out of pretty sim­ple RL al­gorithms. It could be the case that these ap­proaches will “run out of steam” be­fore get­ting to HLAI, but it’s hard to tell at the mo­ment, be­cause our al­gorithms aren’t run­ning with the same amount of com­pute + data as hu­mans (for hu­mans, I am think­ing of our en­tire life­time ex­pe­riences as data, which is used to build a cross-do­main op­ti­mizer).

  re: Uber, I agree that at least in the short term most ap­pli­ca­tions in the real world will fea­ture a fair amount of en­g­ineer­ing by hand. But the need for this could de­crease as more power be­comes available, as has been the case in su­per­vised learn­ing.

• interstice 27 Sep 2018 16:29 UTC

  1 point

  in reply to: Charlie Steiner's comment on: Boltz­mann Brains and Within-model vs. Between-mod­els Probability

  How do the ini­tial sim­ple con­di­tions re­late to the branch­ing? Our uni­verse seems to have had sim­ple ini­tial con­di­tions but there’s still been a lot of ran­dom branch­ing, right? That is, the uni­verse from our per­spec­tive is just one branch of a quan­tum state evolv­ing sim­ply from sim­ple con­di­tions, so you need O(#branch­ing events) bits to de­scribe it. In­ci­den­tally this un­der­mines Eliezer’s ar­gu­ment for MWI based on Solomonoff in­duc­tion, though MWI is prob­a­bly still true

  [EDITED: Oh, from one of your other com­ments I see that you aren’t say­ing the short­est pro­gram in­volves be­gin­ning at the start of the uni­verse. That makes sense]

• interstice 27 Sep 2018 16:11 UTC

  1 point

  in reply to: Ilya Shpitser's comment on: Deep learn­ing—deeper flaws?

  I agree that you do need some sort of causal struc­ture around the func­tion-fit­ting deep net. The ques­tion is how com­plex this struc­ture needs to be be­fore we can get to HLAI. It seems plau­si­ble to me(at least a 10% chance, say) that it could be quite sim­ple, maybe just con­sist­ing of mod­estly more so­phis­ti­cated ver­sions of the RL al­gorithms we have so far, com­bined with re­ally big deep net­works.

• interstice 27 Sep 2018 15:51 UTC

  1 point

  AF

  in reply to: interstice's comment on: Reflec­tive AIXI and Anthropics

  In­ci­den­tally, you can use the same idea to have RO-AIXI do an­thropic rea­son­ing/​bar­gain­ing about ob­servers that are in a broader refer­ence class than ‘ex­act same sense data’, by mak­ing the map­ping O → O’ some sort of coarse-grain­ing.

• interstice 27 Sep 2018 15:44 UTC

  1 point

  AF

  in reply to: Charlie Steiner's comment on: Reflec­tive AIXI and Anthropics

  ″ P(HM)=0.49, P(TM)=0.49, P(TT)=0.2 ”—Are these sup­posed to be mu­tu­ally ex­clu­sive prob­a­bil­ities?

  ″ There is a tur­ing ma­chine that writes the mem­ory-wiped con­tents to tape all in one pass. ”—Yes, this is ba­si­cally what I said. (‘en­vi­ron­ment’ above could in­clude ‘the world’ + bridg­ing laws). But you also need to al­ter the re­ward struc­ture a bit to make it match our usual in­tu­ition of what ‘mem­ory-wiping’ means, and this has sig­nifi­cance for de­ci­sion the­ory.

  Con­sider, if your own mem­ory was erased, you would prob­a­bly still be con­cerned about what was go­ing to hap­pen to you later. But a reg­u­lar AIXI won’t care about what hap­pens to its mem­ory-wiped clone(i.e. an­other AIXI in­duct­ing on the ‘mem­ory-wiped’ stream), be­cause they don’t share an in­put chan­nel. So to fix this you give the origi­nal AIXI all of the re­wards that its clone ends up get­ting.