Tetraspace Grouping

• Tetraspace Grouping 1 Apr 2020 1:46 UTC

  5 points

  on: Blog Post Day II Retrospective

  I’m in­ter­ested in par­ti­ci­pat­ing in a Blog Post Day III! And I ap­prove of one this month, mostly out of a self-in­ter­ested re­gret that I missed out on Blog Post Day II.

• Tetraspace Grouping 2 Jan 2020 3:18 UTC

  10 points

  in reply to: habryka's comment on: Habryka’s Short­form Feed

  Since this hash is pub­li­cly posted, is there any timescale for when we should check back to see the preimage?

• Tetraspace Grouping 2 Jan 2020 3:08 UTC

  6 points

  on: Te­traspace Group­ing’s Shortform

  Life 3.0 Live­blog/​Re­view Thread

  Prelude

  The pro­logue be­gins with a short story called the Tale of the Omega Team. It’s a wish-fulfil­ment pseudo-isekai about a bunch of effec­tive al­tru­ist tech peo­ple work­ing for not-Google called the Omegas who make an AGI and then use it to take over the world.

  But a cy­ber­se­cu­rity spe­cial­ist on their team talked them out of the game plan [...] risk of Prometheus break­ing out and seiz­ing con­trol of its own des­tiny [...] weren’t sure how its goals would evolve [...] go to great lengths to keep Prometheus confined

  For some rea­son, the Omegas in the story claim that the Prometheus (the AI) might be un­safe, and then pro­ceed to do things like have it write soft­ware which they then run on com­put­ers and let it pro­duce long pieces of an­i­mated me­dia and let it send blueprints of tech­nolo­gies to sci­en­tists. There is a cy­ber­se­cu­rity ex­pert in the team who just barely stops them from straight up leav­ing the whole thing un­boxed, and I do not envy her job po­si­tion.

  (Prometheus is safe, it turns out, which I can tell be­cause there are hu­mans al­ive at the end of the story.)

  [...] Omega-con­trol­led [...] con­trol­led by the Omegas [...] the Omegas har­nessed Prometheus [...] the Omegas’ [...] the Omegas’ [...]

  There’s also an­other odd thing where it says that the Omegas are us­ing Prometheus as a tool to do things, in­stead of what’s clearly ac­tu­ally hap­pen­ing which is that Prometheus is achiev­ing its goals with the Omegas be­ing some lumps of atoms that it’s been push­ing around ac­cord­ing to its whims, as it has been since they de­cided to switch it on.

  All-in-all, I like it. It wouldn’t be out of place on r/​ra­tio­nal, if wish-fulfill­ment pseudo-isekai does hap­pen then AGI sweep­ing aside the pre­vi­ous so­cial or­der will be how (a real AGI would come close to some of the ca­pa­bil­ities I’ve seen those pro­tag­o­nists have), and fic­tion about more plau­si­ble robopoca­lypses (or roboutopias) com­ing about is always great.

• Tetraspace Grouping 24 Dec 2019 22:31 UTC

  1 point

  in reply to: agai's comment on: A Cri­tique of Func­tional De­ci­sion Theory

  The note is just set-dress­ing; you could have both the boxes have glass win­dows that let you see whether or not they con­tain a Bomb for the same con­clu­sions if it throws you off.

• Tetraspace Grouping 23 Dec 2019 23:51 UTC

  9 points

  on: Te­traspace Group­ing’s Shortform

  In the Parable of Pre­dict-O-Matic, a sub­net­work of the titu­lar Pre­dict-O-Matic be­comes a mesa-op­ti­miser and be­gins steer­ing the fu­ture to­wards its own goals, in­de­pen­dently of the rest of Pre­dict-O-Matic. It does so in a way that sab­o­tages the other sub­net­works.

  I am re­minded of one speci­fi­ca­tion prob­lem that a run of Eurisko faced:

  Dur­ing one run, Le­nat no­ticed that the num­ber in the Worth slot of one newly dis­cov­ered heuris­tic kept ris­ing, in­di­cat­ing that Eurisko had made a par­tic­u­larly valuable find. As it turned out the heuris­tic performed no use­ful func­tion. It sim­ply ex­am­ined the pool of new con­cepts, lo­cated those with the high­est Worth val­ues, and in­serted its name in their My Creator slots.

  One thing I won­dered is whether this could hap­pen in hu­mans, and if not, why it doesn’t. A sim­plified de­scrip­tion of mem­ory that I learned in a flash game is that “neu­ral con­nec­tions” are “strength­ened” when­ever they are “used”, which sounds sort of like gra­di­ents in RL if you don’t think about it too hard. Maybe the analogue of this would be some mem­ory that “wants” you to re­mem­ber it re­peat­edly at the ex­pense of other mem­o­ries. Trauma?

• Tetraspace Grouping 23 Dec 2019 22:06 UTC

  2 points

  in reply to: ozziegooen's comment on: ozzie­gooen’s Shortform

  Other things that Tim might mean when he says 20%:

  • Tim is be­ing dishon­est, and be­lieves that the listen­ers will up­date away from the rad­i­cal and low-sta­tus figure of 20% to avoid be­ing as­so­ci­ated with the lowly Tim.

  • Tim be­lieves that other listen­ers will be en­couraged to make their own prob­a­bil­ity es­ti­mates with ex­plicit rea­son­ing in re­sponse, which will make their ex­per­tise more leg­ible to Tim and other listen­ers.

  • Tim wants to show cul­tural alle­giance with the Su­perfore­cast­ing tribe.

• Tetraspace Grouping 22 Dec 2019 12:40 UTC

  2 points

  in reply to: [Error communicating with LW2 server]'s comment on: Should We Still Fly?

  Quick es­ti­mate: Global av­er­age is 4.8 tons per per­son = $50 ad­di­tional per year per life saved = ~$1500 to­tal (over 30 ad­di­tional years of life), so over the course of sav­ing an av­er­age per­son’s life the costs if you’re buy­ing offsets are the same or­der as the costs of sav­ing a life via a Givewell char­ity (~half).

  For the peo­ple helped by Givewell recom­mended char­i­ties, the ad­di­tional CO2 emis­sions are prob­a­bly lower; among the world’s poor­est, <1 tons of CO2 per cap­ita per year is pretty com­mon, which is <$300 over a life­time, about an or­der of mag­ni­tude less than the cost of sav­ing a life.

• Tetraspace Grouping 12 Dec 2019 21:18 UTC

  3 points

  on: Te­traspace Group­ing’s Shortform

  Over the past few days I’ve been read­ing about re­in­force­ment learn­ing, be­cause I un­der­stood how to make a neu­ral net­work, say, recog­nise hand­writ­ten digits, but I wasn’t sure how at all that could be turned into get­ting a com­puter to play Atari games. So: what I’ve learned so far. Spin­ning Up’s In­tro to RL prob­a­bly ex­plains this bet­ter.

  (Brief sum­mary, ex­plained prop­erly be­low: The agent is a neu­ral net­work which runs in an en­vi­ron­ment and re­ceives a re­ward. Each pa­ram­e­ter in the neu­ral net­work is in­creased in pro­por­tion to how much it in­creases the prob­a­bil­ity of mak­ing the agent do what it just did, and how good the out­come of what the agent just did was.)

  Re­in­force­ment learn­ers play in­side a game in­volv­ing an agent and an en­vi­ron­ment. On turn $t$, the en­vi­ron­ment hands the agent an ob­ser­va­tion $o_t$, and the agent hands the en­vi­ron­ment an ac­tion $a_t$. For an agent act­ing in re­al­time, there can be sixty turns a sec­ond; this is fine.

  The en­vi­ron­ment has a tran­si­tion func­tion which takes an ob­ser­va­tion-ac­tion pair $o_t{a}_t$ and re­sponds with a prob­a­bil­ity dis­tri­bu­tion over ob­ser­va­tions on the next timestep $o_{t+1}$; the agent has a policy that takes an ob­ser­va­tion $o_t$ and re­sponds with a prob­a­bil­ity dis­tri­bu­tion over ac­tions to take $a_t$.

  The policy is usu­ally writ­ten as $\pi$, and the prob­a­bil­ity that $\pi$ out­puts an ac­tion $a$ in re­sponse to an ob­ser­va­tion $o$ is $\pi \left(a|o\right)$. In prac­tise, $\pi$ is usu­ally a neu­ral net­work that takes ob­ser­va­tions as in­put and has ac­tions as out­put (us­ing some­thing like a soft­max layer to give a prob­a­bil­ity dis­tri­bu­tion); the pa­ram­e­ters of this neu­ral net­work are $\theta$, and the cor­re­spond­ing policy is ${\pi }_{\theta }$.

  At the end of the game, the en­tire tra­jec­tory $\tau =o_1{a}_1{o}_2{a}_2\dots o_T{a}_T$ is as­signed a score, $R\left(\tau \right)$, mea­sur­ing how well the agent has done. The goal is to find the policy ${\pi }_{\theta }$ that max­imises this score.

  Since we’re us­ing ma­chine learn­ing to max­imise, we should be think­ing of gra­di­ent de­scent, which in­volves find­ing the lo­cal di­rec­tion in which to change the pa­ram­e­ters $\theta$ in or­der to in­crease the ex­pected value of $R$ by the great­est amount, and then in­creas­ing them slightly in that di­rec­tion.

  In other words, we want to find ${\nabla }_{\theta }\underset{\tau \sim {\pi }_{\theta }}{E}\left[R\right(\tau \left)\right]$.

  Writ­ing the ex­pec­ta­tion value in terms of a sum over tra­jec­to­ries, this is ${\nabla }_{\theta }{\sum }_{\tau \in D}\left(P\right(\tau |\theta \left)R\right(\tau \left)\right)$ = ${\sum }_{\tau \in D}\left({\nabla }_{\theta }P\right(\tau |\theta \left)R\right(\tau \left)\right)$, where $P\left(\tau |\theta \right)$ is the prob­a­bil­ity of ob­serv­ing the tra­jec­tory $\tau$ if the agent fol­lows the policy ${\pi }_{\theta }$, and $D$ is the space of pos­si­ble tra­jec­to­ries.

  The prob­a­bil­ity of see­ing a spe­cific tra­jec­tory hap­pen is the product of the prob­a­bil­ities of any in­di­vi­d­ual step on the tra­jec­tory hap­pen­ing, and is hence $P\left(\tau |\theta \right)={\prod }_{t=1}^T{\pi }_{\theta }\left(a_t|o_t\right)E\left(o_t|a_{t-1}{o}_{t-1}\right)$ where $E\left(o_{t+1}|a_t{o}_t\right)$ is the prob­a­bil­ity that the en­vi­ron­ment out­puts the ob­ser­va­tion $o_{t+1}$ in re­sponse to the ob­ser­va­tion-ac­tion pair $a_t{o}_t$ . Prod­ucts are awk­ward to work with, but prod­ucts can be turned into sums by tak­ing the log­a­r­ithm - $\ln P\left(\tau |\theta \right)={\sum }_{t=1}^T\ln {\pi }_{\theta }\left(a_t|o_t\right)+\ln E\left(o_t|a_{t-1}{o}_{t-1}\right)$ .

  The gra­di­ent of this is ${\nabla }_{\theta }\ln P\left(\tau |\theta \right)={\sum }_{t=1}^T{\nabla }_{\theta }\ln {\pi }_{\theta }\left(a_t|o_t\right)+{\nabla }_{\theta }\ln E\left(o_t|a_{t-1}{o}_{t-1}\right)$ . But what the en­vi­ron­ment does is in­de­pen­dent of $\theta$, so that en­tire term van­ishes, and we have ${\nabla }_{\theta }\ln P\left(\tau |\theta \right)={\sum }_{t=1}^T{\nabla }_{\theta }\ln {\pi }_{\theta }\left(a_t|o_t\right)$ . The gra­di­ent of the policy is quite easy to find, since our policy is just a neu­ral net­work so you can use back-prop­a­ga­tion.

  Our ex­pres­sion for the ex­pec­ta­tion value is just in terms of the gra­di­ent of the prob­a­bil­ity, not the gra­di­ent of the log­a­r­ithm of the prob­a­bil­ity, so we’d like to ex­press one in terms of the other.

  Con­ve­niently, the chain rule gives ${\nabla }_{\theta }\ln P\left(\tau |\theta \right)=\frac{1}{P\left(\tau |\theta \right)}{\nabla }_{\theta }P\left(\tau |\theta \right)$ , so ${\nabla }_{\theta }P\left(\tau |\theta \right)=P\left(\tau |\theta \right){\nabla }_{\theta }\ln P\left(\tau |\theta \right)$ . Sub­sti­tut­ing this back into the origi­nal ex­pres­sion for the gra­di­ent gives

  ${\sum }_{\tau \in D}\left(P\right(\tau |\theta \left){\nabla }_{\theta }\ln P\right(\tau |\theta \left)R\right(\tau \left)\right)$ ,

  and sub­sti­tut­ing our ex­pres­sion for the gra­di­ent of the log­a­r­ithm of the prob­a­bil­ity gives

  ${\sum }_{\tau \in D}\left(P\right(\tau |\theta \left){\sum }_{t=1}^T{\nabla }_{\theta }\ln {\pi }_{\theta }\right(a_t|o_t\left)R\right(\tau \left)\right)$ .

  No­tice that this is the defi­ni­tion of the ex­pec­ta­tion value of ${\nabla }_{\theta }\ln {\pi }_{\theta }\left(a_t|o_t\right)R\left(\tau \right)$ , so writ­ing the sum as an ex­pec­ta­tion value again we get

  $\underset{\tau \sim {\pi }_{\theta }}{E}\left[{\sum }_{t=1}^T{\nabla }_{\theta }\log {\pi }_{\theta }\right(a_t|s_t\left)R\right(\tau \left)\right]$.

  You can then find this ex­pec­ta­tion value eas­ily by sam­pling a large num­ber of tra­jec­to­ries (by run­ning the agent in the en­vi­ron­ment many times), calcu­lat­ing the term in­side the brack­ets, and then av­er­ag­ing over all of the runs.

  Neat!

  (More so­phis­ti­cated RL al­gorithms ap­ply var­i­ous trans­for­ma­tions to the re­ward to use in­for­ma­tion more effi­ciently, and use var­i­ous gra­di­ent de­scent tricks to use the gra­di­ents ac­quired to con­verge on the op­ti­mal pa­ram­e­ters more effi­ciently)

• Tetraspace Grouping 1 Oct 2019 10:36 UTC

  1 point

  in reply to: Grue_Slinky's comment on: Grue_Slinky’s Shortform

  Are we al­lowed to I-am-Groot the word “cake” to en­code sev­eral bits per word, or do we have to do some­thing like re­peat “cake” un­til the primes that it fac­tors into rep­re­sent a de­sired bi­nary string?

  (edit: ah, only nouns, so I can still use what­ever I want in the other parts of speech. or should I say that the nam­ing cakes must be “cake”, and that any other ver­bal cake may be what­ever this speak­ing cake wants)

• Tetraspace Grouping 28 Sep 2019 1:09 UTC

  2 points

  in reply to: lionhearted's comment on: Honor­ing Petrov Day on LessWrong, in 2019

  Dank EA Memes is a Face­book group. It’s pretty good.

• Tetraspace Grouping 28 Sep 2019 0:59 UTC

  18 points

  on: Fol­low-Up to Petrov Day, 2019

  If any­one asks, I en­tered a code that I knew was in­cor­rect as a pre­com­mit­ment to not nuke the site.

• Tetraspace Grouping 27 Sep 2019 0:38 UTC

  30 points

  in reply to: jp's comment on: Honor­ing Petrov Day on LessWrong, in 2019

  To make sure I have this right and my LW isn’t glitch­ing: TurnTrout’s com­ment is a Drake meme, and the two other replies in this chain are ac­tu­ally blank?

• Tetraspace Grouping 27 Sep 2019 0:35 UTC

  12 points

  in reply to: Scott Garrabrant's comment on: Honor­ing Petrov Day on LessWrong, in 2019

  Well, at least we have a re­sponse to the doubters’ “why would any­one even press the but­ton in this situ­a­tion?”

• Tetraspace Grouping 26 Sep 2019 23:37 UTC

  17 points

  on: Honor­ing Petrov Day on LessWrong, in 2019

  I.

  Click­ing on the but­ton per­ma­nently switches it to a state where it’s pushed-down, be­low which is a prompt to en­ter launch codes. When moused over, the pushed-down but­ton has the tooltip “You have pressed the but­ton. You can­not un-press it.” Screen­shot.

  (On an un­re­lated note, on r/​the­but­ton I have a pur­ple flair that says “60s”.)

  Upon en­ter­ing a string of longer than 8 char­ac­ters, a but­ton say­ing “launch” ap­pears be­low the big red but­ton. Screen­shot.

  II.

  I’m nowhere near the PST time­zone, so I wouldn’t be able to re­li­ably pull a shenani­gan whereby if I had the launch codes I would en­ter or not en­ter them de­pend­ing on the amount of coun­ter­fac­tual money pledged to the Ploughshares Fund in the name of ei­ther launch-code-en­try-state, but this sen­tence is not apopha­sis.

  III.

  Con­spir­acy the­ory: There are no launch codes. Peo­ple who claim to have launch codes are ly­ing. The real test is whether peo­ple will press the but­ton at all. I have failed that test. I came up with this con­spir­acy the­ory ~250 mil­lisec­onds af­ter press­ing the but­ton.

  IV. (Up­date)

  I can no longer see the but­ton when I am logged in. Could this mean that I have won?

• Tetraspace Grouping 24 Sep 2019 1:44 UTC

  16 points

  on: Novum Or­ganum: Preface

  At the start of the Se­quences, you are told that ra­tio­nal­ity is a mar­tial art, used to am­plify the power of the un­aided mind in the same way that a mar­tial art doesn’t nec­es­sar­ily make you stronger but just lets you use your body prop­erly.

  Ba­con, on the other hand, throws the prospect of us­ing the un­aided mind right out; Ba­co­nian ra­tio­nal­ity is a ma­chine, like a pul­ley or a lever, where you ap­ply your mind how­ever fee­bly to one end and by its con­struc­tion the other end moves a great dis­tance or ap­plies a great force (ei­ther would do for the metaphor).

  If I have my his­tory right, Ba­con’s ma­chine is Science. Its func­tion is to ac­cu­mu­late a huge moun­tain of ev­i­dence, so big that even a hu­man could be per­suaded by it, and in­struc­tion in the use of sci­ence is in­struc­tion in be­ing per­suaded by that moun­tain of ev­i­dence. Philoso­phers of old sim­ply ig­nored the moun­tain of ev­i­dence (failed to use the ma­chine) and maybe re­lied on syl­l­o­gisms and defi­ni­tions and hence failed to move the stone column.

  And later, with the aid of Ba­con’s ma­chine, it turns out that one dis­cov­ers that you don’t re­ally need this huge moun­tain of ev­i­dence or the sys­tem­atic stuff and that an ideal rea­soner could sim­ply perform a Bayesian up­date on each bit that comes in and get to the truth way faster, while avoid­ing all the slow­ness or all the mis­takes that come if you in­sist on set­ting up the ma­chine ev­ery sin­gle time. At your own risk, of course—get your stance slightly wrong lift­ing a stone column, and you throw your back out.

• Tetraspace Grouping 15 Sep 2019 14:43 UTC

  6 points

  on: A Cri­tique of Func­tional De­ci­sion Theory

  An agent also faces a guaran­teed pay­offs prob­lem in Parfit’s hitch­hiker, since the driver has already made their pre­dic­tion (the agent knows they’re safe in the town) so the agent’s choice is be­tween los­ing $1,000 and los­ing $0. Is it also a bad idea for the agent to pay the $1000 in this prob­lem?

• Tetraspace Grouping 10 Sep 2019 15:09 UTC

  1 point

  in reply to: zulupineapple's comment on: ozzie­gooen’s Shortform

  There’s some­thing of a prob­lem with sen­si­tivity; if the x-risk from AI is ~0.1, and the differ­ence in x-risk from some grant is ~10^-6, then any differ­ence in the fore­casts is go­ing to be com­pletely swamped by noise.

  (while peo­ple in the mar­ket could fix any in­con­sis­tency be­tween the pre­dic­tions, they would only be able to look for­ward to 0.001% re­turns over the next cen­tury)

• Tetraspace Grouping 6 Sep 2019 21:43 UTC

  1 point

  in reply to: Matthew Barnett's comment on: Open & Wel­come Thread—Septem­ber 2019

  Is the is­sue that it’s pain-based and hence makes my life worse (prob­a­bly false for me: maths is fun and gives me a sense of pride and ac­com­plish­ment when I do it, it’s just that darn Sys­tem 1 always say­ing “bet­ter for you if you play Ker­bal Space Pro­gram”), or that so­cial pun­ish­ment isn’t always available and there­fore ought not to be re­lied on (this is prob­a­bly an is­sue for me), or some third thing?

• Tetraspace Grouping 4 Sep 2019 20:17 UTC

  6 points

  on: Open & Wel­come Thread—Septem­ber 2019

  Pre­vi­ously: Au­gust.

  Dear Diary,

  In the in­ter­ven­ing month I have done chap­ters 8 and 9 of Tao’s Anal­y­sis I, which feels ter­ribly slow. Two chap­ters in a month? I could do the whole book in that time if I tried! And I know that I can be­cause I have, like I’m get­ting a physics de­gree and it definitely feels like I’ve done at least one text­book worth of learn­ing per term.

  One of the ac­tive in­gre­di­ents seems to be time pres­sure, which is pre­sent but not salient here—if I fail, all that hap­pens is the wrong math is de­ployed to steer the fu­ture of the light­cone, which doesn’t hold a can­dle to me los­ing a lit­tle bit of sta­tus. Ah, to be a brain.

  Thus: by Oc­to­ber I’ll have finished Anal­y­sis I; think less of me if I haven’t.

  (And per­haps I’ll have done even more!)

  UPDATE SEP 26: You can rest easy now; I have com­pleted the book.

• Tetraspace Grouping 30 Aug 2019 15:34 UTC

  1 point

  in reply to: habryka's comment on: I think I came up with a good util­ity func­tion for AI that seems too ob­vi­ous. Can you peo­ple poke holes in it?

  This AI wouldn’t be try­ing to con­vince a hu­man to help it, just that it’s go­ing to suc­ceed.

  So in­stead of con­vinc­ing hu­mans that a hell-world is good, it would con­vince the hu­mans that it was go­ing to cre­ate a hell-world (and they would all dis­ap­prove, so it would score low).

  I think what this ends up do­ing is hav­ing ev­ery­one agree with a world that sounds su­perfi­cially good but is ac­tu­ally ter­rible in a way that’s difficult for un­aided hu­mans to re­al­ize e.g. the AI con­vinces ev­ery­one that it will cre­ate an idyl­lic nat­u­ral world where peo­ple live for­ager lifestyles in har­mony etc. etc., ev­ery­one ap­proves be­cause they like na­ture and har­mony and stuff, it pro­ceeds to cre­ate such an idyl­lic nat­u­ral world, and wild an­i­mal suffer­ing out­weighs hu­man en­joy­ment forever­more.