Pattern

• Pattern 22 Sep 2020 4:34 UTC

  2 points

  on: My (Mis)Ad­ven­tures With Al­gorith­mic Ma­chine Learning

  Nit­pick:

  concatonation

  concatenation

• Pattern 21 Sep 2020 3:48 UTC

  2 points

  on: Why GPT wants to mesa-op­ti­mize & how we might change this

  Note that there’s a risk of mesa-op­ti­miza­tion de­vel­op­ing if looka­head im­proves perfor­mance at any point dur­ing GPT’s train­ing.)

  Why?

• Pattern 17 Sep 2020 19:22 UTC

  2 points

  in reply to: RobinHanson’s comment on: Where Ex­pe­rience Con­fuses Physicists

  Did you ever end­ing up post­ing those quotes?

• Pattern 17 Sep 2020 17:09 UTC

  2 points

  in reply to: Mati_Roy’s comment on: What are ex­am­ples of sim­pler uni­verses that have been de­scribed in or­der to ex­plain a con­cept from our more com­plex uni­verse?

  Was the grid world Con­way’s Game of Life?

• Pattern 17 Sep 2020 16:19 UTC

  2 points

  on: Covid 9/​17: It’s Worse

  A vac­cine will be available in Oc­to­ber if Trump is able to over­ride the CDC and FDA, and make it hap­pen by fiat to help its re­elec­tion chances.

  Its? (The trump ad­minis­tra­tion?)

  I have a strong prefer­ence on out­comes, which read­ers can pre­sum­ably guess – but say­ing it out­right wouldn’t con­vince any­one.

  As a util­i­tar­ian, or as a mat­ter of “val­ues”?

• Pattern 17 Sep 2020 16:08 UTC

  2 points

  in reply to: Chris_Leong’s comment on: The Coun­ter­fac­tual Pri­soner’s Dilemma

  I was point­ing out a typo in the Origi­nal Post. That said, that’s a great sum­mary.

  Per­haps an in­ter­me­di­ate po­si­tion could be cre­ated as fol­lows:

  Given a graph of ‘the tree’ (in­clud­ing the branch you’re on), po­si­tion E is

  ex­pected util­ity over branches

  po­si­tion B is

  you only care about your par­tic­u­lar branch.

  Po­si­tion B seems to care about the fu­ture tree (be­cause it is ahead), but not the past tree. So it has a weight of 1 on the cur­rent node and it’s de­scen­dants, but a weight of 0 on past/​averted nodes, while Po­si­tion E has a weight of 1 on the “root node” (what­ever that is). (Node weights are in­her­ited, with the ex­cep­tion of the dis­con­ti­nu­ity in Po­si­tion B.)

  An in­ter­me­di­ate po­si­tion is plac­ing some non-zero weight on ‘past nodes’, go­ing back along the branch, and up­dat­ing the in­her­ited weights. Aside from a weight of ¹⁄₂ be­ing placed along all in branch nodes, an­other se­ries could be used, for ex­am­ple: r, r^2, r^3, … for 0<r<1. (This se­ries might al­low for adopt­ing an ‘in­ter­me­di­ate po­si­tion’ even when the branch his­tory is in­finitely long.)

  There’s prob­a­bly some tech­ni­cal de­tails to work out, like mak­ing all the weights add up to 1, but for a con­ver­gent se­ries that’s prob­a­bly just a mat­ter of ap­ply­ing an ap­pro­pri­ate scale fac­tor for nor­mal­iza­tion. For r=1/​2, the in­finite sum is 1, so no ad­di­tional scal­ing is re­quired. How­ever this might not work (the sum across all node’s re­wards times their weight might di­verge) on an in­finite tree where the re­wards grow too fast...

  (This was an at­tempt at out­lin­ing an in­ter­me­di­ate po­si­tion, but it wasn’t an ar­gu­ment for it.)

• Pattern 16 Sep 2020 20:23 UTC

  2 points

  on: The Coun­ter­fac­tual Pri­soner’s Dilemma

  So [why] do we care about what would have hap­pened if we had?

• Pattern 15 Sep 2020 16:32 UTC

  2 points

  on: Free Money at Pre­dic­tIt: 2020 Gen­eral Election

  Thus, given how crazy this mar­ket could get later, and given I already tied up my funds, I’m [not] go­ing to take the ar­bi­trage here, at least not yet. I might take it later, but for now I want to re­serve the right to make a bet­ter play.

  Thanks for writ­ing this. How pre­dic­tion mar­kets work in prac­tice is in­ter­est­ing.

• Pattern 11 Sep 2020 17:23 UTC

  2 points

  in reply to: Ben Pace’s comment on: Rad­i­cal Probabilism

  • I do not un­der­stand how Jeffrey up­dates lead to path de­pen­dence. Is the trick that my prob­a­bil­ities can change with­out ev­i­dence, there­fore I can just up­date B with­out ob­serv­ing any­thing that also up­dates A, and then use that for ho­cus pocus? Writ­ing that out, I think that’s prob­a­bly it, but as I was read­ing the es­say I wasn’t sure which bit was where the key step was hap­pen­ing.

  TL:DR;

  Based on Rad­i­cal Prob­a­bil­ism and Bayesian Con­di­tion­ing (page 4 and page 5), the path de­pends on the or­der ev­i­dence is re­ceived in, but the des­ti­na­tion does not.

  From the text it­self:

  The “is­sue” is men­tioned:

  An at­trac­tive fea­ture of Jeffrey’s kine­mat­ics is that it al­lows one to be a fal­li­bil­ist about ev­i­dence and yet still make use of it. An ap­par­ent sight­ing of one’s friend across the street, for in­stance, can be re­vised sub­se­quently when you are told that he is out of the coun­try. A closely re­lated fea­ture is the or­der-de­pen­dence of Jeffrey con­di­tion­ing: con­di­tion­ing on a par­tic­u­lar re­dis­tri­bu­tion of prob­a­bil­ity over a par­ti­tion {Ai} and then on a re­dis­tri­bu­tion of prob­a­bil­ity over an­other par­ti­tion {Bi} will not in gen­eral yield the same pos­te­rior prob­a­bil­ity as con­di­tion­ing first on the re­dis­tri­bu­tion over {Bi} and 4 See How­son [8] for a full de­vel­op­ment of this point. A Bayesian might how­ever take this as an ar­gu­ment against full be­lief in any con­tin­gent propo­si­tion. 4 then on that over {Ai}. This prop­erty, in con­trast to the first, has been a mat­ter of con­cern rather than ad­mira­tion; a con­cern for the most part based on a con­fu­sion be­tween the ex­pe­rience or ev­i­dence and its effect on the mind of the agent.5

  And ex­plained:

  Sup­pose, for in­stance, that I ex­pect an es­say from a stu­dent. I ar­rive at work to find an un­named es­say in my pi­geon­hole with fa­mil­iar writ­ing. I am 90% sure that it is from the stu­dent in ques­tion. But then I find that he left me a mes­sage the day be­fore say­ing that he thinks that he may well not be able to bring me the es­say in the next cou­ple of days. In the light of all that I have learnt, I now lower to 30% my prob­a­bil­ity that the es­say was from him. Sup­pose now I got the mes­sage be­fore the es­say. The fi­nal out­come should be the same, but I will get there a differ­ent way: per­haps by my prob­a­bil­ities for the es­say com­ing from him ini­tially go­ing to 10% and then ris­ing to 30% on find­ing the es­say. The im­por­tant thing is this re­ver­sal of the or­der of ex­pe­rience does not pro­duce a re­ver­sal of the or­der of the prob­a­bil­ities: I do not think it 30% likely that I will get the es­say af­ter hear­ing the mes­sage and then re­vise it to 90% af­ter check­ing my pi­geon­hole. The same ex­pe­riences have differ­ent effects on my prob­a­bil­ities de­pend­ing on the or­der in which they oc­cur. (This is, of course, just a par­tic­u­lar ap­pli­ca­tion of the rule that my pos­te­ri­ors de­pend both on the pri­ors and the in­puts).

• Pattern 11 Sep 2020 2:12 UTC

  2 points

  in reply to: adamzerner’s comment on: Up­dates Thread

  What games changed your mind?

• Pattern 3 Sep 2020 20:35 UTC

  2 points

  on: [AN #115]: AI safety re­search prob­lems in the AI-GA framework

  De­ci­sion Points in AI Governance

  ...

  (Th­ese ac­tions should not have been pre­de­ter­mined by ex­ist­ing law and prac­tice.)

  Should not have been, or should not be?

• Pattern 3 Sep 2020 3:02 UTC

  2 points

  in reply to: mingyuan’s comment on: If there were an in­ter­ac­tive soft­ware teach­ing Yud­kowskian ra­tio­nal­ity, what con­cepts would you want to see it teach?

  What’s a MOOC? (And do you have any good/​rep­re­sen­ta­tive ex­am­ples?)

• Pattern 1 Sep 2020 16:41 UTC

  2 points

  on: Ba­sic In­framea­sure Theory

  (Re­for­mat­ted La­tex so the com­ment text ed­i­tor won’t re­ject it.)

  If you have a point $M\in B$, and some other point $M^{\ast }$ that’s an sa-mea­sure, we might as well add $M+M^{\ast }$ to $B$. Why? Well, given some pos­i­tive func­tional $f^{+}$ (and ev­ery­thing we’re query­ing our set $B$ with is a pos­i­tive func­tional by Propo­si­tion 1,

  $f^{\ast }(M+M^{\ast })=f^{\ast }\left(M\right)+f^{\ast }\left(M^{\ast }\right)\ge f^{\ast }\left(M\right)$

  There’s a miss­ing end paren­the­sis (to match the open­ing paren­the­sis on line 2), al­though it’s not com­pletely clear where it goes:

  • re­plac­ing “Propo­si­tion 1,” with “Propo­si­tion 1.” or

  • at the end of the equa­tion (at the end of the quote).

  (Maybe there’s also some­thing else go­ing on with the para­graph af­ter the quote, which con­tinues the sen­tence.)

  “add all the points you pos­si­bly can that don’t af­fect the ${inf}_{(m,b)\in B}\left(m\right(f)+b)$ value for any $f$“[.]

  The set of min­i­mal points is de­noted $B^{min}$[.]

  Look­ing back at our sec­ond desider­a­tum, it says “Our no­tion of a hy­poth­e­sis in this set­ting should col­lapse “se­cretly equiv­a­lent” sets, such that any two dis­tinct hy­pothe­ses be­have differ­ently in some rele­vant as­pect. This will re­quire for­mal­iz­ing what it means for two sets to be “mean­ingfully differ­ent”, find­ing a canon­i­cal form for an equiv­alence class of sets that “be­have the same in all rele­vant ways”, and then prov­ing some the­o­rem that says we got ev­ery­thing.”

  [Em­pha­sis added.]

  Some way of notic­ing that first ” has been opened and isn’t closed un­til the end of the para­graph might make the quotes sec­tion eas­ier to parse—at the risk of mak­ing a sec­tion stick out due to com­plex­ity rather than be­cause you want to em­pha­size it a lot.

• Pattern 1 Sep 2020 14:57 UTC

  2 points

  on: In­tro­duc­tion To The In­fra-Bayesi­anism Sequence

  The REVERSE HEADS En­vi­ron­ment always you 0.5 re­ward if the coin comes up tails, but [if] it comes up heads, say­ing “tails” gets you 1 re­ward and “heads” gets you 0 re­ward. We have Knigh­tian un­cer­tainty be­tween the two en­vi­ron­ments.

  In the next post, (#2)

  The post af­ter that (#3)

  2 more posts to look for­ward to.

  Later posts (not writ­ten yet) will be about the “1 re­ward for­ever” var­i­ant of Nir­vana and In­fraPOMDP’s (~#4), de­vel­op­ing in­framea­sure the­ory more(~#5), ap­pli­ca­tions to var­i­ous ar­eas of al­ign­ment re­search(~#6), the in­ter­nal logic which in­fradis­tri­bu­tions are mod­els of (~#7), un­re­al­iz­able ban­dits (~#8), game the­ory (~#9), at­tempt­ing to ap­ply this to other ar­eas of al­ign­ment re­search (~#10), and… look, we’ve got a lot of ar­eas to work on, alright? (*)

  Plus a spec­u­la­tive/​pos­si­ble 7 more af­ter that as­sum­ing no over­lap or multi-post top­ics. (~#6 and ~#10 already be­ing counted as 2 posts.)

  *More lean­ing on the un­enu­mer­ated pos­si­bil­ities.

  I look for­ward to see­ing more of this!

• Pattern 31 Aug 2020 1:00 UTC

  4 points

  on: [AN #114]: The­ory-in­spired safety solu­tions for pow­er­ful Bayesian RL agents

  Thus, to cover as much [of] the liter­a­ture as possible

• Pattern 28 Aug 2020 18:08 UTC

  5 points

  on: Covid 8/​27: The Fall of the CDC

  It’s a state­ment one makes when one is do­ing word ma­nipu­la­tions. Where one sees a 35% on a piece of pa­per, doesn’t no­tice at all that this is from 11.9% to 8.7%, and de­cides that means that 35% of all pa­tients will go from dy­ing to not dy­ing rather than 3.2%.

  Where does the 35% come from?

  A re­duc­tion from 11.9% to 8.7% means a de­crease of 3.2%. As a frac­tion of the 11.9%, 3.2% is about 27% of the 11.9, not 35%.

• Pattern 21 Aug 2020 20:32 UTC

  3 points

  on: Be­com­ing an EA-in­fluencer on YouTube etc.

  Given the lack of re­sponses so far, per­haps this would get a bet­ter re­cep­tion at the EA fo­rum?

• Pattern 21 Aug 2020 3:28 UTC

  2 points

  on: On Sys­tems—Liv­ing a life of zero willpower

  • Poli­cies—eg what [?] do when I am too tired to fo­cus while working

• Pattern 20 Aug 2020 17:31 UTC

  10 points

  on: Rad­i­cal Probabilism

  This link doesn’t seem to work:

  I was a Teenage Log­i­cal Pos­i­tivist (Now a Sep­tu­a­ge­nar­ian Rad­i­cal Prob­a­bil­ist), Richard Jeffrey.

  I haven’t checked the oth­ers.

• Pattern 19 Aug 2020 16:55 UTC

  LW: 2 AF: 1

  AF

  on: Look­ing for ad­ver­sar­ial col­lab­o­ra­tors to test our De­bate protocol

  If you would be in­ter­ested in par­ti­ci­pat­ing con­di­tional on us offer­ing pay or prizes, that’s also use­ful to know.

  Do you want this feed­back at the same ad­dress?