orthonormal

• orthonormal 15 Feb 2019 22:29 UTC

  17 points

  on: How does OpenAI’s lan­guage model af­fect our AI timeline es­ti­mates?

  It doesn’t move much prob­a­bil­ity mass to the very near term (i.e. 1 year or less), be­cause both this and AlphaS­tar aren’t re­ally do­ing con­se­quen­tial­ist rea­son­ing, they’re just able to get a sur­pris­ing perfor­mance with sim­pler tricks (the very Marko­vian na­ture of hu­man writ­ing, a good po­si­tion eval­u­a­tion func­tion) given a whole lot of com­pute.
  

  How­ever, it does shift my prob­a­bil­ities for­ward in time, in the sense that one new weird trick to do de­duc­tive or con­se­quen­tial­ist rea­son­ing, plus a lot of com­pute, might get you there re­ally quickly.

• orthonormal 5 Oct 2018 18:56 UTC

  53 points

  on: The Rocket Align­ment Problem

  I ex­pect that some oth­er­wise con­vince­able read­ers are not go­ing to re­al­ize that in this fic­tional world, peo­ple haven’t dis­cov­ered New­ton’s physics or calcu­lus, and those read­ers are there­fore go­ing to miss the anal­ogy of “this is how MIRI would talk about the situ­a­tion if they didn’t already know the fun­da­men­tal con­cepts but had rea­sons for search­ing in the right di­rec­tion”. (I’m not think­ing of read­ers in­ca­pable of han­dling that coun­ter­fac­tual, but of read­ers who aren’t great at in­fer­ring im­plicit back­ground facts from a writ­ten di­alogue. Such read­ers might get very con­fused at the un­ex­pected turns of the di­alogue and quit rather than figure out what they’re baf­fled by.)

  I’d sug­gest adding to the pream­ble some­thing like “In a weird world where peo­ple had figured out work­able aero­nau­tics and ba­sic rocket propul­sion by trial and er­ror, but hadn’t figured out New­ton’s laws or calcu­lus”.

• orthonormal 7 May 2018 20:51 UTC

  9 points

  in reply to: Jason Gross's comment on: Fuzzy Boundaries, Real Concepts

  I like your defi­ni­tion, though, and want to try to make a bet­ter one (and I ac­knowl­edge this is not the point of this post).

  I think that’s a perfectly valid thing to do in the com­ments here! How­ever, I think your at­tempt,

  My stab at a re­fine­ment of “con­sent” is “re­spect for an­other’s choices”, where “dis­re­spect” is “de­liber­ately(?) do­ing some­thing to un­der­mine”

  is far too vague to be a use­ful con­cept.

  In most re­al­is­tic cases, I can give a definite an­swer to whether A touched B in a way B clearly did not want to be touched. In the case of my hon­esty defi­ni­tion, it does in­volve in­tent and so I can only in­fer statis­ti­cally when some­one else is be­ing dishon­est vs mis­taken, but for my­self I usu­ally have an an­swer about whether say­ing X to per­son C would be hon­est or not.

  I don’t think I could do the same for your defi­ni­tion; “am I re­spect­ing their choices” is a tough query to bot­tom out in ba­sic facts.

Fuzzy Boundaries, Real Concepts

• orthonormal 6 May 2018 21:21 UTC

  12 points

  in reply to: Wei_Dai's comment on: Lo­cal Val­idity as a Key to San­ity and Civilization

  My com­ment was meant to ex­plain what I un­der­stood Eliezer to be say­ing, be­cause I think you had mis­in­ter­preted that. The OP is sim­ply say­ing “don’t give weight to ar­gu­ments that are lo­cally in­valid, re­gard­less of what else you like about them”. Of course you need to use pri­ors, heuris­tics, and in­tu­itions in ar­eas where you can’t find an ar­gu­ment that car­ries you from be­gin­ning to end. But be­ing able to think “oh, if I move there, then they can take my queen, and I don’t see any­thing else good about that po­si­tion, so let’s not do that then” is a fair bit eas­ier than prov­ing your move op­ti­mal.

• orthonormal 22 Apr 2018 3:14 UTC

  21 points

  in reply to: Wei_Dai's comment on: Lo­cal Val­idity as a Key to San­ity and Civilization

  Rely­ing purely on lo­cal val­idity won’t get you very far in play­ing chess

  The equiv­a­lent of lo­cal val­idity is just me­chan­i­cally check­ing “okay, if I make this move, then they can make that move” for a bunch of cases. Which, first, is a ma­jor de­vel­op­men­tal mile­stone for kids learn­ing chess. So we only think it “won’t get you very far” be­cause all the high-level hu­man play ex­plic­itly or im­plic­itly takes it for granted.

  And sec­ondly, it’s pretty analo­gous to do­ing math; prov­ing the­o­rems is based on the abil­ity to check the lo­cal val­idity of each step, but math­e­mat­i­ci­ans aren’t just brute-forc­ing their way to proofs. They have to de­velop higher-level heuris­tics, some of which are re­ally hard to ex­press in lan­guage, to sug­gest av­enues, and then check lo­cal val­idity once they have a skele­ton of some part of the ar­gu­ment. But if math­e­mat­i­ci­ans stopped do­ing that an­noy­ing bit, well, then af­ter a while you’ll end up with an­other crisis of anal­y­sis when the brilli­ant in­tu­itions are miss­ing some tiny in­gre­di­ent.

  Lo­cal val­idity is an in­cred­ibly im­por­tant part of any sci­en­tific dis­ci­pline; the fact that it’s not a part of most poli­ti­cal dis­course is merely a re­flec­tion that our so­ciety is at about the de­vel­op­men­tal level of a seven-year-old when it comes to poli­ti­cal rea­son­ing.

• orthonormal 4 Apr 2018 5:16 UTC

  4 points

  on: Non-Ad­ver­sar­ial Good­hart and AI Risks

  Bro­ken link on the text “real kil­ling of birds to re­duce pests in China has never been tried”.

• orthonormal 4 Apr 2018 4:52 UTC

  4 points

  on: The Costly Co­or­di­na­tion Mechanism of Com­mon Knowledge

  Much of this ma­te­rial is cov­ered very similarly in Melt­ing Asphalt, es­pe­cially the posts Ads Don’t Work That Way and Doesn’t Mat­ter, Warm Fuzzies.

• orthonormal 4 Apr 2018 4:27 UTC

  14 points

  on: LessWrong Di­as­pora Jar­gon Survey

  If you do fu­ture sur­veys of this sort, I’d like you to ask peo­ple for their prob­a­bil­ities rather than just their best guesses. If peo­ple are un­cer­tain but de­cently cal­ibrated, I’d ar­gue there’s not much of a prob­lem; if peo­ple are con­fi­dently wrong, I’d ar­gue there’s a real prob­lem.

• orthonormal 4 Apr 2018 0:40 UTC

  10 points

  in reply to: Marcello's comment on: The Mean­ing of Right

  This com­ment got linked a decade later, and so I thought it’s worth stat­ing my own thoughts on the ques­tion:

  We can con­sider a refer­ence class of CEV-seek­ing pro­ce­dures; one (mas­sively-un­der­speci­fied, but that’s not the point) ex­am­ple is “em­u­late 1000 copies of Paul Chris­ti­ano liv­ing to­gether com­fortably and im­mor­tally and dis­cussing what the AI should do with the phys­i­cal uni­verse; once there’s a large su­per­ma­jor­ity in fa­vor of an en­actable plan (which can in­clude fur­ther such del­e­gated de­ci­sions), the AI does that”.

  I agree that this is go­ing to be chaotic, in the sense that even slightly differ­ent el­e­ments of this refer­ence class might end up steer­ing the AI to differ­ent bas­ins of at­trac­tion.

  I as­sert, how­ever, that I’d con­sider it a pretty good out­come over­all if the fu­ture of the world were de­ter­mined by a gen­uinely ran­dom draw from this refer­ence class, hon­estly in­stan­ti­ated. (Again with the mas­sive un­der­speci­fi­ca­tion, I know.)

  CEV may be un­der­de­ter­mined and many-val­ued, but that doesn’t mean pa­per­clip­ping is as good an an­swer as any.

  Re: no bas­ins, it would be a bad situ­a­tion in­deed if the vast ma­jor­ity of the refer­ence class never ended up out­putting an ac­tion plan, in­stead defer­ring and del­e­gat­ing for­ever. I don’t have cached thoughts about that.

• orthonormal 1 Apr 2018 20:41 UTC

  32 points

  on: An­nounc­ing: Karma 2.0

  I for one wel­come our new ty­po­graph­i­cal over­lords.

• orthonormal 31 Mar 2018 22:47 UTC

  10 points

  in reply to: Kaj_Sotala's comment on: A model I use when mak­ing plans to re­duce AI x-risk

  That’s a le­git thing to be frus­trated by, but I think you know the rea­son why AI safety re­searchers don’t want “we don’t see a way to get to a good out­come ex­cept for an al­igned pro­ject to grab a de­ci­sive strate­gic ad­van­tage” to filter into pub­lic dis­course: it pat­tern-matches too well to “trust us, you need to let us run the uni­verse”.

• orthonormal 31 Mar 2018 19:51 UTC

  10 points

  in reply to: Kaj_Sotala's comment on: A model I use when mak­ing plans to re­duce AI x-risk

  To be clear, I am mak­ing the claim that, of the peo­ple who have made use­ful ad­vances on Or­a­cle AI safety re­search (Arm­strong counts here; I don’t think Yam­polskiy does), all of them be­lieve that the goal of hav­ing a safe Or­a­cle AI is to achieve a de­ci­sive strate­gic ad­van­tage quickly and get to an al­igned fu­ture. I rec­og­nize that this is a hard claim to eval­u­ate (e.g. be­cause this isn’t a state­ment one could put in a Se­ri­ous Aca­demic Jour­nal Ar­ti­cle in the 2010s, it would have to be dis­cussed on their blog or in pri­vate cor­re­spon­dence), but if any­one has a clear coun­terex­am­ple, I’d be in­ter­ested in see­ing it.

• orthonormal 31 Mar 2018 19:36 UTC

  9 points

  in reply to: Zvi's comment on: Circling

  Yes, this. NVC should be treated with a similar sort of pa­ram­e­ters to Crocker’s Rules, which you can de­clare for your­self at any time, you can in­vite peo­ple to a con­ver­sa­tion where it’s known that ev­ery­one will be us­ing them, but you can­not hold it against any­one if you in­vite them to de­clare Crocker’s Rules and they re­fuse.

• orthonormal 31 Mar 2018 17:39 UTC

  10 points

  in reply to: c0rw1n's comment on: The abrupt­ness of nu­clear weapons

  There’s a lot of Ac­tu­ally Bad things an AI can do just by mak­ing elec­trons move.

• orthonormal 31 Mar 2018 16:35 UTC

  10 points

  in reply to: gsastry's comment on: A model I use when mak­ing plans to re­duce AI x-risk

  I’d be in­ter­ested in a list of well-man­aged gov­ern­ment sci­ence and en­g­ineer­ing pro­jects if one ex­ists. The Man­hat­tan Pro­ject and the Apollo Pro­ject both be­long on that list (de­spite both hav­ing their flaws- leaks to the USSR from the former, and the Apollo 1 dis­aster from the lat­ter); what are other ex­am­ples?

• orthonormal 31 Mar 2018 16:32 UTC

  2 points

  in reply to: Kaj_Sotala's comment on: A model I use when mak­ing plans to re­duce AI x-risk

  I’m pretty sure that, with­out ex­cep­tion, any­one who’s made a use­ful con­tri­bu­tion on Or­a­cle AI rec­og­nizes that “let sev­eral or­ga­ni­za­tions have an Or­a­cle AI for a sig­nifi­cant amount of time” is a world-end­ing failure, and that their work is in­stead progress on ques­tions like “if you can have the only Or­a­cle AI for six months, can you save the world rather than end it?”

  Cor­rect me if I’m wrong.

• orthonormal 7 Jan 2018 2:36 UTC

  4 points

  in reply to: Raemon's comment on: Role­play­ing As Yourself

  Thanks!

Role­play­ing As Yourself

• orthonormal 5 Jan 2018 2:10 UTC

  3 points

  in reply to: zulupineapple's comment on: The Loud­est Alarm Is Prob­a­bly False

  I agree there are bro­ken alarms that are quiet (in­clud­ing those that are bro­ken in the di­rec­tion of failing to go off, which leads to a blind spot of oblivi­ous­ness!), and that there are peo­ple stuck in situ­a­tions where there is a cor­rect loud alarm that hap­pens most of the time.

  I said that habits are eas­ier to change than alarms, not that they’re easy in an ab­solute sense.