riceissa

• riceissa 1 Jul 2019 5:26 UTC

  7 points

  on: What’s the most “stuck” you’ve been with an ar­gu­ment, that even­tu­ally got re­solved?

  “Sam Har­ris and the Is–Ought Gap” might be one ex­am­ple.

• riceissa 29 Jun 2019 3:34 UTC

  3 points

  on: GreaterWrong Ar­bital Viewer

  Some of the math seems to be gar­bled, e.g. this page (com­pare with the ver­sion on obor­mot.net) and this page (com­pare with the ver­sion on obor­mot.net).

• riceissa 7 Jun 2019 3:20 UTC

  10 points

  on: Ar­bital scrape

  The scrape seems to be miss­ing the Solomonoff in­duc­tion di­alogue, which was available at https://​​ar­bital.com/​​p/​​solomonoff_in­duc­tion/​​?l=1hh (at the mo­ment I just get an er­ror).

  ETA (2019-06-28): The new ver­sion of the scrape has this page, and can be viewed on the GreaterWrong ver­sion.

• riceissa 28 May 2019 4:31 UTC

  15 points

  on: Was CFAR always in­tended to be a dis­tinct or­ga­ni­za­tion from MIRI?

  Go­ing by pub­lic doc­u­ments, it seems like the in­ten­tion was to have a sep­a­rate or­ga­ni­za­tion pretty early on (al­though I can’t say about the very be­gin­ning since I wasn’t in­volved).

  The Ar­ti­cles of In­cor­po­ra­tion for CFAR (called Feyn­man Foun­da­tion at the time) are dated July 18, 2011, and 501(c)(3) sta­tus was ap­proved on July 26, 2011.

  MIRI’s 2011 strate­gic plan (up­dated Au­gust 2011; un­clear when it was first writ­ten) says “En­courage a new or­ga­ni­za­tion to be­gin ra­tio­nal­ity in­struc­tion similar to what Sin­gu­lar­ity In­sti­tute did in 2011 with Ra­tion­al­ity Mini­camp and Ra­tion­al­ity Boot Camp.”

  The Mini­camp took place in May–June 2011, and it seems like the Boot Camp took place in June–Au­gust 2011, so it looks like by the time the first work­shop finished and the sec­ond was in progress, the plan was already to start the new or­ga­ni­za­tion. How­ever, it’s still pos­si­ble that there was no plan for a sep­a­rate or­ga­ni­za­tion be­fore or dur­ing the first work­shop.

  MIRI’s De­cem­ber 2011 progress re­port also talks about plans to cre­ate the “Ra­tion­al­ity Org”.

  I also wrote a timeline of CFAR a while back, which has more links.

• riceissa 20 May 2019 9:12 UTC

  2 points

  in reply to: Said Achmiz's comment on: “One Man’s Mo­dus Po­nens Is Another Man’s Mo­dus Tol­lens”

  The ex­am­ples in the buck­ets er­ror post have “modus de­lens” as the cor­rect re­sponse. To take the diet ex­am­ple from the post, A = “diet worth be­ing on”, B = “zero stud­ies sug­gest­ing health risks”. Adam has $A⟹B$ stored in his brain, and Betty pre­sents $\neg B$, so Adam’s brain com­putes $\neg A$. The “pro­tect­ing episte­mol­ogy” move is to in­stead adamantly be­lieve A (“I need to stay mo­ti­vated!”) which ends up re­ject­ing what Betty said. But the de­sired re­sponse is to in­stead deny B but also ac­cept A, and hence to deny the im­pli­ca­tion $A⟹B$.

  So in these buck­ets er­ror ex­am­ples, modus po­nens cor­re­sponds to the “au­to­matic” rea­son­ing, modus tol­lens cor­re­sponds to the “flinch­ing away from the truth” move, and modus de­lens cor­re­sponds to the “ra­tio­nal” move that avoids the buck­ets er­ror.

  I ex­plained this more in a com­ment on the post.

  I can’t com­ment as to the rel­a­tive fre­quency of this re­sponse and how of­ten it’s cor­rect (this sort of ques­tion seems difficult to an­swer).

• riceissa 19 May 2019 15:44 UTC

  5 points

  in reply to: Zvi's comment on: An­nounce­ment: AI al­ign­ment prize round 4 winners

  Was this post ever pub­lished? (Look­ing at Zvi’s posts since Jan­uary, I don’t see any­thing that looks like it.)

• riceissa 14 May 2019 20:04 UTC

  15 points

  on: Flash­cards for AI Safety?

  I’ve made around 250 Anki cards about AI safety. I haven’t pri­ori­tized shar­ing my cards be­cause I think find­ing a spe­cific card use­ful re­quires some­one to have read the source ma­te­rial gen­er­at­ing the card (e.g. if I made the card based on a blog post, one would need to read that ex­act blog post to get value out of re­view­ing the card; see learn be­fore you mem­o­rize). Since there are many AI safety blog posts and I don’t have the sense that lots of Anki users read any par­tic­u­lar blog post, it seems to me that the value gen­er­ated from shar­ing a set of cards about a blog post isn’t high enough to over­come the an­noy­ance cost of pol­ish­ing, pack­ag­ing, and up­load­ing the cards.

  More gen­er­ally, from a con­sumer per­spec­tive, I think peo­ple tend to be pretty bad at mak­ing good Anki cards (I’m of­ten em­bar­rassed at the cards I’ve cre­ated sev­eral months ago!), which makes it un­ex­cit­ing for me to spend a lot of effort try­ing to col­lab­o­rate with oth­ers on mak­ing cards (be­cause I ex­pect to re­ceive poorly-made cards in re­turn for the cards I provide). I think col­lab­o­ra­tive card-mak­ing can be done though, e.g. Michael Niel­sen and Andy Ma­tuschak’s quan­tum com­put­ing guide comes with pre-made cards that I think are pretty good.

  Differ­ent peo­ple also have differ­ent goals/​in­ter­ests so even given a sin­gle source ma­te­rial, the speci­fics one wants to Ankify can be differ­ent. For ex­am­ple, some­one who wants to un­der­stand the tech­ni­cal de­tails of log­i­cal in­duc­tion will want to Ankify the com­mon ob­jects used (mar­ket, pric­ing, trader, val­u­a­tion fea­ture, etc.), the the­o­rems and proof tech­niques, and so forth, whereas some­one who just wants a high-level overview and the “so what” of log­i­cal in­duc­tion can get away with Ankify­ing much less de­tail.

  Some­thing I’ve no­ticed is that many AI safety posts aren’t very good at ex­plain­ing things (not enough con­crete ex­am­ples, not enough em­pha­sis on com­mon mis­con­cep­tions and edge cases, not enough effort to an­swer what I think of as “ob­vi­ous” ques­tions); this fact is of­ten re­vealed by the com­ments peo­ple make in re­sponse to a post. This makes it hard to make Anki cards be­cause one doesn’t re­ally un­der­stand the con­tent of the post, at least not well enough to con­fi­dently gen­er­ate Anki cards (one of the benefits of be­ing an Anki user is hav­ing a greater sen­si­tivity to when one does not un­der­stand some­thing; see “illu­sion of ex­plana­tory depth” and re­lated terms). There are other prob­lems like con­flict­ing us­age of ter­minol­ogy (e.g. mul­ti­ple defi­ni­tions of “be­nign”, “al­igned”, “cor­rigible”) and the fact that some of the de­bates are on­go­ing/​some of the knowl­edge is still be­ing worked out.

  For “What would be a good strat­egy for gen­er­at­ing use­ful flash­cards?”: I try to read a post or a se­ries of posts and once I feel that I un­der­stand the ba­sic idea, I will usu­ally reread it to add cards about the ba­sic terms and ask my­self sim­ple ques­tions. Some ex­am­ple cards for iter­ated am­plifi­ca­tion:

  • what kind of train­ing does the Distill step use?

  • in the pseu­docode, what step gets re­peated/​iter­ated?

  • how do we get A[0]?

  • write A[1] in terms of H and A[0]

  • when Paul says IDA is go­ing to be com­pet­i­tive with tra­di­tional RL agents in terms of time and re­source costs, what ex­actly does he mean?

  • ad­van­tages of A[0] over H

  • sym­bolic ex­pres­sion for the overseer

  • why should the am­plified sys­tem (of hu­man + mul­ti­ple copies of the AI) be ex­pected to perform bet­ter than the hu­man alone?

• riceissa 5 May 2019 20:18 UTC

  18 points

  on: How much do ma­jor foun­da­tions grant per hour of staff time?

  You might have in mind a Face­book post that Vipul Naik wrote in 2017.

• riceissa 25 Apr 2019 14:33 UTC

  15 points

  on: When is ra­tio­nal­ity use­ful?

  We can model suc­cess as a com­bi­na­tion of do­ing use­ful things and avoid­ing mak­ing mis­takes. As a par­tic­u­lar ex­am­ple, we can model in­tel­lec­tual suc­cess as a com­bi­na­tion of com­ing up with good ideas and avoid­ing bad ideas. I claim that ra­tio­nal­ity helps us avoid mis­takes and bad ideas, but doesn’t help much in gen­er­at­ing good ideas and use­ful work.

  Eliezer Yud­kowsky has made similar points in e.g. “Un­teach­able Ex­cel­lence” (“much of the most im­por­tant in­for­ma­tion we can learn from his­tory is about how to not lose, rather than how to win”, “It’s eas­ier to avoid du­pli­cat­ing spec­tac­u­lar failures than to du­pli­cate spec­tac­u­lar suc­cesses. And it’s of­ten eas­ier to gen­er­al­ize failure be­tween do­mains.”) and “Teach­ing the Un­teach­able”.

[Question] De­gree of du­pli­ca­tion and co­or­di­na­tion in pro­jects that ex­am­ine com­put­ing prices, AI progress, and re­lated top­ics?

• riceissa 14 Apr 2019 19:57 UTC

  11 points

  on: Di­ag­o­nal­iza­tion Fixed Point Exercises

  Thoughts on #10:

  I am con­fused about this ex­er­cise. The stan­dard/​mod­ern proof of Gödel’s sec­ond in­com­plete­ness the­o­rem uses the Hilbert–Ber­nays–Löb deriv­abil­ity con­di­tions, which are stated as (a), (b), (c) in ex­er­cise #11. If the ex­er­cises are meant to be solved in se­quence, this seems to im­ply that #10 is solv­able with­out us­ing the deriv­abil­ity con­di­tions. I tried do­ing this for a while with­out get­ting any­where.

  Maybe an­other way to state my con­fu­sion is that I’m pretty sure that up to ex­er­cise #10, noth­ing that dis­t­in­guishes Peano ar­ith­metic from Robin­son ar­ith­metic has been in­tro­duced (it is only with the in­tro­duc­tion of the deriv­abil­ity con­di­tions in #11 that this differ­ence be­comes ap­par­ent). It looks like there is a ver­sion of the sec­ond in­com­plete­ness the­o­rem for Robin­son ar­ith­metic, but the pa­per says “The proof is by the con­struc­tion of a non­stan­dard model in which this for­mula [i.e. for­mula ex­press­ing con­sis­tency] is false”, so I’m guess­ing this proof won’t work for Peano ar­ith­metic.

• riceissa 14 Apr 2019 19:41 UTC

  11 points

  on: Di­ag­o­nal­iza­tion Fixed Point Exercises

  My solu­tion for #12:

  Sup­pose for the sake of con­tra­dic­tion that such a for­mula $\varphi$ ex­ists. By the di­ag­o­nal lemma ap­plied to $\neg \varphi$, there is some sen­tence $\psi$ such that, prov­ably, $\neg \varphi (┌\psi ┐)\leftrightarrow \psi$. By the sound­ness of our the­ory, in fact $\neg \varphi (┌\psi ┐)\leftrightarrow \psi$. But by the prop­erty for $\varphi$ we also have $\varphi (┌\psi ┐)\leftrightarrow \psi$, which means $\neg \varphi (┌\psi ┐)\leftrightarrow \varphi (┌\psi ┐)$, a con­tra­dic­tion.

  This seems to be the “se­man­tic” ver­sion of the the­o­rem, where the prop­erty for $\varphi$ is stated out­side the sys­tem. There is also a “syn­tac­tic” ver­sion where the prop­erty for $\varphi$ is stated within the sys­tem.

• riceissa 1 Apr 2019 18:56 UTC

  9 points

  on: Di­ag­o­nal­iza­tion Fixed Point Exercises

  At­tempted solu­tion and some thoughts on #9:

  Define a for­mula $\xi \left(v\right)$ tak­ing one free vari­able to be $f(┌\varphi ┐,┌f(v,v)┐)$.

  Now define $\psi$ to be $f(┌\xi ┐,┌\xi ┐)$. By the defi­ni­tion of $f$ we have $\psi \leftrightarrow \xi (┌\xi ┐)$.

  We have $$\varphi (┌\psi ┐)\leftrightarrow f(┌\varphi ┐,┌\psi ┐)\leftrightarrow f(┌\varphi ┐,┌f(┌\xi ┐,┌\xi ┐)┐)\leftrightarrow \xi (┌\xi ┐)\leftrightarrow \psi$$

  The first step fol­lows by the defi­ni­tion of $f$, the sec­ond by the defi­ni­tion of $\psi$, the third by the defi­ni­tion of $\xi$, and the fourth by the prop­erty of $\psi$ men­tioned above. Since $\psi \in S_0$ by the type sig­na­ture of $f$, this com­pletes the proof.

  Things I’m not sure about:

  It’s a lit­tle un­clear to me what the $Syn(A,B)$ no­ta­tion means. In par­tic­u­lar, I’ve as­sumed that $f$ takes as in­puts Gödel num­bers of for­mu­las rather than the for­mu­las them­selves. If $f$ takes as in­puts the for­mu­las them­selves, then I don’t think we can as­sume that the for­mula $\xi$ ex­ists with­out do­ing more ar­ith­me­ti­za­tion work (i.e. the equiv­a­lent of $\xi$ would need to know how to con­vert from the Gödel num­ber of a for­mula to the for­mula it­self).

  If the bi­con­di­tional “$\leftrightarrow$” is a con­nec­tive in the logic it­self, then I think the same proof works but we would need to as­sume more about $f$ than is given in the prob­lem state­ment, namely that the the­ory we have can prove the sub­sti­tu­tion prop­erty of $f$.

  The as­sump­tion about the quan­tifier com­plex­ity of $S_0$ and $S_1$ was barely used. It was just given to us in the type sig­na­ture for $f$, and the same proof would have worked with­out this as­sump­tion, so I am con­fused about why the prob­lem in­cludes this as­sump­tion.

• riceissa 26 Mar 2019 4:22 UTC

  1 point

  in reply to: Diffractor's comment on: Open Prob­lems Re­gard­ing Coun­ter­fac­tu­als: An In­tro­duc­tion For Beginners

  That link works, thanks!

Com­par­i­son of de­ci­sion the­o­ries (with a fo­cus on log­i­cal-coun­ter­fac­tual de­ci­sion the­o­ries)

• riceissa 14 Mar 2019 23:13 UTC

  10 points

  AF

  on: Open Prob­lems Re­gard­ing Coun­ter­fac­tu­als: An In­tro­duc­tion For Beginners

  The link no longer works (I get “This pro­ject has not yet been moved into the new ver­sion of Over­leaf. You will need to log in and move it in or­der to con­tinue work­ing on it.”) Would you be will­ing to re-post it or move it so that it is visi­ble?

• riceissa 3 Jan 2019 7:13 UTC

  12 points

  on: What ex­er­cises go best with 3 blue 1 brown’s Lin­ear Alge­bra videos?

  Some other sources of ex­er­cises you might want to check out (that have solu­tions and that I have used at least partly):

  • Mul­ti­ple choice quizzes (the ones re­lated to lin­ear alge­bra are de­ter­mi­nants, el­e­men­tary ma­tri­ces, in­ner product spaces, lin­ear alge­bra, lin­ear sys­tems, lin­ear trans­for­ma­tions, ma­tri­ces, and vec­tor spaces)

  • Vipul Naik’s quizzes (dis­clo­sure: I am friends with Vipul and also do con­tract work for him)

  Re­gard­ing Axler’s book (since it has been men­tioned in this thread): there are sev­eral “lev­els” of lin­ear alge­bra, and Axler’s book is at a higher level (em­pha­sis on ab­stract vec­tor spaces and co­or­di­nate-free ways of do­ing things) than the 3Blue1Brown videos (more con­crete, work­ing in $R^n$). Axler’s book also as­sumes that the reader has had ex­po­sure to the lower level ma­te­rial (e.g. he does not talk about row re­duc­tion and el­e­men­tary ma­tri­ces). So I’m not sure I would recom­mend it to some­one start­ing out try­ing to learn the ba­sics of lin­ear alge­bra.

  Gra­tu­itous re­marks:

  • I think differ­ent re­sources cov­er­ing ma­te­rial in a differ­ent or­der and us­ing differ­ent ter­minol­ogy is in some sense a fea­ture, not a bug, be­cause it al­lows one to look at the sub­ject from differ­ent per­spec­tives. For in­stance, the “done right” in Axler’s book comes from one such change in per­spec­tive.

  • I find that learn­ing math­e­mat­ics well takes an un­in­tu­itively long time; it might be un­re­al­is­tic to ex­pect to learn the ma­te­rial well un­less one puts in a lot of effort.

  • I think there is a case to be made for the im­por­tance of strug­gling in learn­ing (dis­clo­sure: I am the au­thor of the page).

GraphQL tu­to­rial for LessWrong and Effec­tive Altru­ism Forum

• riceissa 8 Dec 2018 5:32 UTC

  4 points

  on: LW Up­date 2018-12-06 – Table of Con­tents and Q&A

  Is there (or will there be) a way to see a list of the lat­est posts, re­stricted to posts that are ques­tions? (I am won­der­ing about this both in the GraphQL API and in the site UI.)

• riceissa 29 Nov 2018 20:40 UTC

  3 points

  in reply to: TurnTrout's comment on: Turn­ing Up the Heat: In­sights from Tao’s ‘Anal­y­sis II’

  I think we are work­ing off differ­ent edi­tions. Ac­cord­ing to the er­rata, the con­di­tion for strict con­trac­tion was changed to $|f\left(x\right)-f\left(y\right)|<|x-y|$ for all dis­tinct $x,y\in [a,b]$.