riceissa

• riceissa 28 Oct 2014 7:52 UTC

  31 points

  on: 2014 Less Wrong Cen­sus/​Survey

  I took the sur­vey.

• riceissa 14 May 2019 20:04 UTC

  15 points

  on: Flash­cards for AI Safety?

  I’ve made around 250 Anki cards about AI safety. I haven’t pri­ori­tized shar­ing my cards be­cause I think find­ing a spe­cific card use­ful re­quires some­one to have read the source ma­te­rial gen­er­at­ing the card (e.g. if I made the card based on a blog post, one would need to read that ex­act blog post to get value out of re­view­ing the card; see learn be­fore you mem­o­rize). Since there are many AI safety blog posts and I don’t have the sense that lots of Anki users read any par­tic­u­lar blog post, it seems to me that the value gen­er­ated from shar­ing a set of cards about a blog post isn’t high enough to over­come the an­noy­ance cost of pol­ish­ing, pack­ag­ing, and up­load­ing the cards.

  More gen­er­ally, from a con­sumer per­spec­tive, I think peo­ple tend to be pretty bad at mak­ing good Anki cards (I’m of­ten em­bar­rassed at the cards I’ve cre­ated sev­eral months ago!), which makes it un­ex­cit­ing for me to spend a lot of effort try­ing to col­lab­o­rate with oth­ers on mak­ing cards (be­cause I ex­pect to re­ceive poorly-made cards in re­turn for the cards I provide). I think col­lab­o­ra­tive card-mak­ing can be done though, e.g. Michael Niel­sen and Andy Ma­tuschak’s quan­tum com­put­ing guide comes with pre-made cards that I think are pretty good.

  Differ­ent peo­ple also have differ­ent goals/​in­ter­ests so even given a sin­gle source ma­te­rial, the speci­fics one wants to Ankify can be differ­ent. For ex­am­ple, some­one who wants to un­der­stand the tech­ni­cal de­tails of log­i­cal in­duc­tion will want to Ankify the com­mon ob­jects used (mar­ket, pric­ing, trader, val­u­a­tion fea­ture, etc.), the the­o­rems and proof tech­niques, and so forth, whereas some­one who just wants a high-level overview and the “so what” of log­i­cal in­duc­tion can get away with Ankify­ing much less de­tail.

  Some­thing I’ve no­ticed is that many AI safety posts aren’t very good at ex­plain­ing things (not enough con­crete ex­am­ples, not enough em­pha­sis on com­mon mis­con­cep­tions and edge cases, not enough effort to an­swer what I think of as “ob­vi­ous” ques­tions); this fact is of­ten re­vealed by the com­ments peo­ple make in re­sponse to a post. This makes it hard to make Anki cards be­cause one doesn’t re­ally un­der­stand the con­tent of the post, at least not well enough to con­fi­dently gen­er­ate Anki cards (one of the benefits of be­ing an Anki user is hav­ing a greater sen­si­tivity to when one does not un­der­stand some­thing; see “illu­sion of ex­plana­tory depth” and re­lated terms). There are other prob­lems like con­flict­ing us­age of ter­minol­ogy (e.g. mul­ti­ple defi­ni­tions of “be­nign”, “al­igned”, “cor­rigible”) and the fact that some of the de­bates are on­go­ing/​some of the knowl­edge is still be­ing worked out.

  For “What would be a good strat­egy for gen­er­at­ing use­ful flash­cards?”: I try to read a post or a se­ries of posts and once I feel that I un­der­stand the ba­sic idea, I will usu­ally reread it to add cards about the ba­sic terms and ask my­self sim­ple ques­tions. Some ex­am­ple cards for iter­ated am­plifi­ca­tion:

  • what kind of train­ing does the Distill step use?

  • in the pseu­docode, what step gets re­peated/​iter­ated?

  • how do we get A[0]?

  • write A[1] in terms of H and A[0]

  • when Paul says IDA is go­ing to be com­pet­i­tive with tra­di­tional RL agents in terms of time and re­source costs, what ex­actly does he mean?

  • ad­van­tages of A[0] over H

  • sym­bolic ex­pres­sion for the overseer

  • why should the am­plified sys­tem (of hu­man + mul­ti­ple copies of the AI) be ex­pected to perform bet­ter than the hu­man alone?

• riceissa 25 Apr 2019 14:33 UTC

  15 points

  on: When is ra­tio­nal­ity use­ful?

  We can model suc­cess as a com­bi­na­tion of do­ing use­ful things and avoid­ing mak­ing mis­takes. As a par­tic­u­lar ex­am­ple, we can model in­tel­lec­tual suc­cess as a com­bi­na­tion of com­ing up with good ideas and avoid­ing bad ideas. I claim that ra­tio­nal­ity helps us avoid mis­takes and bad ideas, but doesn’t help much in gen­er­at­ing good ideas and use­ful work.

  Eliezer Yud­kowsky has made similar points in e.g. “Un­teach­able Ex­cel­lence” (“much of the most im­por­tant in­for­ma­tion we can learn from his­tory is about how to not lose, rather than how to win”, “It’s eas­ier to avoid du­pli­cat­ing spec­tac­u­lar failures than to du­pli­cate spec­tac­u­lar suc­cesses. And it’s of­ten eas­ier to gen­er­al­ize failure be­tween do­mains.”) and “Teach­ing the Un­teach­able”.

• riceissa 19 Nov 2018 2:33 UTC

  14 points

  AF

  on: Topolog­i­cal Fixed Point Exercises

  My solu­tion for #3:

  Define $g:[0,1]\to R$ by $g\left(x\right)=f\left(x\right)-x$. We know that $g$ is con­tin­u­ous be­cause $f$ and the iden­tity map both are, and by the limit laws. Ap­ply­ing the in­ter­me­di­ate value the­o­rem (prob­lem #2) we see that there ex­ists $x\in [0,1]$ such that $g\left(x\right)=0$. But this means $f\left(x\right)=x$, so we are done.

  Coun­terex­am­ple for the open in­ter­val: con­sider $f:(0,1)\to (0,1)$ defined by $f\left(x\right)=x/2$. First, we can ver­ify that if $0<x<1$ then $0<x/2<1/2<1$, so $f$ in­deed maps to $(0,1)$. To see that there is no fixed point, note that the only solu­tion to $x/2=x$ in $R$ is $0$, which is not in $(0,1)$. (We can also view this graph­i­cally by plot­ting both $y=x$ and $y=x/2$ and check­ing that they do not in­ter­sect in $(0,1)$.)

• riceissa 3 Jan 2019 7:13 UTC

  12 points

  on: What ex­er­cises go best with 3 blue 1 brown’s Lin­ear Alge­bra videos?

  Some other sources of ex­er­cises you might want to check out (that have solu­tions and that I have used at least partly):

  • Mul­ti­ple choice quizzes (the ones re­lated to lin­ear alge­bra are de­ter­mi­nants, el­e­men­tary ma­tri­ces, in­ner product spaces, lin­ear alge­bra, lin­ear sys­tems, lin­ear trans­for­ma­tions, ma­tri­ces, and vec­tor spaces)

  • Vipul Naik’s quizzes (dis­clo­sure: I am friends with Vipul and also do con­tract work for him)

  Re­gard­ing Axler’s book (since it has been men­tioned in this thread): there are sev­eral “lev­els” of lin­ear alge­bra, and Axler’s book is at a higher level (em­pha­sis on ab­stract vec­tor spaces and co­or­di­nate-free ways of do­ing things) than the 3Blue1Brown videos (more con­crete, work­ing in $R^n$). Axler’s book also as­sumes that the reader has had ex­po­sure to the lower level ma­te­rial (e.g. he does not talk about row re­duc­tion and el­e­men­tary ma­tri­ces). So I’m not sure I would recom­mend it to some­one start­ing out try­ing to learn the ba­sics of lin­ear alge­bra.

  Gra­tu­itous re­marks:

  • I think differ­ent re­sources cov­er­ing ma­te­rial in a differ­ent or­der and us­ing differ­ent ter­minol­ogy is in some sense a fea­ture, not a bug, be­cause it al­lows one to look at the sub­ject from differ­ent per­spec­tives. For in­stance, the “done right” in Axler’s book comes from one such change in per­spec­tive.

  • I find that learn­ing math­e­mat­ics well takes an un­in­tu­itively long time; it might be un­re­al­is­tic to ex­pect to learn the ma­te­rial well un­less one puts in a lot of effort.

  • I think there is a case to be made for the im­por­tance of strug­gling in learn­ing (dis­clo­sure: I am the au­thor of the page).

• riceissa 19 Nov 2018 1:54 UTC

  12 points

  AF

  in reply to: Davidmanheim's comment on: Topolog­i­cal Fixed Point Exercises

  Here is my at­tempt, based on Hoagy’s proof.

  Let $n\ge 1$ be an in­te­ger. We are given that $f\left(0/n\right)=f\left(0\right)\le 0$ and $f\left(n/n\right)=f\left(1\right)\ge 0$. Now con­sider the points $0/n,1/n,\dots ,(n-1)/n,n/n$ in the in­ter­val $[0,1]$. By 1-D Sperner’s lemma, there are an odd num­ber of $j\in \{0,\dots ,n\}$ such that $f\left(j/n\right)\ge 0$ and $f\left(\right(j-1\left)/n\right)\le 0$ (i.e. an odd num­ber of “seg­ments” that be­gin be­low zero and end up above zero). In par­tic­u­lar, $0$ is an even num­ber, so there must be at least one such num­ber $j$. Choose the small­est and call this num­ber $j_n$.

  Now con­sider the se­quence $(j_n/n{)}_{n=1}^{\infty }$. Since this se­quence takes val­ues in $[0,1]$, it is bounded, and by the Bolzano–Weier­strass the­o­rem there must be some sub­se­quence $(k_n/n{)}_{n=1}^{\infty }$ that con­verges to some num­ber $x\in [0,1]$.

  Con­sider the se­quences $\left(f\right((k_n-1)/n){)}_{n=1}^{\infty }$ and $\left(f\right(k_n/n){)}_{n=1}^{\infty }$. We have $f\left(\right(k_n-1\left)/n\right)\le 0\le f\left(k_n/n\right)$ for each $n\ge 1$. By the limit laws, $(k_n-1)/n\to x$ as $n\to \infty$. Since $f$ is con­tin­u­ous, we have $f\left(\right(k_n-1\left)/n\right)\to f\left(x\right)$ and $f\left(k_n/n\right)\to f\left(x\right)$ as $n\to \infty$. Thus $f\left(x\right)\le 0$ and $f\left(x\right)\ge 0$, show­ing that $f\left(x\right)=0$, as de­sired.

• riceissa 14 Apr 2019 19:57 UTC

  11 points

  on: Di­ag­o­nal­iza­tion Fixed Point Exercises

  Thoughts on #10:

  I am con­fused about this ex­er­cise. The stan­dard/​mod­ern proof of Gödel’s sec­ond in­com­plete­ness the­o­rem uses the Hilbert–Ber­nays–Löb deriv­abil­ity con­di­tions, which are stated as (a), (b), (c) in ex­er­cise #11. If the ex­er­cises are meant to be solved in se­quence, this seems to im­ply that #10 is solv­able with­out us­ing the deriv­abil­ity con­di­tions. I tried do­ing this for a while with­out get­ting any­where.

  Maybe an­other way to state my con­fu­sion is that I’m pretty sure that up to ex­er­cise #10, noth­ing that dis­t­in­guishes Peano ar­ith­metic from Robin­son ar­ith­metic has been in­tro­duced (it is only with the in­tro­duc­tion of the deriv­abil­ity con­di­tions in #11 that this differ­ence be­comes ap­par­ent). It looks like there is a ver­sion of the sec­ond in­com­plete­ness the­o­rem for Robin­son ar­ith­metic, but the pa­per says “The proof is by the con­struc­tion of a non­stan­dard model in which this for­mula [i.e. for­mula ex­press­ing con­sis­tency] is false”, so I’m guess­ing this proof won’t work for Peano ar­ith­metic.

• riceissa 14 Apr 2019 19:41 UTC

  11 points

  on: Di­ag­o­nal­iza­tion Fixed Point Exercises

  My solu­tion for #12:

  Sup­pose for the sake of con­tra­dic­tion that such a for­mula $\varphi$ ex­ists. By the di­ag­o­nal lemma ap­plied to $\neg \varphi$, there is some sen­tence $\psi$ such that, prov­ably, $\neg \varphi (┌\psi ┐)\leftrightarrow \psi$. By the sound­ness of our the­ory, in fact $\neg \varphi (┌\psi ┐)\leftrightarrow \psi$. But by the prop­erty for $\varphi$ we also have $\varphi (┌\psi ┐)\leftrightarrow \psi$, which means $\neg \varphi (┌\psi ┐)\leftrightarrow \varphi (┌\psi ┐)$, a con­tra­dic­tion.

  This seems to be the “se­man­tic” ver­sion of the the­o­rem, where the prop­erty for $\varphi$ is stated out­side the sys­tem. There is also a “syn­tac­tic” ver­sion where the prop­erty for $\varphi$ is stated within the sys­tem.

• riceissa 19 Nov 2018 1:29 UTC

  11 points

  AF

  in reply to: Hoagy's comment on: Topolog­i­cal Fixed Point Exercises

  I’m hav­ing trou­ble un­der­stand­ing why we can’t just fix $n=2$ in your proof. Then at each iter­a­tion we bi­sect the in­ter­val, so we wouldn’t be us­ing the “full power” of the 1-D Sperner’s lemma (we would just be us­ing some­thing close to the base case).

  Also if we are only given that $f$ is con­tin­u­ous, does it make sense to talk about the gra­di­ent?

• riceissa 14 Mar 2019 23:13 UTC

  10 points

  AF

  on: Open Prob­lems Re­gard­ing Coun­ter­fac­tu­als: An In­tro­duc­tion For Beginners

  The link no longer works (I get “This pro­ject has not yet been moved into the new ver­sion of Over­leaf. You will need to log in and move it in or­der to con­tinue work­ing on it.”) Would you be will­ing to re-post it or move it so that it is visi­ble?

• riceissa 1 Apr 2019 18:56 UTC

  9 points

  on: Di­ag­o­nal­iza­tion Fixed Point Exercises

  At­tempted solu­tion and some thoughts on #9:

  Define a for­mula $\xi \left(v\right)$ tak­ing one free vari­able to be $f(┌\varphi ┐,┌f(v,v)┐)$.

  Now define $\psi$ to be $f(┌\xi ┐,┌\xi ┐)$. By the defi­ni­tion of $f$ we have $\psi \leftrightarrow \xi (┌\xi ┐)$.

  We have $$\varphi (┌\psi ┐)\leftrightarrow f(┌\varphi ┐,┌\psi ┐)\leftrightarrow f(┌\varphi ┐,┌f(┌\xi ┐,┌\xi ┐)┐)\leftrightarrow \xi (┌\xi ┐)\leftrightarrow \psi$$

  The first step fol­lows by the defi­ni­tion of $f$, the sec­ond by the defi­ni­tion of $\psi$, the third by the defi­ni­tion of $\xi$, and the fourth by the prop­erty of $\psi$ men­tioned above. Since $\psi \in S_0$ by the type sig­na­ture of $f$, this com­pletes the proof.

  Things I’m not sure about:

  It’s a lit­tle un­clear to me what the $Syn(A,B)$ no­ta­tion means. In par­tic­u­lar, I’ve as­sumed that $f$ takes as in­puts Gödel num­bers of for­mu­las rather than the for­mu­las them­selves. If $f$ takes as in­puts the for­mu­las them­selves, then I don’t think we can as­sume that the for­mula $\xi$ ex­ists with­out do­ing more ar­ith­me­ti­za­tion work (i.e. the equiv­a­lent of $\xi$ would need to know how to con­vert from the Gödel num­ber of a for­mula to the for­mula it­self).

  If the bi­con­di­tional “$\leftrightarrow$” is a con­nec­tive in the logic it­self, then I think the same proof works but we would need to as­sume more about $f$ than is given in the prob­lem state­ment, namely that the the­ory we have can prove the sub­sti­tu­tion prop­erty of $f$.

  The as­sump­tion about the quan­tifier com­plex­ity of $S_0$ and $S_1$ was barely used. It was just given to us in the type sig­na­ture for $f$, and the same proof would have worked with­out this as­sump­tion, so I am con­fused about why the prob­lem in­cludes this as­sump­tion.

• riceissa 29 Nov 2018 7:44 UTC

  9 points

  on: Turn­ing Up the Heat: In­sights from Tao’s ‘Anal­y­sis II’

  Can you say more about why ex­er­cise 17.6.3 is wrong?

  If we define $f:[0,1]\to R$ by $f\left(x\right):=x/(1+x)$ then for dis­tinct $x,y\in [0,1]$ we have $$|f\left(x\right)-f\left(y\right)|=|\frac{x}{1+x}-\frac{y}{1+y}|=\left|\frac{x-y}{(1+x)(1+y)}\right|<|x-y|$$

  We also have $f^{\prime }\left(0\right)=1$ since $$\underset{x\to 0;x\in (0,1]}{lim}\frac{f\left(x\right)-f\left(0\right)}{x-0}=\underset{x\to 0;x\in (0,1]}{lim}\frac{x}{(x+1)x}=1$$

  In gen­eral, the deriva­tive is $f^{\prime }\left(x\right)=1/(1+x{)}^2$, which is con­tin­u­ous on $[0,1]$.

• riceissa 25 Jul 2018 22:09 UTC

  8 points

  in reply to: avturchin's comment on: Prob­a­bil­ity is Real, and Value is Complex

  I was con­fused about this too, but now I think I have some idea of what’s go­ing on.

  Nor­mally prob­a­bil­ity is defined for events, but ex­pected value is defined for ran­dom vari­ables, not events. What is hap­pen­ing in this post is that we are tak­ing the ex­pected value of events, by way of the con­di­tional ex­pected value of the ran­dom vari­able (con­di­tion­ing on the event). In sym­bols, if $A\subset \Omega$ is some event in our sam­ple space, we are say­ing $E\left(A\right)=E(X\mid A)$, where $X:\Omega \to R$ is some ran­dom vari­able (this ran­dom vari­able is sup­posed to be clear from the con­text, so it doesn’t ap­pear on the left hand side of the equa­tion).

  Go­ing back to cousin_it’s lot­tery ex­am­ple, we can for­mal­ize this as fol­lows. The sam­ple space can be $\Omega =\{\text{win},\text{lose}\}$ and the prob­a­bil­ity mea­sure is defined as $P\left(\right\{\text{win}\left\}\right)=1/{10}^6$ and $P\left(\right\{\text{lose}\left\}\right)=1-1/{10}^6$. The ran­dom vari­able $L:\Omega \to R$ rep­re­sents the lot­tery, and it is defined by $L\left(\text{win}\right)={10}^6$ and $L\left(\text{lose}\right)=0$.

  Now we can calcu­late. The ex­pected value of the lot­tery is:

  $$E\left(L\right)=L\left(\text{win}\right)P\left(\right\{\text{win}\left\}\right)+L\left(\text{lose}\right)P\left(\right\{\text{lose}\left\}\right)={10}^6/{10}^6+0\cdot (1-{10}^6)=1$$

  The ex­pected value of win­ning is:

  $$\begin{array}{cc}E\left(\right\{\text{win}\left\}\right)& =E(L\mid \{\text{win}\left\}\right)\\ & =L\left(\text{win}\right)P(\text{win}\mid \{\text{win}\left\}\right)+L\left(\text{lose}\right)P(\text{lose}\mid \{\text{win}\left\}\right)\\ & ={10}^6\cdot 1+0\cdot 0\\ & ={10}^6\end{array}$$

  The “probu­til­ity” of win­ning is:

  $$Q\left(\right\{\text{win}\left\}\right)=P\left(\right\{\text{win}\left\}\right)E\left(\right\{\text{win}\left\}\right)=\frac{1}{{10}^6}\cdot {10}^6=1$$

  So in this case, the “probu­til­ity” of win­ning is the same as the ex­pected value of the lot­tery. How­ever, this is only the case be­cause the situ­a­tion is so sim­ple. In par­tic­u­lar, if $L\left(\text{lose}\right)$ was not equal to zero (while win­ning and los­ing re­mained ex­clu­sive events), then the two would have been differ­ent (the ex­pected value of the lot­tery would have changed while the “probu­til­ity” would have re­mained the same).

• riceissa 19 May 2019 15:44 UTC

  4 points

  in reply to: Zvi's comment on: An­nounce­ment: AI al­ign­ment prize round 4 winners

  Was this post ever pub­lished? (Look­ing at Zvi’s posts since Jan­uary, I don’t see any­thing that looks like it.)

• riceissa 8 Dec 2018 5:32 UTC

  4 points

  on: LW Up­date 2018-12-06 – Table of Con­tents and Q&A

  Is there (or will there be) a way to see a list of the lat­est posts, re­stricted to posts that are ques­tions? (I am won­der­ing about this both in the GraphQL API and in the site UI.)

• riceissa 29 Nov 2018 20:40 UTC

  3 points

  in reply to: TurnTrout's comment on: Turn­ing Up the Heat: In­sights from Tao’s ‘Anal­y­sis II’

  I think we are work­ing off differ­ent edi­tions. Ac­cord­ing to the er­rata, the con­di­tion for strict con­trac­tion was changed to $|f\left(x\right)-f\left(y\right)|<|x-y|$ for all dis­tinct $x,y\in [a,b]$.

• riceissa 27 Aug 2018 4:56 UTC

  2 points

  in reply to: Qiaochu_Yuan's comment on: “Flinch­ing away from truth” is of­ten about *pro­tect­ing* the epistemology

  I had a similar thought while read­ing this post, but I’m not sure in­vok­ing causal­ity is nec­es­sary (hav­ing a di­rec­tion still seems nec­es­sary). Just in terms of propo­si­tional logic, I would ex­plain this post as fol­lows:

  1. Ini­tially, one has the im­pli­ca­tion $X⟹Y$ stored in one’s mind.

  2. Some­one as­serts $X$.

  3. Now one’s mind (per­haps sub­con­sciously) does a modus po­nens, and ob­tains $Y$.

  4. How­ever, $Y$ is an un­de­sir­able be­lief, so one wants to deny it.

  5. In­stead of re­ject­ing the im­pli­ca­tion $X⟹Y$, one adamantly de­nies $X$.

  The “buck­ets er­ror” is the im­pli­ca­tion $X⟹Y$, and “flinch­ing away” is the de­nial of $X$. Flinch­ing away is about pro­tect­ing one’s episte­mol­ogy be­cause deny­ing $X$ is still bet­ter than ac­cept­ing $Y$. Of course, it would be best to re­ject the im­pli­ca­tion $X⟹Y$, but since one can’t do this (by as­sump­tion, one makes the buck­ets er­ror), it is prefer­able to “flinch away” from $X$.

  ETA (2019-02-01): It oc­curred to me that this is ba­si­cally the same thing as “one man’s modus po­nens is an­other man’s modus tol­lens” (see e.g. this post) but with some ex­tra emo­tional con­no­ta­tions.

• riceissa 4 Apr 2018 23:17 UTC

  2 points

  on: Op­por­tu­ni­ties for in­di­vi­d­ual donors in AI safety

  I don’t see refer­ence num­ber 17 (“Per­sonal cor­re­spon­dence with Carl Shul­man”) used in the body of the post. What in­for­ma­tion from that refer­ence is used in the post?

• riceissa 5 Jul 2017 19:19 UTC

  2 points

  on: Timeline of Carl Shul­man publications

  I re­cently wrote an up­dated timeline. It in­cludes not just for­mal pub­li­ca­tions, but also blog posts and con­ver­sa­tions. To see just the for­mal pub­li­ca­tions, it is pos­si­ble to sort by the “For­mat” column in the full timeline and look at the rows with “Paper”.

• riceissa 28 Sep 2016 23:03 UTC

  2 points

  in reply to: VipulNaik's comment on: Linkposts now live!

  I con­firm that I also ex­pe­rience this prob­lem, but I don’t have ad­di­tional in­sight on the cause.