riceissa

• riceissa 4 Dec 2019 21:45 UTC

  3 points

  on: What ad­di­tional fea­tures would you like on LessWrong?

  I won­der if you saw Oliver’s re­ply here?

• riceissa 4 Dec 2019 7:48 UTC

  3 points

  on: Open & Wel­come Thread—De­cem­ber 2019

  Here are some ex­am­ples I found of non-Wikipe­dia-re­lated wikis dis­cour­ag­ing the use of ex­ter­nal links:

  Doom Wiki:

  Links to ex­ter­nal sites should be used in mod­er­a­tion. To be can­di­date for link­ing, an ex­ter­nal site should con­tain in­for­ma­tion that serves as a refer­ence for the ar­ti­cle, is the sub­ject of the ar­ti­cle it­self, is offi­cial in some ca­pac­ity (for ex­am­ple, run by id Soft­ware), or con­tains ad­di­tional read­ing that is not ap­pro­pri­ate in the en­cy­clo­pe­dic set­ting of this wiki. We are not a search en­g­ine. Ex­ten­sive lists of links cre­ate clut­ter and are ex­ceed­ingly difficult to main­tain. They may also de­grade the search en­g­ine rank­ing of this site.

  Sa­suke­pe­dia:

  Elinks should be con­strained to one sec­tion ti­tled “Ex­ter­nal links” at the end of a page. Elinks within the main con­tent of a page are dis­cour­aged, and should be avoided where pos­si­ble.

  Wook­ieepe­dia:

  If you want to link to a site out­side of Wook­ieepe­dia, it should al­most always go un­der an “Ex­ter­nal links” head­ing at the end of an ar­ti­cle. Avoid us­ing an ex­ter­nal link when it’s pos­si­ble to ac­com­plish the same thing with an in­ter­nal link to a Wook­ieepe­dia ar­ti­cle.

  IMSMA wiki:

  Avoid us­ing ex­ter­nal links in the body of a page. Pages can in­clude an ex­ter­nal links sec­tion at the end, point­ing to fur­ther in­for­ma­tion out­side IMSMA Wiki.

  Feed The Beast Wiki:

  Ex­ter­nal links should not be used in­stead of wik­ilinks un­less ab­solutely nec­es­sary.

• riceissa 3 Dec 2019 9:22 UTC

  12 points

  on: Open & Wel­come Thread—De­cem­ber 2019

  When think­ing about in­for­ma­tion asym­me­try in trans­ac­tions (e.g. in­surance, mar­ket for lemons), I can think of sev­eral axes for com­par­i­son:

  1. whether the thing hap­pens be­fore or af­ter the transaction

  2. whether the buyer or sel­ler has more information

  3. whether the asym­me­try is about a good or about an action

  In­surance-like trans­ac­tions pick “af­ter”, “buyer”, and “ac­tion”: the per­son buy­ing the in­surance can choose to act more care­lessly af­ter pur­chas­ing the in­surance.

  Mar­ket for lemons cases pick “be­fore”, “sel­ler”, and “good”: prior to the trans­ac­tion, the sel­ler of the good knows more about the qual­ity of the good.

  So in many typ­i­cal cases, the three axes “al­ign”, and the former is called a moral haz­ard and the lat­ter is called ad­verse se­lec­tion.

  But there are ex­am­ples like an all-you-can-eat buf­fet that sets a sin­gle price (which en­courages high-ap­petite peo­ple to eat there). This case picks “be­fore”, “buyer”, and “ac­tion”. So in this case, ²⁄₃ of the axes agree with the in­surance-like situ­a­tion, but this case is still clas­sified as ad­verse se­lec­tion be­cause the offi­cial dis­tinc­tion is about (1).

  Wikipe­dia states “Where ad­verse se­lec­tion de­scribes a situ­a­tion where the type of product is hid­den from one party in a trans­ac­tion, moral haz­ard de­scribes a situ­a­tion where there is a hid­den ac­tion that re­sults from the trans­ac­tion” (i.e. claims that (3) is the rele­vant axis) but then on the same page also states “For ex­am­ple, an all-you-can-eat buf­fet restau­rant that sets one price for all cus­tomers risks be­ing ad­versely se­lected against by high ap­petite” (i.e. clas­sifies this ex­am­ple as ad­verse se­lec­tion, even though clas­sify­ing ac­cord­ing to (3) would re­sult in call­ing this a moral haz­ard).

  Does any­one know why (1) is the most in­ter­est­ing axis (which I’m in­fer­ring based on how only this axis seems to have names for the two ends)?

• riceissa 3 Dec 2019 8:45 UTC

  10 points

  on: Open & Wel­come Thread—De­cem­ber 2019

  I’ve been won­der­ing about de­sign differ­ences be­tween blogs and wikis. For ex­am­ple:

  • Most of the wikis I know use a vari­able width for the body text, rather than a nar­row fixed width that is com­mon on many web­sites (in­clud­ing blogs)

  • Most of the wikis I know have a sep­a­rate dis­cus­sion page, whereas most blogs have a com­ments sec­tion on the same page as the content

  • I think wikis tend to have smaller font size than blogs

  • Wikis make a hard dis­tinc­tion be­tween in­ter­nal links (wik­ilinks) and ex­ter­nal links, go­ing so far as to dis­cour­age the use of ex­ter­nal links in the body text in some cases

  I find the above differ­ences in­ter­est­ing be­cause they can’t be ex­plained (or are not so easy to ex­plain) just by say­ing some­thing like “a wiki is a col­lab­o­ra­tive on­line refer­ence where each page is a dis­tinct topic while a blog is a chronolog­i­cal list of ar­ti­cles where each ar­ti­cle tends to have a sin­gle au­thor”; this ex­pla­na­tion only works for things like em­pha­sis on pub­li­ca­tion date (wikis are not chronolog­i­cal, so don’t need to em­pha­size pub­li­ca­tion date), availa­bil­ity of full his­tory (wikis are col­lab­o­ra­tive, so hav­ing a full his­tory helps to see who added what and to re­vert van­dal­ism), dis­play of au­thor­ship (blogs usu­ally have a sin­gle au­thor per post so list­ing this makes sense, but wiki pages have many au­thors so list­ing all of them makes less sense), stan­dard­ized sec­tion names (a blog au­thor can just ram­ble about what­ever, but wikis need to build con­sis­tency in how top­ics are cov­ered), and tone/​writ­ing style (blogs can just be one au­thor’s opinions, whereas wikis need to agree on some con­sis­tent tone).

  Has any­one thought about these differ­ences, es­pe­cially what would ex­plain them? Search­ing vari­a­tions of “wikis vs blogs” on the in­ter­net yields ir­rele­vant re­sults.

• riceissa 1 Dec 2019 20:42 UTC

  1 point

  on: Open & Wel­come Thread—Novem­ber 2019

  Re­lated dis­cus­sion: https://​​www.face­book.com/​​vipul­naik.r/​​posts/​​10213221225461126

• riceissa 20 Nov 2019 0:14 UTC

  3 points

  on: What I’ll be do­ing at MIRI

  [Meta] At the mo­ment, Oliver’s com­ment has 15 karma across 1 vote (and 6 AF karma). If I’m un­der­stand­ing LW’s vot­ing sys­tem cor­rectly, the only way this could have hap­pened is if Oliver un­did his de­fault vote on the com­ment, and then Eliezer Yud­kowsky did a strong-up­vote on the com­ment (see here for a list of users by vot­ing power). But my in­tu­ition says this sce­nario is im­plau­si­ble, so I’m cu­ri­ous what hap­pened in­stead.

  (This isn’t im­por­tant, but I’m cu­ri­ous any­way.)

• riceissa 28 Oct 2019 22:54 UTC

  6 points

  on: [AN #62] Are ad­ver­sar­ial ex­am­ples caused by real but im­per­cep­ti­ble fea­tures?

  Based on the Oc­to­ber 2019 up­date, it looks like Ought is now us­ing “fac­tored cog­ni­tion” as an um­brella term that in­cludes both fac­tored gen­er­a­tion (which used to be called fac­tored cog­ni­tion) and fac­tored eval­u­a­tion.

  (Com­ment­ing here be­cause as far as I know this post is one of the main places that dis­cusses this dis­tinc­tion.)

• riceissa 26 Oct 2019 22:29 UTC

  6 points

  on: Jacy Reese (born Jacy An­this)?

  It’s in­ter­est­ing this minor fact that, to sev­eral peo­ple in­clud­ing me, has seemed like an ob­vi­ous omis­sion, doesn’t meet Wikipe­dia’s stan­dards for in­clu­sion. But if Wikipe­dia had less strict stan­dards it would be very hard to keep out false in­for­ma­tion.

  Eliezer Yud­kowsky has made similar dis­tinc­tions when talk­ing about sci­en­tific vs le­gal vs ra­tio­nal ev­i­dence (see this wiki page) and sci­ence vs prob­a­bil­ity the­ory.

  I think there is an in­ter­est­ing ques­tion of “what ought to count as ev­i­dence, if we want to pro­duce the best on­line en­cy­clo­pe­dia we can, given the flawed hu­mans we have to write it?” My own view is that Wikipe­dia’s stan­dards for ev­i­dence have be­come too strict in cases like this.

• riceissa 26 Oct 2019 1:54 UTC

  1 point

  on: De­liber­a­tion as a method to find the “ac­tual prefer­ences” of humans

  Thanks, I think I agree (but want to think about this more). I might edit the post in the fu­ture to in­cor­po­rate this change.

• riceissa 26 Oct 2019 1:51 UTC

  1 point

  on: De­liber­a­tion as a method to find the “ac­tual prefer­ences” of humans

  I agree with this, and didn’t mean to im­ply any­thing against it in the post.

• riceissa 26 Oct 2019 1:42 UTC

  8 points

  on: Two ex­pla­na­tions for vari­a­tion in hu­man abilities

  Re­gard­ing your foot­note, liter­acy rates de­pend on the defi­ni­tion of liter­acy used. Un­der min­i­mal defi­ni­tions, “pretty close to 100 per­cent of the pop­u­la­tion is ca­pa­ble of read­ing” is true, but un­der stric­ter defi­ni­tions, “maybe 20 or 30 per­cent” seems closer to the mark.

  https://​​en.wikipe­dia.org/​​wiki/​​Liter­acy#United_States

  https://​​en.wikipe­dia.org/​​wiki/​​Func­tional_illiter­acy#Prevalence

  https://​​en.wikipe­dia.org/​​wiki/​​Liter­acy_in_the_United_States

  “Cur­rent liter­acy data are gen­er­ally col­lected through pop­u­la­tion cen­suses or house­hold sur­veys in which the re­spon­dent or head of the house­hold de­clares whether they can read and write with un­der­stand­ing a short, sim­ple state­ment about one’s ev­ery­day life in any writ­ten lan­guage. Some sur­veys re­quire re­spon­dents to take a quick test in which they are asked to read a sim­ple pas­sage or write a sen­tence, yet clearly liter­acy is a far more com­plex is­sue that re­quires more in­for­ma­tion.” http://​​uis.un­esco.org/​​en/​​topic/​​literacy

  I’m not sure why you are so op­ti­mistic about peo­ple learn­ing calcu­lus.

• riceissa 24 Oct 2019 0:07 UTC

  5 points

  on: AI Align­ment Open Thread Oc­to­ber 2019

  Thanks!

  My sense (guess­ing from your post his­tory) is that you are mostly con­fused about com­ments.

  I am more con­fused about posts than com­ments. For posts, only my com­par­i­son of de­ci­sion the­o­ries post is cur­rently cross-posted to AF, but I ac­tu­ally think my post about de­liber­a­tion, ques­tion about iter­a­tive ap­proaches to al­ign­ment (along with Ro­hin’s an­swer), and ques­tion about co­or­di­na­tion on AI progress pro­jects are more rele­vant to AF (ei­ther be­cause they make new claims or be­cause they en­courage oth­ers to do so). If I see that a par­tic­u­lar post hasn’t been cross-posted to AF, I’m won­der­ing if I should be think­ing more like “ev­ery sin­gle mod­er­a­tor has looked at the post, and be­lieves it doesn’t be­long on AF” or more like “ei­ther the mod­er­a­tors are busy, or some­thing about the post ti­tle caused them to not look at the post, and it sort of fell through the cracks”.

• riceissa 23 Oct 2019 22:39 UTC

  1 point

  on: AI Align­ment Open Thread Oc­to­ber 2019

  [Meta] I’m not a full mem­ber on Align­ment Fo­rum, but I’ve had some of my LW con­tent cross-posted to AF. How­ever, this cross-post­ing seems hap­haz­ard, and does not cor­re­spond to my in­tu­itive feel­ing of which of my posts/​com­ments “should” end up on AF. I would like for one of the fol­low­ing to hap­pen:

  • More in­sight into the mechanism that de­cides what gets cross-posted, so I feel less an­noyed at the ar­bi­trary-seem­ing na­ture of it.

  • More con­trol over what gets cross-posted (if this re­quires ap­ply­ing for full mem­ber­ship, I would be will­ing to do that).

  • Have all my AF cross-post­ing be un­done so that read­ers don’t get a mis­lead­ing im­pres­sion of my AI al­ign­ment con­tent. (I would like to avoid peo­ple vis­it­ing my AF pro­file, read­ing con­tent there, and con­clud­ing some­thing about my AI al­ign­ment out­put based on that.)

• riceissa 23 Oct 2019 21:40 UTC

  3 points

  on: An1lam’s Short Form Feed

  If you’ve saved some of your fa­vorite links, please share!

  • I like CheCheDaWaff’s com­ments on r/​Anki; see here for a de­cent place to start. In par­tic­u­lar, for proofs, I’ve shifted to­ward adding “prove this the­o­rem” cards rather than try­ing to break the proof into many small pieces. (The lat­ter ad­heres more to the spaced rep­e­ti­tion philos­o­phy, but I found it just doesn’t re­ally work.)

  • Richard Reitz has a Google doc with a bunch of stuff.

  • I like this fo­rum com­ment (as a data point, and as mo­ti­va­tion to try to avoid similar failures).

  • I like https://​​es­ha­pard.github.io

  • Master How To Learn also has some in­sights but most posts are low-qual­ity.

  One thing I should men­tion is that a lot of the above links aren’t writ­ten well. See this Quora an­swer for a view I ba­si­cally agree with.

  I couldn’t stop think­ing about it

  I agree that think­ing about this is pretty ad­dict­ing. :) I think this kind of mo­ti­va­tion helps me to find and read a bunch on­line and to make oc­ca­sional com­ments (such as the grand­par­ent) and brain dumps, but I find it’s not quite enough to get me to in­vest the time to write a com­pre­hen­sive post about ev­ery­thing I’ve learned.

• riceissa 23 Oct 2019 4:10 UTC

  5 points

  on: An1lam’s Short Form Feed

  I would be sur­prised if Gw­ern hasn’t already thought about the claim go­ing to make

  I briefly looked at gw­ern’s pub­lic database sev­eral months ago, and got the im­pres­sion that he isn’t us­ing Anki in the in­cre­men­tal read­ing/​learn­ing way that you (and Michael Niel­sen) de­scribe. In­stead, he seems to just add a bunch of ran­dom facts. This isn’t to say gw­ern hasn’t thought about this, but just that if he has, he doesn’t seem to be mak­ing use of this in­sight.

  In the Pla­tonic graph of this do­main’s knowl­edge on­tol­ogy, how cen­tral is this node?

  I feel like the cen­ter of­ten shifts as I learn more about a topic (be­cause I de­velop new in­ter­ests within it). The ques­tions I ask my­self are more like “How em­bar­rassed would I be if some­one asked me this and I didn’t know the an­swer?” and “How much does know­ing this help me learn more about the topic or re­lated top­ics?” (Th­ese aren’t ideal phras­ings of the ques­tions my gut is ask­ing.)

  know­ing that I’ll re­mem­ber at least the stuff I’ve Anki-ized has a sur­pris­ingly strong mo­ti­va­tional im­pact on me on a gut level

  In my ex­pe­rience, I of­ten still for­get things I’ve en­tered into Anki ei­ther be­cause the card was poorly made or be­cause I didn’t add enough “sur­round­ing cards” to ce­ment the knowl­edge. So I’ve shifted away from this to think­ing some­thing more like “at least Anki will make it very ob­vi­ous if I didn’t in­ter­nal­ize some­thing well, and will give me an op­por­tu­nity in the fu­ture to come back to this topic to un­der­stand it bet­ter in­stead of just hav­ing it fade with­out de­tec­tion”.

  there’s O(5) ac­tual blog posts about it

  I’m con­fused about what you mean by this. (One guess I have is big-O no­ta­tion, but big-O no­ta­tion is not sen­si­tive to con­stants, so I’m not sure what the 5 is do­ing, and big-O no­ta­tion is also about asymp­totic be­hav­ior of a func­tion and I’m not sure what in­put you’re con­sid­er­ing.)

  I think there are few well-re­searched and com­pre­hen­sive blog posts, but I’ve found that there is a lot of ad­di­tional wis­dom the spaced rep­e­ti­tion com­mu­nity has ac­cu­mu­lated, which is mostly writ­ten down in ran­dom Red­dit com­ments and smaller blog posts. I feel like I’ve benefited some­what from read­ing this wis­dom (but have benefited more from just try­ing a bunch of things my­self). For my­self, I’ve con­sid­ered writ­ing up what I’ve learned about us­ing Anki, but it hasn’t been a pri­or­ity be­cause (1) other top­ics seem more im­por­tant to work on and write about; (2) most new­com­ers can­not dis­t­in­guish been good and bad ad­vice, so I an­ti­ci­pate hav­ing low im­pact by writ­ing about Anki; (3) I’ve only been ex­per­i­ment­ing in­for­mally and per­son­ally, and it’s difficult to tell how well my les­sons gen­er­al­ize to oth­ers.

• riceissa 22 Oct 2019 4:29 UTC

  9 points

  on: Ra­tion­al­ity Ex­er­cises Prize of Septem­ber 2019 ($1,000)

  Were the win­ners ever an­nounced? If I’m count­ing cor­rectly, it has now been over four weeks since Septem­ber 20, so the win­ners should have been an­nounced around two weeks ago. (I checked for new posts by Ben, this post, and the com­ments on this post.)

• riceissa 21 Oct 2019 20:28 UTC

  1 point

  on: AI Safety “Suc­cess Sto­ries”

  I think I was imag­in­ing that the pivotal tool AI is de­vel­oped by highly com­pe­tent and safety-con­scious hu­mans who use it to perform a pivotal act (or se­ries of pivotal acts) that effec­tively pre­cludes the kind of is­sues men­tioned in Wei’s quote there.

  Even if you make this as­sump­tion, it seems like the re­li­ance on hu­man safety does not go down. I think you’re think­ing about some­thing more like “how likely it is that lack of hu­man safety be­comes a prob­lem” rather than “re­li­ance on hu­man safety”.

• riceissa 21 Oct 2019 1:22 UTC

  7 points

  on: We tend to for­get com­pli­cated things

  I think you are de­scribing over­learn­ing and chunk­ing (once con­cepts be­come chun­ked they “feel easy”, and one re­li­able way to chunk ideas is to over­learn them).

  Some re­lated links:

  • A com­ment by Qiaochu Yuan

  • Bryan Ca­plan’s “Liber­tar­i­anism as Mo­ral Over­learn­ing”

  • Aug­ment­ing Long-term Me­mory (search “chunk”)

• riceissa 19 Oct 2019 2:17 UTC

  1 point

  on: Hu­mans can be as­signed any val­ues what­so­ever…

  I’m cu­ri­ous what you think of my com­ment here, which sug­gests that Kol­mogorov com­plex­ity might be enough af­ter all, as long as we are will­ing to change our no­tion of com­pat­i­bil­ity.

  (I’m also cu­ri­ous what you think of Daniel’s post, al­though to a lesser ex­tent.)

• riceissa 18 Oct 2019 1:32 UTC

  4 points

  on: AI Safety “Suc­cess Sto­ries”

  I think pivotal tool story has low re­li­ance on hu­man safety (al­though I’m con­fused by that row in gen­eral).

  From the Task-di­rected AGI page on Ar­bital:

  The ob­vi­ous dis­ad­van­tage of a Task AGI is moral haz­ard—it may tempt the users in ways that a Sovereign would not. A Sovereign has moral haz­ard chiefly dur­ing the de­vel­op­ment phase, when the pro­gram­mers and users are per­haps not yet in a po­si­tion of spe­cial rel­a­tive power. A Task AGI has on­go­ing moral haz­ard as it is used.

  (My un­der­stand­ing is that task AGI = ge­nie = Pivotal Tool.)

  Wei Dai gives some ex­am­ples of what could go wrong in this post:

  For ex­am­ple, such AIs could give hu­mans so much power so quickly or put them in such novel situ­a­tions that their moral de­vel­op­ment can’t keep up, and their value sys­tems no longer ap­ply or give es­sen­tially ran­dom an­swers. AIs could give us new op­tions that are ir­re­sistible to some parts of our mo­ti­va­tional sys­tems, like more pow­er­ful ver­sions of video game and so­cial me­dia ad­dic­tion. In the course of try­ing to figure out what we most want or like, they could in effect be search­ing for ad­ver­sar­ial ex­am­ples on our value func­tions. At our own re­quest or in a sincere at­tempt to help us, they could gen­er­ate philo­soph­i­cal or moral ar­gu­ments that are wrong but ex­tremely per­sua­sive.

  The un­der­ly­ing prob­lem seems to be that when hu­mans are in con­trol over long-term out­comes, we are rely­ing more on the hu­mans to have good judg­ment, and this be­comes in­creas­ingly a prob­lem the more task-shaped the AI be­comes.

  I’m cu­ri­ous what your own think­ing is (e.g. how would you fill out that row?).