Rob Bensinger

• Rob Bensinger 14 May 2019 19:46 UTC

  4 points

  in reply to: Said Achmiz's comment on: Co­her­ent de­ci­sions im­ply con­sis­tent utilities

  We’d talked about get­ting a dump out as well, and your plan sounds great to me! The LW team should get back to you with a list at some point (un­less they think of a bet­ter idea).

• Rob Bensinger 14 May 2019 3:44 UTC

  15 points

  in reply to: Said Achmiz's comment on: Co­her­ent de­ci­sions im­ply con­sis­tent utilities

  I asked Eliezer if it made sense to cross-post this from Ar­bital, and did the cross-post­ing when he ap­proved. I’m sorry it wasn’t clear that this was a cross-post! I in­tended to make this clearer, but my idea was bad (putting the in­for­ma­tion on the se­quence page) and I also im­ple­mented it wrong (the se­quence didn’t pre­vi­ously dis­play on the top of this post).

  This post was origi­nally writ­ten as a non­tech­ni­cal in­tro­duc­tion to ex­pected util­ity the­ory and co­her­ence ar­gu­ments. Although it be­gins in me­dia res stylis­ti­cally, it doesn’t have any pre­reqs or con­text be­yond “this is part of a col­lec­tion of in­tro­duc­tory re­sources cov­er­ing a wide va­ri­ety of tech­ni­cal and semitech­ni­cal top­ics.”

  Per the first sen­tence, the main pur­pose is for this to be a link­able re­source for con­ver­sa­tions/​in­quiry about hu­man ra­tio­nal­ity and con­ver­sa­tions/​in­quiry about AGI:

  So we’re talk­ing about how to make good de­ci­sions, or the idea of ‘bounded ra­tio­nal­ity’, or what suffi­ciently ad­vanced Ar­tifi­cial In­tel­li­gences might be like; and some­body starts drag­ging up the con­cepts of ‘ex­pected util­ity’ or ‘util­ity func­tions’. And be­fore we even ask what those are, we might first ask, Why?

  There have been loose plans for a while to cross-post con­tent from Ar­bital to LW (maybe all of it; maybe just the best or most in­ter­est­ing stuff), but as I men­tioned down­thread, we’re do­ing more cross-post ex­per­i­ments sooner than we would have be­cause Ar­bital’s been hav­ing se­ri­ous perfor­mance is­sues.

• Rob Bensinger 14 May 2019 3:34 UTC

  4 points

  in reply to: jimrandomh's comment on: Co­her­ent de­ci­sions im­ply con­sis­tent utilities

  I as­sume you mean ‘no one has this re­spon­si­bil­ity for Ar­bital any­more’, and not that there’s some­one else who has this re­spon­si­bil­ity.

• Rob Bensinger 14 May 2019 2:01 UTC

  9 points

  in reply to: Said Achmiz's comment on: Co­her­ent de­ci­sions im­ply con­sis­tent utilities

  Ar­bital has been get­ting in­creas­ingly slow and un­re­spon­sive. The LW team is look­ing for fixes or work-arounds, but they aren’t fa­mil­iar with the Ar­bital code­base. In the mean­time, I’ve been helping cross-post some con­tent from Ar­bital to LW so it’s available at all.

• Rob Bensinger 12 May 2019 1:28 UTC

  21 points

  on: Any re­but­tals of Chris­ti­ano and AI Im­pacts on take­off speeds?

  MIRI folks are the most promi­nent pro­po­nents of fast take­off, and we un­for­tu­nately haven’t had time to write up a thor­ough re­sponse. Oli already quoted the quick com­ments I posted from Nate and Eliezer last year, and I’ll chime in with some of the fac­tors that I think are lead­ing to dis­agree­ments about take­off:

  • Some MIRI peo­ple (Nate is one) sus­pect we might already be in hard­ware over­hang mode, or closer to that point than some other re­searchers in the field be­lieve.

  • MIRI folks tend to have differ­ent views from Paul about AGI, some of which im­ply that AGI is more likely to be novel and de­pen­dent on new in­sights. (Un­fair car­i­ca­ture: Imag­ine two peo­ple in the early 20th cen­tury who don’t have a tech­ni­cal un­der­stand­ing of nu­clear physics yet, try­ing to ar­gue about how pow­er­ful a nu­clear-chain-re­ac­tion-based bomb might be. If one side were to model that kind of bomb as “sort of like TNT 3.0” while the other is mod­el­ing it as “sort of like a small Sun”, they’re likely to dis­agree about whether nu­clear weapons are go­ing to be a small v. large im­prove­ment over TNT. Note I’m just us­ing nu­clear weapons as an anal­ogy, not giv­ing an out­side-view ar­gu­ment “some­times tech­nolo­gies are dis­con­tin­u­ous, ergo AGI will be dis­con­tin­u­ous”.)

  This list isn’t at all in­tended to be suffi­ciently-de­tailed or ex­haus­tive.

  I’m hop­ing we have time to write up more thoughts on this be­fore too long, be­cause this is an im­por­tant is­sue (even given that we’re try­ing to min­i­mize the re­searcher time we put into things other than ob­ject-level de­con­fu­sion re­search). I don’t want MIRI to be a blocker on other re­searchers mak­ing progress on these is­sues, though — it would be bad if peo­ple put a pause on hash­ing out take­off is­sues for them­selves (or put a pause on al­ign­ment re­search that’s re­lated to take­off views) un­til Eliezer had time to put out a blog post. I pri­mar­ily wanted to make sure peo­ple know that the lack of a sub­stan­tive re­sponse doesn’t mean that Nate+Eliezer+Benya+etc. agree with Paul on take­off is­sues now, or that we don’t think this dis­agree­ment mat­ters. Our tar­di­ness is be­cause of op­por­tu­nity costs and be­cause our views have a lot of pieces to ar­tic­u­late.

• Rob Bensinger 11 May 2019 20:21 UTC

  2 points

  in reply to: Said Achmiz's comment on: Rob B’s Short­form Feed

  Fan­tas­tic!

• Rob Bensinger 11 May 2019 20:18 UTC

  2 points

  in reply to: jimrandomh's comment on: Rob B’s Short­form Feed

  That counts! :) Part of why I’m ask­ing is in case we want to build a proper LW glos­sary, and Ra­tion­al­ity Car­di­nal­ity could at least provide ideas for terms we might be miss­ing.

• Rob Bensinger 10 May 2019 23:19 UTC

  4 points

  in reply to: Rob Bensinger's comment on: Rob B’s Short­form Feed

  Are there any other OK-qual­ity ra­tio­nal­ist glos­saries out there? https://​​wiki.less­wrong.com/​​wiki/​​Jar­gon is the only one I know of. I vaguely re­call there be­ing one on http://​​www.bayra­tional­ity.com/​​ at some point, but I might be mis­re­mem­ber­ing.

• Rob Bensinger 10 May 2019 23:13 UTC

  7 points

  on: Rob B’s Short­form Feed

  The wiki glos­sary for the se­quences /​ Ra­tion­al­ity: A-Z ( https://​​wiki.less­wrong.com/​​wiki/​​RAZ_Glos­sary ) is up­dated now with the glos­sary en­tries from the print edi­tion of vol. 1-2.

  New en­tries from Map and Ter­ri­tory:

  an­throp­ics, availa­bil­ity heuris­tic, Bayes’s the­o­rem, Bayesian, Bayesian up­dat­ing, bit, Blue and Green, cal­ibra­tion, causal de­ci­sion the­ory, cog­ni­tive bias, con­di­tional prob­a­bil­ity, con­fir­ma­tion bias, con­junc­tion fal­lacy, de­on­tol­ogy, di­rected acyclic graph, elan vi­tal, Everett branch, ex­pected value, Fermi para­dox, fooza­l­ity, hind­sight bias, in­duc­tive bias, in­stru­men­tal, in­ten­tion­al­ity, iso­mor­phism, Kol­mogorov com­plex­ity, like­li­hood, max­i­mum-en­tropy prob­a­bil­ity dis­tri­bu­tion, prob­a­bil­ity dis­tri­bu­tion, statis­ti­cal bias, two-box­ing

  New en­tries from How to Ac­tu­ally Change Your Mind:

  af­fect heuris­tic, causal graph, cor­re­spon­dence bias, episte­mol­ogy, ex­is­ten­tial risk, fre­quen­tism, Friendly AI, group se­lec­tion, halo effect, hu­mil­ity, in­tel­li­gence ex­plo­sion, joint prob­a­bil­ity dis­tri­bu­tion, just-world fal­lacy, koan, many-wor­lds in­ter­pre­ta­tion, mod­esty, transhuman

  A bunch of other en­tries from the M&T and HACYM glos­saries were already on the wiki; most of these have been im­proved a bit or made more con­cise.

Rob B’s Short­form Feed

• Rob Bensinger 6 May 2019 6:02 UTC

  6 points

  AF

  in reply to: rohinmshah's comment on: Align­ment Newslet­ter One Year Retrospective

  One op­tion that’s smaller than link posts might be to men­tion in the AF/​LW ver­sion of the newslet­ter which en­tries are new to AIAF/​LW as far as you know; or make com­ment threads in the newslet­ter for those en­tries. I don’t know how use­ful these would be ei­ther, but it’d be one way to cre­ate com­mon knowl­edge ‘this is cur­rently the one and only place to dis­cuss these things on LW/​AIAF’.

• Rob Bensinger 6 May 2019 1:54 UTC

  18 points

  in reply to: MakoYass's comment on: [Meta] Hid­ing nega­tive karma no­tifi­ca­tions by default

  Pos­si­ble com­pro­mise idea: send ev­ery­one their karma up­votes along with down­votes reg­u­larly, but send the up­votes in daily batches and the down­votes in monthly batches. Hav­ing your down­votes sent to you at known, pre­dictable times rather than in ran­dom bursts, and hav­ing the up­dates oc­cur less of­ten, might let users take in the rele­vant in­for­ma­tion with­out hav­ing it to­tally dom­i­nate their day-to-day ex­pe­rience of vis­it­ing the site. This also makes it eas­ier to spot pat­terns and to prop­erly dis­count very small aver­sive changes in vote to­tals.

  On the whole, I’m not sure how use­ful this would be as a sitewide de­fault. Some con­cerns:

  • It’s not clear to me that karma on its own is all that use­ful or con­tent­ful. Ray re­cently noted that a com­ment of his had got­ten down­voted some­what, and that this had been su­per salient and pointed feed­back for him. But I’m pretty sure that the ‘down­vote’ Ray was talk­ing about was ac­tu­ally just me turn­ing a strong up­vote into a nor­mal up­vote for minor /​ not-worth-in­de­pen­dently-track­ing rea­sons. Plenty of peo­ple vote for ob­scure or com­pli­cated or just-wrong rea­sons.

  • The peo­ple who get down­voted the most are likely to have less fa­mil­iar­ity with LW norms and con­text, so they’ll be es­pe­cially ill-equipped to ex­tract ac­tion­able in­for­ma­tion from down­votes. If all peo­ple are learn­ing is ‘<con­fus­ing noisy so­cial dis­ap­proval>’, I’m not sure that’s go­ing to help them very much in their jour­ney as a ra­tio­nal­ist.

  Upvotes tend to be a clearer sig­nal in my ex­pe­rience, while need­ing to meet a lower bar. (Cf.: we have a higher epistemic bar for es­tab­lish­ing a norm ‘let’s start in­sult­ing/​crit­i­ciz­ing/​call­ing out our col­leagues when­ever they make a mis­take’ than for es­tab­lish­ing a norm ‘let’s start com­pli­ment­ing/​prais­ing/​thank­ing our col­leagues when­ever they do some­thing cool’, and it would be odd to say that the lat­ter is cat­e­gor­i­cally bad in any en­vi­ron­ment where we don’t also es­tab­lish the former norm.)

  I’m not con­fi­dent of what the right an­swer is; this is just me lay­ing out some counter-con­sid­er­a­tions. I like Mako’s com­ment be­cause it’s ad­vo­cat­ing for an im­por­tant value, and ex­press­ing a not-ob­vi­ously-wrong con­cern about that value get­ting com­pro­mised. I lean to­ward ‘don’t make down-votes this salient’ right now. I’d like more clar­ity in­side my head about how much the down­vote-hid­ing worry is shaped like ‘we need to make down­votes more salient so we can ac­tu­ally get the im­por­tant in­tel­lec­tual work done’ vs. ‘we need to make down­votes more salient so we can bet­ter sym­bol­ize/​re­sem­ble Ra­tion­al­ity’.

• Rob Bensinger 3 May 2019 5:06 UTC

  5 points

  in reply to: davelaing's comment on: Open Thread May 2019

  ! Hi! I am a bi­ased MIRI per­son, but I quite dig all the things you men­tioned. :)

• Rob Bensinger 2 May 2019 22:09 UTC

  7 points

  on: Habryka’s Short­form Feed

  I like this short­form feed idea!

• Rob Bensinger 1 May 2019 18:06 UTC

  4 points

  in reply to: Benito's comment on: Habryka’s Short­form Feed

  Yeah, strong up­vote to this point. Hav­ing an Ar­bital-style sys­tem where peo­ple’s prob­a­bil­ities aren’t promi­nently times­tamped might be the worst of both wor­lds, though, since it dis­cour­ages up­dat­ing and makes it look like most peo­ple never do it.

  I have an in­tu­ition that some­thing so­cially good might be achieved by see­ing high-sta­tus ra­tio­nal­ists treat ass num­bers as ass num­bers, brazenly as­sign wildly differ­ent prob­a­bil­ities to the same propo­si­tion week-by-week, etc., es­pe­cially if this is a ca­sual and in­ci­den­tal thing rather than be­ing the fo­cus of any blog posts or com­ments. This might work bet­ter, though, if the ear­lier prob­a­bil­ities van­ish by de­fault and only show up again if the user de­cides to high­light them.

  (Also, if a user re­peat­edly abuses this fea­ture to look a lot more ac­cu­rate than they re­ally were, this war­rants mod in­ter­ven­tion IMO.)

• Rob Bensinger 30 Apr 2019 23:36 UTC

  4 points

  in reply to: Rob Bensinger's comment on: Habryka’s Short­form Feed

  Also, if you do some­thing Ar­bital-like, I’d find it valuable if the in­ter­face en­courages peo­ple to keep up­dat­ing their prob­a­bil­ities later as they change. E.g., some (prefer­ably op­tional) way of track­ing how your view has changed over time. Prob­a­bly also make it easy for peo­ple to re-vote with­out check­ing (and get­ting an­chored by) their old prob­a­bil­ity as­sign­ment, for peo­ple who want that.

• Rob Bensinger 30 Apr 2019 23:35 UTC

  4 points

  in reply to: habryka's comment on: Habryka’s Short­form Feed

  One small thing you could do is to have prob­a­bil­ity tools be col­lapsed by de­fault on any AIAF posts (and maybe even on the LW ver­sions of AIAF posts).

  Also, maybe some­one should write a blog post that’s a canon­i­cal refer­ence for ‘the rele­vant risks of us­ing prob­a­bil­ities that haven’t already been writ­ten up’, in ad­vance of the fea­ture be­ing re­leased. Then you could just link to that a bunch. (Maybe even in­clude it in the post that ex­plains how the prob­a­bil­ity tools work, and/​or link to that post from all in­stances of the prob­a­bil­ity tool.)

  Another idea: Ar­bital had a mix of (1) ‘spe­cial­ized pages that just in­clude a sin­gle prob­a­bil­ity poll and noth­ing else’; (2) ‘pages that are mainly just about list­ing a ton of prob­a­bil­ity polls’; and (3) ‘pages that have a bunch of other con­tent but in­ci­den­tally in­clude some prob­a­bil­ity polls’.

  If prob­a­bil­ity polls on LW mostly looked like 1 and 2 rather than 3, then that might make it eas­ier to dis­t­in­guish the parts of LW that should be very prob­a­bil­ity-fo­cused from the parts that shouldn’t. I.e., you could avoid adding Ar­bital’s fea­ture for eas­ily em­bed­ding prob­a­bil­ity polls in ar­bi­trary posts (and/​or ar­bi­trary com­ments), and in­stead treat this more as a dis­tinct kind of page, like ‘Ques­tions’.

  You could still link to the ‘Prob­a­bil­ity’ pages promi­nently in your post, but the re­duced promi­nence and site sup­port might cause there to be less so­cial pres­sure for peo­ple to avoid writ­ing/​post­ing things out of fears like ‘if I don’t provide prob­a­bil­ity as­sign­ments for all my claims in this blog post, or don’t add a prob­a­bil­ity poll about some­thing at the end, will I be seen as a Bad Ra­tion­al­ist?’

• Rob Bensinger 30 Apr 2019 23:17 UTC

  2 points

  in reply to: habryka's comment on: Habryka’s Short­form Feed

  I’ve never checked my karma to­tal on LW 2.0 to see how it’s changed.

• Rob Bensinger 28 Apr 2019 3:40 UTC

  5 points

  in reply to: habryka's comment on: Habryka’s Short­form Feed

  I am most wor­ried that this will dras­ti­cally in­crease the clut­ter of com­ment threads and make things a lot harder to parse. In par­tic­u­lar if the or­der of the re­acts is differ­ent on each com­ment, since then there is no re­li­able way of scan­ning for the differ­ent kinds of in­for­ma­tion.

  I like the re­ac­tions UI above, partly be­cause sep­a­rat­ing it from karma makes it clearer that it’s not chang­ing how com­ments get sorted, and partly be­cause I do want ‘agree’/​‘dis­agree’ to be non-anony­mous by de­fault (un­like nor­mal karma).

  I agree that the or­der of re­acts should always be the same. I also think ev­ery com­ment/​post should dis­play all the re­acts (even just to say ‘0 Agree, 0 Disagree...‘) to keep things uniform. That means I think there should only be a few per­mit­ted re­acts—maybe start with just ‘Agree’ and ‘Disagree’, then wait 6+ months and see if users are es­pe­cially clam­ber­ing for some­thing ex­tra.

  I think the ob­vi­ous other re­acts I’d want to use some­times are ‘agree and down­vote’ + ‘dis­agree and up­vote’ (maybe shorten to Agree+Down, Disagree+Up), since oth­er­wise some­one might not re­al­ize that one and the same per­son is do­ing both, which loses a fair amount of this thing I want to be fluidly able to sig­nal. (I don’t think there’s much value to clearly sig­nal­ing that the same per­son agreed and up­voted or dis­agree and down­voted a thing.)

  I would also some­times click both the ‘agree’ and ‘dis­agree’ but­tons, which I think is fine to al­low un­der this UI. :)

• Rob Bensinger 27 Apr 2019 20:38 UTC

  8 points

  in reply to: Vladimir_Nesov's comment on: Speak­ing for my­self (re: how the LW2.0 team com­mu­ni­cates)

  *dis­agrees with and ap­proves of this rele­vant, in­ter­est­ing, and non-con­fused com­ment*