Benito

• Benito 25 Jun 2019 18:53 UTC

  7 points

  in reply to: Benito's comment on: The Hacker Learns to Trust

  When Robin Han­son is in­ter­viewed about The Elephant in the Brain, he is of­ten asked “Are you say­ing that sta­tus ac­counts for all of our be­havi­our?”. His re­ply is that he+Kev­inSim­ler aren’t ar­gu­ing that the hid­den mo­tives are the only mo­tive, but that they’re a far more com­mon mo­tive than we give credit for in our nor­mal dis­course. Here’s an ex­am­ple of him say­ing this kind of thing on the 80k pod­cast:

  As we just said the ex­am­ple that, in ed­u­ca­tion, your mo­tive isn’t to learn the ma­te­rial, or when you go to the doc­tor, your mo­tive isn’t to get well pri­mar­ily, and the hid­den mo­tives are the ac­tual mo­tive. Now, how could I know what the hid­den mo­tives are, you might ask? The plan here, that’s where the book is … In each area, we iden­tify the usual story, then we col­lect a set of puz­zles that don’t make sense from the point of view of the usual story, strange em­piri­cal pat­terns, and then we offer an al­ter­na­tive mo­tive that makes a lot more sense of those em­piri­cal pat­terns, and then we sug­gest that that is a stronger mo­tive than the one we usu­ally say.

  Now, just to be clear, al­most ev­ery area of hu­man life is com­pli­cated, and there’s a lot of peo­ple with a lot of differ­ent de­tails and so, of course, al­most ev­ery pos­si­ble mo­tive shows up in al­most ev­ery area of hu­man life, so we can’t be talk­ing about the only mo­tive, and so the usual mo­tive does ac­tu­ally ap­ply some­times. Ac­tu­ally, you could think of the anal­ogy to the ex­cuse that the dog ate my home­work. It only works be­cause some­times dogs eat home­work. We don’t say the dragon ate my home­work. That wouldn’t fly, so the usual story is part of the story. It’s just a smaller part than we like to ad­mit, and what we’re go­ing to call the hid­den mo­tive, the real mo­tive is a big­ger part of the story, but it’s still not the only part.

• Benito 25 Jun 2019 18:51 UTC

  23 points

  in reply to: 9eB1's comment on: The Hacker Learns to Trust

  I agree with a lot of claims in your com­ment, and I think it’s valuable to think through how sta­tus plays a role in many situ­a­tions, in­clud­ing this.

  There is an ap­proach in your com­ments to­ward ex­plain­ing some­one’s be­havi­our that I dis­agree with, though it may just be a ques­tion of em­pha­sis. A few ex­am­ples:

  My real model is that they took those ideas ex­tra se­ri­ously be­cause the peo­ple were nice and high sta­tus.

  
  

  ...a pre­req­ui­site for them chang­ing their mind was that they were taken se­ri­ously and en­gaged with respectfully

  Th­ese seem to me definitely true and si­mul­ta­neously not that im­por­tant*.

  When I read that es­say, at least half of it is heav­ily-laden with sta­tus con­cerns and psy­cholog­i­cal mo­ti­va­tions. But, to re­it­er­ate: though pro-so­cial com­mu­nity norms left this per­son open to hav­ing their mind changed by ar­gu­ment, prob­a­bly the ar­gu­ments still had to be made. (em­pha­sis added)

  The word ‘prob­a­bly’ in that sen­tence feels false to me. I feel some­what analo­gous to hear­ing some­one ar­gue that a suc­cess­ful tech startup is 100s of peo­ple work­ing to­gether in a com­pany, and that ba­si­cally run­ning a tech startup is about sta­tus and in­cen­tives, though “prob­a­bly code still had to be writ­ten” to make it suc­cess­ful. They’re both nec­es­sary.

  More gen­er­ally, there are two types of games go­ing on. One we’re al­lowed to talk about, and one we’re not, or at least not very di­rectly. And we have to co­or­di­nate on both lev­els to suc­ceed. This gen­er­ally warps how our words re­late to re­al­ity, be­cause we’re also us­ing those words to do things we’re pre­tend­ing to our­selves we’re not do­ing, to let ev­ery­one ex­press their prefer­ences and co­or­di­nate in the silent games. Th­ese silent games have real and cru­cial im­pli­ca­tions for how well we can co­or­di­nate and where re­sources must be spent. But once you re­al­ise the silent games are be­ing played, it isn’t the right move to say that the silent games are the only games, or always the pri­mary games.

  I think the fact that the model doesn’t ac­tu­ally work is an im­por­tant as­pect of this. Many hack­ers would have done it as a cool pro­ject and re­leased it with­out pomp, but this per­son put to­gether a long es­say, ex­plic­itly tout­ing the im­por­tance of what they’d done and the im­pact it would have on his­tory. Then, it turned out the model did not work, which must have been very em­bar­rass­ing. It is fairly rea­son­able to sug­gest that the per­son then took the ac­tion that made them feel the best about their legacy and sta­tus: writ­ing an es­say about why they were not re­leas­ing the model for good ra­tio­nal­ist ap­proved rea­sons. It is not even nec­es­sar­ily the case that the per­son is aware that this is in­fluenc­ing the de­ci­sion, this is a fully Elephant in the Brain situ­a­tion.

  Again, I agree that some­thing in this refer­ence class is likely hap­pen­ing. But, for ex­am­ple, the long es­say was not only about in­creas­ing the per­ceived im­por­tance of the ac­tion. It was also a strongly pro-so­cial and co­op­er­a­tive move to the broader AI com­mu­nity to al­low coun­ter­ar­gu­ments to be pre­sented, which is what suc­cess­fully hap­pened. There are mul­ti­ple mo­tives here, and (I think) it’s the case that the mo­tive you point to was not the main one, even while it is a silent mo­tive folks sys­tem­at­i­cally avoid dis­cussing.

  --

  *Ac­tu­ally I think that Con­nor in par­tic­u­lar would’ve en­gaged with ar­gu­ments even if they’d not been de­liv­ered re­spect­fully, given that he re­sponded sub­stan­tively to many com­ments on Twit­ter/​Hack­erNews/​Medium, some of which were pre­dom­i­nantly snark.

• Benito 24 Jun 2019 19:25 UTC

  2 points

  on: Min­i­mize Use of Stan­dard In­ter­net Food Delivery

  I just opened up UberEats, and the search bar’s prompt is the phrase “What Are You Crav­ing?”. This is a bad de­ci­sion-pro­ce­dure for food on a reg­u­lar ba­sis. It seems to me that the at­ten­tion-econ­omy is even at­tempt­ing to get me to make im­pulse food de­ci­sions, and never longer-term, con­sid­ered de­ci­sions.

• Benito 23 Jun 2019 17:29 UTC

  6 points

  in reply to: rohinmshah's comment on: No, it’s not The In­cen­tives—it’s you

  This seems re­lated to the ideas in this post on un­con­scious economies.

• Benito 23 Jun 2019 4:48 UTC

  6 points

  in reply to: Benito's comment on: The Hacker Learns to Trust

  Yeah, this is quite im­por­tant, the at­tempted copy was weaker than the nerfed model OpenAI ini­tially re­leased. Thanks for em­pha­sis­ing this 9eB1, I’ve up­dated my post in a few places ac­cord­ingly.

• Benito 22 Jun 2019 20:31 UTC

  8 points

  in reply to: 9eB1's comment on: The Hacker Learns to Trust

  it turned out the model did not work… It is fairly rea­son­able to sug­gest that the per­son then took the ac­tion that made them feel the best about their legacy and status

  Read­ing this I re­al­ise I de­vel­oped most of my at­ti­tudes to­ward the topic when I be­lieved that the copy was full-strength, and only in writ­ing the post did I find out that it wasn’t—in fact it seems that it was weaker than the ini­tial 117M ver­sion OpenAI re­leased. You’re right that this makes the ‘re­lease’ op­tion less ex­cit­ing from the per­spec­tive of one’s per­sonal sta­tus, which (the sta­tus lens) would then pre­dict tak­ing whichever differ­ent ac­tion would give more per­sonal sta­tus, and this is ar­guably one of those ac­tions.

  Just now I found this com­ment in the medium com­ment sec­tion, where Con­nor agrees with you about it be­ing sym­bolic, and men­tions how this af­fected his think­ing.

  ...I did ad­mit failure as I linked to said failure in the very first para­graph, and I have no in­ten­tions of hid­ing that. In fact, af­ter learn­ing of my failure I was con­vinced I might as well re­lease, since most safety is­sues were no longer a threat any­ways (though there re­mains the pos­si­bil­ity it could be used as a “warm start” to train a bet­ter model). So if any­thing, my failure en­couraged me to dump it, apol­o­gize and let his­tory take its course.

  My de­ci­sion not to re­lease is mostly sym­bolic. I’m do­ing it to sig­nal good faith co­op­er­a­tion. Even if I failed to­day, some day some­one will suc­ceed, and we should have a de­fault of co­op­er­a­tion be­fore that.

  (Meta: Wow, Medium re­quires you to click twice to go down one step in a com­ment thread! Turns out there are like 20 com­ments on the OP.)

• Benito 22 Jun 2019 19:53 UTC

  4 points

  in reply to: 9eB1's comment on: The Hacker Learns to Trust

  The phe­nomenon I was point­ing out wasn’t ex­actly that the per­son’s de­ci­sion was made be­cause of sta­tus. It was that a pre­req­ui­site for them chang­ing their mind was that they were tak­ing se­ri­ously and en­gaged with re­spect­fully.

  Yeah, re­spect­ful and se­ri­ous en­gage­ment with peo­ple’s ideas, even when you’re on the op­po­site sides of policy/​norm dis­putes, is very im­por­tant.

• Benito 22 Jun 2019 19:46 UTC

  4 points

  in reply to: rohinmshah's comment on: The Hacker Learns to Trust

  I’m imag­in­ing here some­thing like a policy of emailing OpenAI and tel­ling them your plan and offer­ing them as much time to talk as pos­si­ble, and say­ing that in a week you’ll pub­li­cly pub­lish your rea­son­ing too so that other peo­ple can re­spond + po­ten­tially change your mind. I also think it would’ve been quite rea­son­able to not ex­pect any re­sponse from a big or­gani­sa­tion like OpenAI, and to be do­ing it only out of cour­tesy.

  It seems from above that talk­ing to OpenAI didn’t change Con­nor’s mind, and that pub­lic dis­course was very use­ful. I ex­pect Buck would not have talked to him if he hadn’t done this pub­li­cly (I will ask Buck when I see him) (Added: Buck says this is true). Given the OP I don’t think it would’ve been able to re­solve pri­vately, and I think I am quite ac­tively happy that it has re­solved the way it has: Some­one pub­li­cly de­cid­ing to not unilat­er­ally break an im­por­tant new norm, even while they strongly be­lieve this par­tic­u­lar ap­pli­ca­tion of the norm is re­dun­dant/​un­helpful.

  I’d be in­ter­ested to know if you think that it would’ve been perfectly pro-so­cial to give OpenAI a week’s heads-up and then writ­ing your rea­son­ing pub­li­cly and read­ing ev­ery­one else’s cri­tiques (100% of ran­dom peo­ple from Hacker News and Twit­ter and longer chats with Buck). I have a sense that you wouldn’t but I’m not fully sure why.

• Benito 22 Jun 2019 7:13 UTC

  18 points

  in reply to: 9eB1's comment on: The Hacker Learns to Trust

  While I agree that this is an im­por­tant fac­tor when mod­el­ling peo­ple’s de­ci­sion-mak­ing, I think there is some straight­for­ward ev­i­dence that this was not the pri­mary fac­tor here.

  Firstly, af­ter the per­son spent an hour talk­ing to friendly and helpful peo­ple from the high-sta­tus com­pany, they did not change their de­ci­sion, which is ev­i­dence against most par­si­mo­nious of sta­tus-based mo­tives. (Re­lat­edly, there was not a small set of peo­ple the au­thor promised to read feed­back from, but liter­ally 100% of re­spon­dents, which is over-and-above what would be use­ful for get­ting the at­ten­tion of key peo­ple.)

  And sec­ondly, which is more per­sua­sive for me though harder to com­mu­ni­cate, I read the ex­ten­sive rea­sons for their de­ci­sions for do­ing so, and they seemed clear and well-rea­soned, and then the rea­sons against were im­por­tant fac­tors that are gen­uinely nu­anced and hard to no­tice. It seemed to me more of a situ­a­tion where some­one ac­tu­ally im­proves their un­der­stand­ing of the world than one in which they were wait­ing for cer­tain high-sta­tus-to-them peo­ple to give them at­ten­tion. My sense is that writ­ing that ex­plains some­one’s de­ci­sions that is wholly mo­ti­vated by sta­tus makes less sense than these two posts did.

  You might still be right and I might have missed some­thing, or just not have a cyn­i­cal enough prior. Though I do be­lieve peo­ple do some­times change their ac­tions due to good rea­son­ing about the world and not solely due to im­me­di­ate sta­tus con­sid­er­a­tions, and I feel very skep­ti­cal of any lens on the world that can’t (“As is always the case”) reg­ister a pos­i­tive re­sult on the ques­tion “Did this per­son make their de­ci­sion due to up­dat­ing their world model rather than short-sighted sta­tus-grab­bing?”.

  Am in­ter­ested to hear fur­ther thoughts of yours on the broader topic of mod­el­ling peo­ple’s de­ci­sion mak­ing as pri­mar­ily sta­tus based, if you have more things to add to the dis­cus­sion.

The Hacker Learns to Trust

Wel­come to LessWrong!

• Benito 2 Jun 2019 1:04 UTC

  2 points

  in reply to: Zvi's comment on: FB/​Dis­cord Style Reacts

  When you say so­cial me­dia—do you avoid us­ing them in team slack/​dis­cord?

Von Neu­mann’s cri­tique of au­tomata the­ory and logic in com­puter science

Ed Boy­den on the State of Science

• Benito 12 May 2019 19:44 UTC

  2 points

  on: Why books don’t work

  Still read­ing, but I cer­tainly got off the bus a bit with this para­graph (which seems largely false to me).

  Let’s be­gin by look­ing at text­books in prac­tice. It’s strik­ing that aca­demic courses are of­ten struc­tured around text­books, but lots of peo­ple spend the ex­tra time and money to en­roll in those courses—rather than just study­ing the text­books in­de­pen­dently. In­deed, I sus­pect that text­books are mostly pur­chased for course syl­labi, not for self-study. Sure: some peo­ple take courses be­cause they want a cre­den­tial. But plenty of stu­dents gen­uinely feel they’ll learn more by tak­ing courses than they would by study­ing those courses’ text­books. As­sum­ing stu­dents’ feel­ings aren’t com­pletely mis­placed, courses must be offer­ing some­thing ex­tra that’s im­por­tant to how peo­ple learn.

• Benito 3 May 2019 1:33 UTC

  9 points

  in reply to: davelaing's comment on: Open Thread May 2019

  Wel­come, and I hope you en­joyed read­ing ~2 mil­lion words of con­tent!

  Be­cause we’re mak­ing more. So much more.

  (And have fun with the po­ten­tial LW meetup :) )

• Benito 2 May 2019 22:15 UTC

  4 points

  on: Liter­a­ture Re­view: Distributed Teams

  Dat­a­point: Stripe’s Fifth Eng­ineer­ing Hub is Re­mote. HN dis­cus­sion.

• Benito 1 May 2019 1:03 UTC

  14 points

  in reply to: Rob Bensinger's comment on: Habryka’s Short­form Feed

  Note that Paul Chris­ti­ano warns against en­courag­ing slug­gish up­dat­ing by mas­sively pub­li­cis­ing peo­ple’s up­dates and judg­ing them on it. Not sure what im­ple­men­ta­tion de­tails this sug­gests yet, but I do want to think about it.

  https://​​side­ways-view.com/​​2018/​​07/​​12/​​epistemic-in­cen­tives-and-slug­gish-up­dat­ing/​​

• Benito 30 Apr 2019 21:35 UTC

  8 points

  in reply to: Zvi's comment on: Asym­met­ric Justice

  Pretty sure the only in­ter­est­ing thing here is twit­ter and how it puts differ­ent cul­tures with differ­ent ideas of what count as norm vi­o­la­tions into a big room with each other and how this doesn’t lead to tol­er­ance but in­stead leads to in­ter­minable anger and slap-downs, due to enough peo­ple think­ing their own norms are ‘ob­vi­ous’ and not ‘op­ti­mised for a par­tic­u­lar en­vi­ron­ment’. Friend groups and sci­en­tists and jour­nal­ists and busi­ness­peo­ple ap­ply­ing their ar­eas’ norms to each other 100% of the time? Ugh.

• Benito 29 Apr 2019 1:01 UTC

  3 points

  in reply to: habryka's comment on: Habryka’s Short­form Feed

  Do our karma karma no­tifi­ca­tions dis­ap­pear if you don’t check them that day? My model of Zvi sug­gested to me this is at­ten­tion-grab­bing and bad. I won­der if it’s bet­ter to let folks be no­tified of all days’ karma up­dates ’til their most re­cent check in, and maybe also see all his­tor­i­cal ones or­dered by date if they click on a fur­ther but­ton, so that the info isn’t lost and doesn’t feel scarce.