Davidmanheim

• Davidmanheim 3 Dec 2019 8:46 UTC

  2 points

  in reply to: Ian David Moss's comment on: Up­dat­ing a Com­plex Men­tal Model—An Ap­plied Elec­tion Odds Example

  Sorry, I should have clar­ified that the news was US GDP Growth: https://​​www.bea.gov/​​news/​​2019/​​gross-do­mes­tic-product-third-quar­ter-2019-sec­ond-es­ti­mate-cor­po­rate-prof­its-third-quar­ter

• Davidmanheim 28 Nov 2019 10:57 UTC

  7 points

  on: Spaghetti Towers

  This idea has be­come part of my con­cep­tual toolkit for dis­cussing /​ de­scribing a key failure mode.

  (Note: the be­low part of the com­ment is from my Face­book com­ment about the ar­ti­cle when it came out.)

  There’s a great con­nec­tion you make to bu­reau­cracy, and it’s definitely worth ex­plor­ing.

  This gives me a good lan­guage to dis­cuss some­thing I’ve noted a num­ber of times. I’d posit that se­lec­tion pres­sure for bu­reau­cracy limits how stupid the sys­tem gets as a func­tion of the best sim­ple al­ter­na­tive, and the difficulty of tran­si­tion­ing to it with­out turn­ing off the sys­tem. This means that for crit­i­cal sys­tems where there is no in­cre­men­tal path­way to im­prove, it’s near-per­ma­nent even if there are bet­ter al­ter­na­tives—see the US health­care sys­tem. For less crit­i­cal sys­tems, once an al­ter­na­tive is found, as long as the tran­si­tion isn’t too long/​too ex­pen­sive, and there are in­cen­tives that ac­tu­ally pro­mote effi­ciency, it will hap­pen. The crit­i­cal fact is that the in­cen­tives need to ex­ist for ev­ery­one that is in­volved—not just the end user. So if bob in ac­count­ing doesn’t like the change, un­less some­one else can in­duce co­op­er­a­tion (like se­nior man­age­ment,) it never hap­pens.

• Davidmanheim 28 Nov 2019 10:50 UTC

  10 points

  on: On Do­ing the Improbable

  This post has in­fluenced my eval­u­a­tions of what I am do­ing in prac­tice by forc­ing me to con­sider low­er­ing the bar for ex­pected suc­cess for high re­turn ac­tivi­ties. De­spite “know­ing” about how to shut up and mul­ti­ply, and need­ing to ex­pect a high failure rate if tak­ing rea­son­able lev­els of risk, I didn’t con­sciously place enough weight on these. This helped move me more in that di­rec­tion, which has led to both an in­creased num­ber of failures to get what I hoped, and a num­ber of mostly un­ex­pected suc­cesses when ap­ply­ing for /​ re­quest­ing /​ at­tempt­ing things.

  It is worth not­ing that I still need to work on the re­ac­tion I have to failing at these low cost, high-risk ac­tivi­ties. I some­times have a sig­nifi­cant emo­tional re­ac­tion to failing, which is es­pe­cially prob­le­matic be­cause the emo­tional re­ac­tion to failing at a long-shot can in­fluence my mood for mul­ti­ple days or weeks af­ter­wards.

• Davidmanheim 28 Nov 2019 10:43 UTC

  4 points

  on: The Costly Co­or­di­na­tion Mechanism of Com­mon Knowledge

  Un­til see­ing this post, I did not have a clear way of talk­ing about com­mon knowl­edge. De­spite un­der­stand­ing the con­cept fairly well, this post made the points more clearly than I had seen them made be­fore, and pro­vided a use­ful refer­ence when talk­ing to oth­ers about the is­sue.

  What links here?

  • The Re­view Phase: Helping LessWrongers Eval­u­ate Old Posts by Ben Pace (9 Dec 2019 0:54 UTC; 52 points)

• Davidmanheim 28 Nov 2019 10:42 UTC

  6 points

  on: A LessWrong Crypto Autopsy

  This post has been a clear ex­am­ple of how ra­tio­nal­ity has and has not worked in prac­tice. It is also a sub­ject of crit­i­cal prac­ti­cal im­por­tance for fu­ture de­ci­sions, so it fre­quently oc­curs to me as a use­ful ex­am­ple of how and why ra­tio­nal­ity does and does not help with (in ret­ro­spect) crit­i­cal de­ci­sions.

• Davidmanheim 28 Nov 2019 10:40 UTC

  LW: 10 AF: 3

  AF

  on: Embed­ded Agents

  This post has sig­nifi­cant changed my men­tal model of how to un­der­stand key challenges in AI safety, and also given me a clearer un­der­stand­ing of and lan­guage for de­scribing why com­plex game-the­o­retic challenges are poorly speci­fied or un­der­stood. The terms and con­cepts in this se­ries of posts have be­come a key part of my ba­sic in­tel­lec­tual toolkit.

Up­dat­ing a Com­plex Men­tal Model—An Ap­plied Elec­tion Odds Example

• Davidmanheim 27 Nov 2019 12:50 UTC

  LW: 11 AF: 6

  AF

  in reply to: Donald Hobson's comment on: A test for sym­bol ground­ing meth­ods: true zero-sum games

  I don ’t think this is straight­for­ward in prac­tice—and putting a carte­sian bound­ary in place is avoid­ing ex­actly the key prob­lem. Any fea­ture of the world used as the item to min­i­mize/​max­i­mize is mea­sured, and un­cor­rupt­able mea­sure­ment sys­tems seems like a non-triv­ial prob­lem. For in­stance, how do I get my GAI to max­i­mize blue in an area in­stead of max­i­miz­ing the blue in­put into their sen­sor when pointed at that area? We need to es­sen­tially solve value load­ing and un­der­stand a bunch of em­bed­ded agent is­sues to re­ally talk about this.

• Davidmanheim 24 Nov 2019 10:03 UTC

  2 points

  in reply to: Raemon's comment on: Towards op­ti­mal play as Villager in a mixed game

  There is also over­head to scal­ing and difficulty al­ign­ing goals that they want to avoid. (As above, I think my Rib­bon­farm post makes this clear.) Once you get big­ger, the only way to en­sure al­ign­ment is to mon­i­tor—trust, but ver­ify. And ver­ifi­ca­tion is a large part of why man­age­ment is so costly—it takes time away from ac­tu­ally do­ing work, it is pure over­head for the man­ager, and even then, it’s not foolproof.

  When you’re small, on the other hand, high-trust is al­most un­nec­es­sary, be­cause the en­tire org is leg­ible, and you can see that ev­ery­one is (or isn’t) buy­ing in to the goals. In typ­i­cal star­tups, they are also well al­igned be­cause they all have similar lev­els of pay­off if things go re­ally well.

• Davidmanheim 24 Nov 2019 9:57 UTC

  4 points

  in reply to: Raemon's comment on: Towards op­ti­mal play as Villager in a mixed game

  My claim is that *com­pe­tence* isn’t the crit­i­cal limit­ing fac­tor in most cases be­cause struc­ture doesn’t usu­ally al­low de­cou­pling, not that it’s not limited. When it IS the limit­ing fac­tor, I agree with you, but it rarely is. And I think al­ign­ment is a differ­ent ar­gu­ment.

  In EA orgs, al­ign­ment can solve the del­e­ga­tion-with­out-man­age­ment prob­lem be­cause it can miti­gate prin­ci­pal-agent is­sues. Once we agree on goals, we’re work­ing to­wards them, and we can do so in par­allel and co­or­di­nate only when needed. In most orgs, al­ign­ment can­not ac­com­plish this, be­cause it’s hard to get peo­ple to per­son­ally buy into your goals when those goals are profit max­i­miza­tion for a com­pany. (In­stead, you use in­cen­tive struc­tures like bonuses to al­ign them. But then you need to mon­i­tor them, etc.)

• Davidmanheim 20 Nov 2019 19:57 UTC

  6 points

  in reply to: Raemon's comment on: Towards op­ti­mal play as Villager in a mixed game

  On your points about scal­ing, I mostly agree, but want to note that there are fun­da­men­tal is­sues with scal­ing that I ex­plained in a post here: https://​​www.rib­bon­farm.com/​​2016/​​03/​​17/​​go-cor­po­rate-or-go-home/​​

  The post is rather long. In short, how­ever, I don’t think that your King­dom metaphor works, be­cause large bu­reau­cra­cies are big *not* be­cause they have many mini-king­doms do­ing similar things in par­allel, but be­cause they need to spe­cial­ize and al­low cross-func­tional col­lab­o­ra­tion, which re­quires lots of man­age­ment.

• Davidmanheim 20 Nov 2019 16:14 UTC

  9 points

  in reply to: ryan_b's comment on: The new dot com bub­ble is here: it’s called on­line advertising

  I partly agree, but bur­den of proof is of­ten the wrong fram­ing for truth seek­ing.

  The ar­ti­cle pro­vides strong ev­i­dence that ads are in­effec­tive in cer­tain classes of cases, and that fact in turn pro­vides weaker ev­i­dence that ads are in­effec­tive more gen­er­ally. To sup­port Ak­shat’s skep­ti­cism that the re­sult gen­er­al­izes, we’d need to ev­i­dence or pri­ors that points to­wards ads be­ing differ­en­tially effec­tive de­pend­ing on the type—tar­geted key­words vs. brand-ad key­words, and brand pres­ence verus no brand pres­ence.

  In the first case, I’m some­what skep­ti­cal that the differ­ence be­tween tar­geted and brand key­words will be large. My prior for the sec­ond differ­ence is that there would be some differ­ence, as Gor­don ar­gued in an­other com­ment. I don’t know of any ev­i­dence in ei­ther di­rec­tion, but I haven’t looked. (The ac­tual re­sult doesn’t mat­ter to me ex­cept as an ex­er­cise in Bayesian rea­son­ing, but if it mat­ters to you or oth­ers, it’s plau­si­ble high VoI to search a bit. )

• Davidmanheim 20 Nov 2019 8:20 UTC

  4 points

  in reply to: ofer's comment on: Book Re­view: De­sign Prin­ci­ples of Biolog­i­cal Circuits

  It’s not just se­lec­tion effects on or­ganisms—it’s within or­ganisms. The ex­am­ples given are NOT fully un­der­stood, so (for ex­am­ple,) the bac­te­rial tran­scrip­tion net­work mo­tifs only con­tain the re­la­tion­ships that we un­der­stand. Given that loops and com­plex con­nec­tions are harder to de­tect, that has se­lected for sim­plic­ity.

  Given that, I still want to read the book and see the ar­gu­ment more fully.

• Davidmanheim 20 Nov 2019 8:12 UTC

  9 points

  in reply to: Dagon's comment on: The new dot com bub­ble is here: it’s called on­line advertising

  There was a bub­ble, and there is also sec­u­lar growth in the mar­ket, with a lot of churn that makes buy-and-hold a fairly bad idea. Those aren’t in­con­sis­tent. Here’s a graphic of the churn. Most of the early com­pa­nies died.

  But if you put all your money in the hot IPOs of Netscape, Ya­hoo, Ly­cos, and Ex­cite in 1995, you’d have done very poorly. If you ex­tended this to 1996, you could add Mind­spring and Check­point (the only one that did well, so far, which is up 29x, for a 16% an­nu­al­ized re­turn to date.) It took un­til 1997 to get any long-term fan­tas­tic re­turn, for Ama­zon—which is up 1000x since 1997, or a 37% an­nual re­turn—fan­tas­tic, but if you were pre­scient, and it was an en­tire tenth of your port­fo­lio, on av­er­age you did just OK. Skip­ping ahead to 1999-2000, the height of the bub­ble, here’s the list. Noth­ing made big bucks.

  So we can con­struct a port­fo­lio with 10 stocks, 8 of which went bust, and 2 of which, check­point and Ama­zon, did well. Your com­pound 22-year re­turn? 5.25% (And if you bought an S&P in­dex fund in 1998 at 1,000, you’d have made 6.5% an­nu­ally.)

• Davidmanheim 20 Nov 2019 7:52 UTC

  4 points

  in reply to: G Gordon Worley III's comment on: The new dot com bub­ble is here: it’s called on­line advertising

  Agreed on both points.

• Davidmanheim 19 Nov 2019 15:32 UTC

  2 points

  on: Au­mann Agree­ment Game

  For those in­ter­ested in run­ning the game, I put to­gether a Python script that pulls ques­tions from an API for trivia ques­tions, and out­puts 2 LaTeX files that can be com­piled and used, one for the quiz­mas­ter, plus a set of cor­re­spond­ing cut-out cards for the play­ers.

  https://​​github.com/​​david­man­heim/​​Au­mann_Game_Printouts

  Pull re­quests and im­prove­ment sug­ges­tions are wel­come!

• Davidmanheim 19 Nov 2019 5:52 UTC

  14 points

  on: The new dot com bub­ble is here: it’s called on­line advertising

  To this I’ll just add that this prob­lem is some­what solv­able, but it’s tricky.

  
  

  This is a very im­por­tant point. I will self-pro­mote and men­tion my pre-print pa­per on met­ric de­sign and avoid­ing Good­hart­ing (not in the con­text of AI): https://​​mpra.ub.uni-muenchen.de/​​90649/​​1/​​MPRA_pa­per_90649.pdf

  Ab­stract: Met­rics are use­ful for mea­sur­ing sys­tems and mo­ti­vat­ing be­hav­iors. Un­for­tu­nately, naive ap­pli­ca­tion of met­rics to a sys­tem can dis­tort the sys­tem in ways that un­der­mine the origi­nal goal. The prob­lem was noted in­de­pen­dently by Camp­bell and Good­hart, and in some forms it is not only com­mon, but un­avoid­able due to the na­ture of met­rics. There are two dis­tinct but in­ter­re­lated prob­lems that must be over­come in build­ing bet­ter met­rics; first, spec­i­fy­ing met­rics more closely re­lated to the true goals, and sec­ond, pre­vent­ing the re­cip­i­ents from gam­ing the differ­ence be­tween the re­ward sys­tem and the true goal. This pa­per de­scribes sev­eral ap­proaches to de­sign­ing met­rics, be­gin­ning with de­sign con­sid­er­a­tions and pro­cesses, then dis­cussing spe­cific strate­gies in­clud­ing se­crecy, ran­dom­iza­tion, di­ver­sifi­ca­tion, and post-hoc speci­fi­ca­tion. Fi­nally, it will dis­cuss im­por­tant desider­ata and the trade-offs in­volved in each ap­proach.

  (Cur­rently work­ing on a rewrite, but feed­back on the ideas and any­thing miss­ing is es­pe­cially ap­pre­ci­ated.)

• Davidmanheim 6 Nov 2019 13:23 UTC

  2 points

  in reply to: Ruby's comment on: Theater Tick­ets, Sleep­ing Pills, and the Idiosyn­crasies of Del­e­gated Risk Management

  Yeah, I could see giv­ing sleep­ing pills as a phe­nomenon a differ­ent name, but (while I am un­sure,) in the in­dus­try I think the term is well known, so I’m stick­ing with it.

  Still, I’m open to bet­ter sug­ges­tions, but sugar pill /​ placebo doesn’t seem bet­ter, since it’s much less clear as an anal­ogy.

• Davidmanheim 6 Nov 2019 13:21 UTC

  2 points

  in reply to: Dagon's comment on: Theater Tick­ets, Sleep­ing Pills, and the Idiosyn­crasies of Del­e­gated Risk Management

  Ab­solutely true—they are slightly less re­lated to prin­ci­ple agent prob­lems, but I can add sto­ries illus­trat­ing these, be­cause I have some good ones.

  Re: Risk dis­tri­bu­tion, there’s a great game that some­one sug­gested to un­der­stand what 100 year floods look like, since peo­ple com­plain they hap­pen ev­ery year. Get a room full of, say, 40 peo­ple, and give them each 2 dice. Tell them snake-eyes is a flood, then have ev­ery­one roll, and to call out if there was a flood. Then roll and an­nounce again. And again. Yes, many “years” will have no floods, but some will have 2 or even 3 - be­cause the risk isn’t highly cor­re­lated across ar­eas.

  Re: Thresh­old of mis­ery, I knew an un­der­writer who said they would write New York City nu­clear ter­ror­ism in­surance when­ever they had the chance, at ba­si­cally any price. His rea­son­ing? He lives and works in NYC, so if a nu­clear bomb goes off, he’s dead, and the fact that he lost money isn’t what mat­ters. Other­wise he col­lects pre­miums.

• Davidmanheim 30 Oct 2019 10:38 UTC

  13 points

  on: Prospect­ing for Con­cep­tual Holes

  It’s worth not­ing that these types ba­si­cally match the Jo­hari Win­dow for types of risks; known knowns, known un­knowns, un­known knowns, and un­known un­knowns. This is be­cause, at least ac­cord­ing to one defi­ni­tion of the term, risks are things we don’t ex­pect. Given that defi­ni­tion, a risk is the product of a con­cep­tual hole of some sort—and so the two have a some­what triv­ial map­ping.