jacobjacob

Pre­dic­tion as coordination

• jacobjacob 23 Jul 2019 4:23 UTC

  2 points

  in reply to: Pattern's comment on: ja­cob­ja­cob’s Short­form Feed

  If you’re ask­ing whether your para­phrase ac­tu­ally cap­tures my model, it doesn’t. If you’re mak­ing a point, I’m afraid I don’t get it.

• jacobjacob 23 Jul 2019 3:16 UTC

  11 points

  on: ja­cob­ja­cob’s Short­form Feed

  I’m con­fused about the effects of the in­ter­net on so­cial groups. [1]

  On the one hand… the in­ter­net en­ables much larger so­cial groups (e.g. red­dit com­mu­ni­ties with tens of thou­sands of mem­bers) and much larger cir­cles of so­cial in­fluence (e.g. in­sta­gram celebri­ties hav­ing mil­lions of fol­low­ers). Both of these will tend to dis­play net­work effects. How­ever, A) so­cial sta­tus is zero-sum, and B) there are diminish­ing re­turns to the health benefits of so­cial sta­tus [2]. This sug­gests that on the mar­gin mov­ing from a very large num­ber of small com­mu­ni­ties (e.g. bowl­ing clubs) to a smaller num­ber of larger com­mu­ni­ties (e.g. bowl­ing YouTu­bers) will be net nega­tive, be­cause the benefits of sta­tus gains at the top will diminish faster than the harms from losses at the me­dian.

  On the other hand… the in­ter­net en­ables much more niched so­cial groups. Ce­teris paribus, this sug­gests it should en­able more groups and higher qual­ity groups.

  I don’t know how to weigh these effects, but cur­renly ex­pect the former to be a fair bit larger.

  [1] In ad­di­tion to be­ing con­fused I’m also un­cer­tain, due to lack of data I could ob­tain in a few hours, prob­a­bly.

  [2] In con­trast to the psy­cholog­i­cal benefits of sta­tus, I think so­cial cap­i­tal can some­times have in­creas­ing re­turns. One mechanism: if you grow your net­work, the num­ber of con­nec­tions be­tween units in your net­work grows even faster.

• jacobjacob 23 Jul 2019 3:00 UTC

  13 points

  on: ja­cob­ja­cob’s Short­form Feed

  Had a good con­ver­sa­tion with eli­tyre to­day. Two nuggets of in­sight:

  • If you want to level up, find peo­ple who seem mys­te­ri­ously bet­ter than you at some­thing, and try to learn from them.

  • Work on pro­jects which are plau­si­bly cru­cial. That is, pro­jects such that if we look back from a win sce­nario, it’s not too im­plau­si­ble we’d say they were part of the rea­son.

ja­cob­ja­cob’s Short­form Feed

• jacobjacob 19 Jul 2019 18:56 UTC

  4 points

  in reply to: quanticle's comment on: When does adding more peo­ple re­li­ably make a sys­tem bet­ter?

  I’m not con­fi­dent that was ac­tu­ally the de­cline and shouldn’t have sounded so con­fi­dent in my post.

  Though your ex­pla­na­tion is con­fus­ing to me, be­cause it doesn’t ex­plain the data-point that LW ended up hav­ing a lot of bad con­tent and dis­cus­sion, rather than no con­tent and dis­cus­sion.

  Any­how, I be­lieve this dis­cus­sion should be had in the meta sec­tion of the site, and that we should fo­cus more on the ob­ject-level of the ques­tion here.

• jacobjacob 19 Jul 2019 18:52 UTC

  2 points

  in reply to: cousin_it's comment on: When does adding more peo­ple re­li­ably make a sys­tem bet­ter?

  My claim is definitely not about the global fi­nan­cial sys­tem. It’s about sin­gle fi­nan­cial mar­kets be­com­ing more ac­cu­rate at track­ing the met­rics they’re mea­sur­ing as more peo­ple join, by de­fault.

  If I be­came con­vinced that the proper anal­ogy is that com­pa­nies by de­fault be­come bet­ter at op­ti­mis­ing profit as more em­ploy­ees join, I’d change my mind about the im­por­tance of pre­dic­tion/​fi­nan­cial mar­kets. But I’d bet strongly that that is not the case.

• jacobjacob 19 Jul 2019 18:49 UTC

  3 points

  in reply to: quanticle's comment on: When does adding more peo­ple re­li­ably make a sys­tem bet­ter?

  Don’t think I dis­agree, I’ve made a very similar point to yours in a pre­vi­ous LW thread here.

  Also, my point is not that the gains from be­ing a cor­rect con­trar­ian in the fi­nan­cial mar­ket always out­weigh the so­cial pun­ish­ment for con­trar­i­anism, or that you can always trade be­tween the two cur­ren­cies. But de­spite be­ing frozen out of in­vest­ing, Michael Burry is still a multi-mil­lion­aire. That is an in­ter­est­ing ob­ser­va­tion. It’s re­lated to why I think Robin Han­son is ex­cited about pre­dic­tion mar­kets—they pre­sent a re­mark­able de­gree of ro­bust­ness to so­cial sta­tus games and ma­nipu­la­bil­ity.

  ___

  Also I’m very cu­ri­ous about the out­come of Taleb’s in­vest­ments (some peo­ple say they’re go­ing awfully, which is why he’s sel­l­ing books...), so please share any links.

[Question] When does adding more peo­ple re­li­ably make a sys­tem bet­ter?

• jacobjacob 19 Jul 2019 3:38 UTC

  6 points

  on: A Key Power of the Pres­i­dent is to Co­or­di­nate the Ex­e­cu­tion of Ex­ist­ing Con­crete Plans

  I’d like to coin a new term for that thing which the US Pres­i­dent has a lot of: co­or­di­na­tion cap­i­tal.

  This seems to re­quire some com­bi­na­tion of:

  • trust

  • long-term stability

  • Schel­ling point-y-ness

  • per­sonal con­nec­tions

  ____________________________________________________________________________________________

  Some prop­er­ties

  • Co­or­di­na­tion cap­i­tal is de­pre­ci­ated as it is used

  Con­sider the priest Kalil men­tions. He’s able to de­clare peo­ple mar­ried be­cause peo­ple think he is. It’s the equil­ibrium, and ev­ery­one benefits from main­tain­ing it. But if he tests his pow­ers and start declar­ing strange mar­riages not en­dorsed by the lo­cal so­cial norm, the equil­ibrium might shift. Similarly, if the pres­i­dent tries to rally com­pa­nies around a stag hunt, but does so poorly and some choose rab­bit, they’re all more lik­ley to choose rab­bit in fu­ture.

  • There are re­turns-to-scale to co­or­di­na­tion cap­i­tal

  The more plan ex­e­cu­tions you suc­cess­fully co­or­di­nate, the more will­ing fu­ture pro­jects will be to ap­proach you with their plans.

  • There is an up­per bound to the amount of co­or­di­na­tion capital

  If you have a Schel­ling co­or­di­na­tion point, and some­one finds it bad and de­clares they will build a new, bet­ter co­or­di­na­tion point, there is risk that you’ll end up not with two but with zero co­or­di­na­tion points. Similarly, co­or­di­na­tion cap­i­tal is scarce and it can re­sult in lock-in sce­nar­ios if held by the wrong en­tities.

  ____________________________________________________________________________________________

  Back­ground and implications

  Part of the rea­son I want a term for this thing is that I’ve been ex­pe­rienc­ing a lack of this thing when work­ing on co­or­di­na­tion in­fras­truc­ture for the EA and x-risk com­mu­ni­ties. I’m try­ing to build a fore­cast­ing plat­form and com­mu­nity to (among other things) build com­mon knowl­edge of some timelines con­sid­er­a­tions, to co­or­di­nate around them.

  How­ever, to get peo­ple to use it, I can’t just call up Holden Karnofsky, Nick Bostrom, and Nate Soares in or­der to kick­start the thing and make it a de facto Schel­ling point. Rather, I have to do some amount of “hus­tling”, and things that don’t scale—find­ing peo­ple in the com­mu­nity with nat­u­ral in­ter­est in stuff, reach­ing out to them per­son­ally, putting in leg­work here and there to keep dis­cus­sions go­ing and add a miss­ing piece to a quan­ti­ta­tive model… and try to do this enough to hit some kind of es­cape ve­loc­ity.

  I don’t have enough co­or­di­na­tion cap­i­tal, so I try to com­pen­sate by other means. Another ex­am­ple is Uber—they’re try­ing to move rid­ers and drivers to a new equil­ibrium, they didn’t have much co­or­di­na­tion cap­i­tal ini­tially, and this re­quires them to burn a lot cash/​free en­ergy.

  Writ­ing this I’m a bit wor­ried that all the lead­ers of the EA /​x-risk com­mu­ni­ties are lead­ers of par­tic­u­lar or­ga­ni­za­tions with an ob­ject-level mis­sion. They’re pri­mar­ily in­cen­tivised to achieve the or­gani­sa­tion’s mis­sion, and there is no one who, like the pres­i­dent, sim­ply serves to co­or­di­nate the com­mu­nity around the ex­e­cu­tion of plans. This sug­gests this func­tion might be un­der­util­ised on the mar­gin.

• jacobjacob 11 Jul 2019 2:00 UTC

  4 points

  in reply to: habryka's comment on: How can guessti­mates work?

  Nit­pick: “We’ve got­ten much bet­ter at mak­ing guessti­mates” and “Guessti­mates have be­come more effec­tive” are quite differ­ent claims, and it’s not clear which one(s) you dis­agree with.

[Question] How can guessti­mates work?

[Question] Can we use ideas from ecosys­tem man­age­ment to cul­ti­vate a healthy ra­tio­nal­ity memes­pace?

• jacobjacob 16 May 2019 20:49 UTC

  12 points

  on: Eight Books To Read

  [Epistemic sta­tus: this com­ment is much less clear in elu­ci­dat­ing the in­puts rather than out­puts of my think­ing than I would have preferred, but I share it writ­ten roughly rather than not at all.]

  On pri­ors, it would be in­cred­ibly sur­pris­ing to me if the best in­tro­duc­tion to learn­ing how to think about so­ciety did not in­clude any of the progress we’ve made in fields like microe­co­nomics and statis­tics (which only reached ma­tu­rity in the last 100 years or so), or even sim­ply em­piri­cism and quan­ti­ta­tive think­ing (which only reached ma­tu­rity in the last 500 years or so).

  I be­lieve there has been an ab­solutely out­stand­ing amount of gen­uine con­cep­tual and dis­til­la­tion progress in un­der­stand­ing so­ciety since An­cient Greece.

  Another part of my ex­pe­rience feed­ing into this prior is that my un­der­grad was in philos­o­phy at Oxford, and some pro­fes­sors re­ally liked deeply study­ing an­cient origi­nals and crit­i­cis­ing trans­la­tions. In my ex­pe­rience this mostly didn’t cor­re­late with a pro­duc­tive or healthy epistemic cul­ture.

• jacobjacob 14 May 2019 9:56 UTC

  6 points

  on: Un­der­stand­ing in­for­ma­tion cascades

  This is an up­date on the timeline for pay­ing out the boun­ties on this ques­tion. They will be awarded for work done be­fore May 13th, but we’re de­layed by an­other few weeks in de­cid­ing on the al­lo­ca­tion. Apolo­gies!

AI Fore­cast­ing on­line workshop

• jacobjacob 3 May 2019 1:46 UTC

  4 points

  on: Swarm AI (tool)

  The fact that they’re mea­sur­ing ac­cu­racy in a pretty bad way is ev­i­dence against them hav­ing a good al­gorithm.

  Here’s An­thony Aguirre (Me­tac­u­lus) and Ju­lia Galef on Ra­tion­ally Speak­ing.

  An­thony: On the re­sults side, there’s now an ac­crued track record of a cou­ple of hun­dred pre­dic­tions that have been re­solved, and you can just look at the num­bers. So, that shows that it does work quite well.

  Ju­lia: Oh, how do you mea­sure how well it works?

  An­thony: There’s a few ways — go­ing from the bad but easy to ex­plain, to the bet­ter but harder to ex­plain…

  Ju­lia: That’s a good pro­gres­sion.

  An­thony: And there’s the worst way, which I won’t even use — which is just to give you some ex­am­ples of great pre­dic­tions that it made. This I hate, so I won’t even do it.

  Ju­lia: Good for you for shun­ning that.

  An­thony: So look­ing over sort of the last half year or so, since De­cem­ber 1st, for ex­am­ple… If you ask for how many pre­dic­tions was Me­tac­u­lus on the right side of 50% — above 50% if it hap­pened or be­low 50% if it didn’t hap­pen — that hap­pens 77 out of 81 times the ques­tion re­solved, so that’s quite good.

  And some of the afi­ciona­dos will know about Brier scores. That’s sort of the fairly easy to un­der­stand way to do it, which is that you as­sign a zero if some­thing doesn’t hap­pen, and a one if some­thing does hap­pen. Then you take the differ­ence be­tween the pre­dicted prob­a­bil­ity and that num­ber. So if you pre­dict at 20% and it didn’t hap­pen, you’d take that as a .2, or if it’s 80% and it does hap­pen and that’s also a .2, be­cause it’s a differ­ence be­tween the 80% and a one, and then you square that num­ber.

  So Brier scores can run from ba­si­cally zero to one, where low num­bers are good. And if you calcu­late that for that same set of 80 ques­tions, it’s .072, which is a pretty good score.

• jacobjacob 4 Apr 2019 10:13 UTC

  2 points

  in reply to: rohinmshah's comment on: What are CAIS’ bold­est near/​medium-term pre­dic­tions?

  This is a pre­dic­tion I make, with “gen­eral-seem­ing” re­placed by “more gen­eral”, and I think of this as a pre­dic­tion in­spired much more by CAIS than by EY/​Bostrom.

  I no­tice I’m con­fused. My model of CAIS pre­dicts that there would be poor re­turns to build­ing gen­eral ser­vices com­pared to spe­cial­ised ones (though this might be more of a claim about eco­nomics than a claim about the na­ture of in­tel­li­gence).

• jacobjacob 29 Mar 2019 14:01 UTC

  12 points

  in reply to: jacobjacob's comment on: What are CAIS’ bold­est near/​medium-term pre­dic­tions?

  The fol­low­ing ex­change is also rele­vant:

  [-] Raiden 1y link 30

  Robin, or any­one who agrees with Robin:

  What ev­i­dence can you imag­ine would con­vince you that AGI would go FOOM?

  Re­ply[-] jprwg 1y link 22

  While I find Robin’s model more con­vinc­ing than Eliezer’s, I’m still pretty un­cer­tain.

  That said, two pieces of ev­i­dence that would push me some­what strongly to­wards the Yud­kowskian view:

  • A fairly con­fi­dent sci­en­tific con­sen­sus that the hu­man brain is ac­tu­ally sim­ple and ho­mo­ge­neous af­ter all. This could per­haps be the full blank-slate ver­sion of Pre­dic­tive Pro­cess­ing as Scott Alexan­der dis­cusse­drecently, or some­thing along similar lines.

  • Long-run data show­ing AI sys­tems grad­u­ally in­creas­ing in ca­pa­bil­ity with­out any in­crease in com­plex­ity. The AGZ ex­am­ple here might be part of an over­all trend in that di­rec­tion, but as a sin­gle data point it re­ally doesn’t say much.

  Re­ply[-] RobinHan­son 1y link 23

  This seems to me a rea­son­able state­ment of the kind of ev­i­dence that would be most rele­vant.

• jacobjacob 29 Mar 2019 13:11 UTC

  6 points

  in reply to: rohinmshah's comment on: What are CAIS’ bold­est near/​medium-term pre­dic­tions?

  EY seems to have in­ter­preted AlphaGo Zero as strong ev­i­dence for his view in the AI-foom de­bate, though Han­son dis­agrees.

  EY:

  Show­ing ex­cel­lent nar­row perfor­mance *us­ing com­po­nents that look gen­eral* is ex­tremely sug­ges­tive [of a fu­ture sys­tem that can de­velop lots and lots of differ­ent “nar­row” ex­per­tises, us­ing gen­eral com­po­nents].

  Han­son:

  It is only broad sets of skills that are sug­ges­tive. Be­ing very good at spe­cific tasks is great, but doesn’t sug­gest much about what it will take to be good at a wide range of tasks. [...] The com­po­nents look MORE gen­eral than the spe­cific prob­lem on which they are ap­plied, but the ques­tion is: HOW gen­eral over­all, rel­a­tive to the stan­dard of achiev­ing hu­man level abil­ities across a wide scope of tasks.

  It’s some­what hard to hash this out as an ab­solute rather than con­di­tional pre­dic­tion (e.g. con­di­tional on there be­ing break­throughs in­volv­ing some do­main-spe­cific hacks, and ma­jor labs keep work­ing on them, they will some­what quickly su­per­seded by break­throughs with gen­eral-sem­ing ar­chi­tec­tures).

  Maybe EY would be more bullish on Star­craft with­out imi­ta­tion learn­ing, or AlphaFold with only 1 or 2 mod­ules (rather than ⁴⁄₅ or ⁸⁄₉ de­pend­ing on how you count).