Vaniver

• Vaniver 21 Oct 2020 17:51 UTC

  36 points

  in reply to: AspiringRationalist’s comment on: A tale from Com­mu­nist China

  In gen­eral, I think peo­ple try to time mar­kets much more than they have skill for. Sup­pose you think there’s a stock bub­ble; the temp­ta­tion is to buy, hold as it rises, and then sell at the top of the bub­ble be­fore ev­ery­thing comes crash­ing down. But enough peo­ple are try­ing to do that that you need some spe­cial skill in or­der to be in the lead­ing edge, able to sell when there’s a buyer. It’s much less risky to just sell the stocks as soon as you think there’s a bub­ble, which fore­goes any ad­di­tional gains but means you avoid the crash en­tirely (by tak­ing it on vol­un­tar­ily, sort of).

  A similar thing hap­pened at the start of the pan­demic, where my plan was ba­si­cally “look, I don’t think I’m go­ing to be es­pe­cially good at the risk as­sess­ment, I want to just lock down now in­stead of stay­ing open to get the marginal few weeks of meet­ings or hang­outs or what­ever,” and var­i­ous other friends said “look, I be­lieve in math, I think it’s just para­noia to lock down im­me­di­ately in­stead of do­ing so based on a cost-benefit anal­y­sis.” None of us knew it at the time, but the offi­cial tests were faulty and de­layed and other test­ing was be­ing sup­pressed, and so the num­bers be­ing used for the cost-benefit analy­ses were sig­nifi­cantly un­der­es­ti­mat­ing the true amount of viral trans­mis­sion.

  Also rele­vant was at the start of the pan­demic, there were var­i­ous bor­der clo­sures and re­gional quaran­tines that by de­sign had as lit­tle warn­ing as pos­si­ble. Sup­pose Disease City has too many cases, and also peo­ple in Disease City would want to leave if they think they’re go­ing to get locked in (be­cause re­gard­less of whether or not they have the dis­ease, it’s bet­ter for them to be out than in), and the re­gional /​ na­tional gov­ern­ment wants to lock them in; then the gov­ern­ment want to im­pose the lock­down and then an­nounce it, since that min­i­mizes the num­ber of peo­ple who can flee.

  No­tice­ably, if there’s a rev­olu­tion in some coun­try, it’s much bet­ter for the new gov­ern­ment to mur­der all of the peo­ple who would want to leave than let them leave. Con­trast “Who, af­ter all, speaks to­day of the an­nihila­tion of the Ar­me­ni­ans?” with anti-Cuban poli­tics in the US be­ing driven by Cuban ex­pa­tri­ates who felt wronged by the gov­ern­ment they fled, or var­i­ous po­ten­tial lead­ers kept by other pow­ers as po­ten­tial pup­pets ready to be in­stalled when the time is right. If more ed­u­cated /​ cap­i­tal­ist Chi­nese had been al­lowed to leave China, and ended up in the US, it seems likely could have been a vot­ing bloc much like the Cuban vot­ing bloc, and im­pacted US-Chi­nese re­la­tions ac­cord­ingly.

  [As it hap­pens, most of the peo­ple that I know who speak of the an­nihila­tion of the Ar­me­ni­ans are them­selves part of the Ar­me­nian di­as­pora. Out of the var­i­ous geno­cides and purges I’m fa­mil­iar with, peo­ple mostly seem in­ter­ested in ones to peo­ple similar to them, and cu­ri­ously un­in­ter­ested in ones that are of their poli­ti­cal en­e­mies, or even of the poli­ti­cal en­e­mies of peo­ple they would like to be al­lied with.]

  In sum­mary, there’s no fire alarm for when to leave the coun­try, in part be­cause the situ­a­tion is ad­ver­sar­ial, and this isn’t the sort of thing you should ex­pect peo­ple to be well-cal­ibrated on, since they don’t have many ex­am­ples to learn from.

  ---

  I think there’s also a fac­tor where ‘the best time to plant a tree is 20 years ago, and the sec­ond best time is to­day.’ If you ex­pect to want to be in Canada in­stead of the US in 2030, say, then you might ex­pect to pre­fer the timeline where you move to Canada in 2020 and live there for ten years to the timeline where you live in the US for nine and a half years and then move to Canada the night be­fore the civil war breaks out (or what­ever), and then live there for six months. If the first, you’ll have been able to in­vest in lots of things that you ex­pect to stick around; in the sec­ond, you might skip mak­ing use­ful home im­prove­ments be­cause they aren’t mo­bile. [Similarly, imag­in­ing our Chi­nese in­tel­lec­tu­als in the 1950s, if they knew they would want to move to the US in 1965, they might have de­cided to get the move over with as soon as pos­si­ble, since then their chil­dren might have grown up speak­ing English, they might have been able to get po­si­tions at uni­ver­si­ties be­fore there was a flood of similar re­fugees, they might have been able to liqui­date Chi­nese as­sets at more fa­vor­able prices, and so on. Most of the benefit of de­lay­ing comes from the hope that ac­tu­ally you won’t have to move; this is why the illu­sory short-term pos­i­tive signs are one of the most im­por­tant parts of the OP.]

  This story is less ob­vi­ous when there’s a huge differ­ence in value be­tween the op­tions. If you’re mak­ing net $60k a year in the US and would make net $30k a year in Canada, or what­ever, each year you de­lay mov­ing dou­bles your effec­tive in­come. But if the differ­ence is smaller, or you’re weigh­ing differ­ent psy­cholog­i­cal benefits against each other, it’s prob­a­bly bet­ter to get it over with than pro­cras­ti­nate.

• Vaniver 10 Oct 2020 16:25 UTC

  3 points

  in reply to: Edward Kmett’s comment on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  Austin checks a lot of the same boxes, ex­cept for the hub air­port one, and is ar­guably a bet­ter cul­tural fit. There was some talk in 2018 of Delta mak­ing it a “mini-hub”, but who knows where that went. I don’t have enough travel ex­pe­rience in/​out of Austin to com­pare.

  I didn’t travel that much out of Austin, and mostly to other hubs, but I never had a bad time and of­ten could get di­rect flights. The main has­sle is just that it’s far from the other places, and so the flights take a while, but that’s always go­ing to be true for at least some peo­ple. [I sus­pect it’s bet­ter to be close to some places and fur­ther from oth­ers than medium dis­tance from ev­ery­one, but that’s not ob­vi­ous.]

• Vaniver 8 Oct 2020 22:20 UTC

  2 points

  in reply to: jasoncrawford’s comment on: In­dus­trial literacy

  Yes, a lot of the in­crease came from child­hood mor­tal­ity, but life ex­pec­tancy in­creased at ev­ery age.

  Note that life ex­pec­tancy at 50 and the gap be­tween life ex­pec­tancy at birth and life ex­pec­tancy at 1 year ba­si­cally didn’t budge from 1850 to 1900, whereas life ex­pec­tancy at birth jumped by 10 years over the same time range. I do think there are at least two dis­tinct things go­ing on (prob­a­bly all of which are re­lated to in­creased wealth and im­proved med­i­cal care).

• Vaniver 6 Oct 2020 5:31 UTC

  6 points

  in reply to: jasoncrawford’s comment on: In­dus­trial literacy

  While I agree with you that re­duc­ing child mor­tal­ity is one of the big wins of progress, I have a sense that you’re rea­son­ing about it the wrong way?

  Like, I think many (most? Nearly all?) peo­ple in the long past did have the view that in­fants or young chil­dren are re­place­able, be­cause it was adap­tive to their cir­cum­stances, and their cul­ture pro­moted adap­tive re­sponses to those cir­cum­stances. [Prac­tices like not nam­ing a child for a year are a strat­egy for the par­ents to not get too at­tached, and that makes more sense the less likely it is that a child will last a year.] If they saw the level of at­tach­ment our cul­ture en­courages par­ents have to their in­fants, they would (rightly!) see it as profli­gate spend­ing only made pos­si­ble by our mas­sive wealth and tech­nolog­i­cal know-how.

  And so in my view, the largest com­po­nent of the benefit from be­ing in a low in­fant-mor­tal­ity world is that par­ents can af­ford to treat their chil­dren as ir­re­place­able, which is bet­ter for ev­ery­one in­volved. [Like, in the world that’s dis­tant to ame­lio­rate the likely pain of child mor­tal­ity, also the peo­ple who sur­vive have their early ex­pe­rience of the world char­ac­ter­ized by dis­tance and low parental in­vest­ment, in­clud­ing nu­tri­tional in­vest­ment.] The longer you ex­pect things to last, the more you can in­vest in them—and that goes for re­la­tion­ships and friend­ships as well.

• Vaniver 4 Oct 2020 21:28 UTC

  19 points

  in reply to: Elizabeth’s comment on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  Ease of join­ing and leav­ing.

  See also Let Me Go Back, which looks at ‘bar­rier to exit’ as one of the ‘bar­ri­ers to en­try’; try­ing some­thing out is ex­pen­sive not just be­cause you have to switch to it, but also be­cause you have to switch back if you don’t like it.

• Vaniver 4 Oct 2020 21:21 UTC

  21 points

  in reply to: Kaj_Sotala’s comment on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  Could some­one elab­o­rate on their think­ing?

  I think there are lots of differ­ent good things that come out of hav­ing a pri­mary hub, with pretty differ­ent val­u­a­tions. I ba­si­cally don’t ex­pect peo­ple to have the same goals here, or for them to maybe have them al­tru­is­ti­cally but not per­son­ally.

  For ex­am­ple, one big benefit of a hub is that it makes dat­ing eas­ier if you’re look­ing to date other ra­tio­nal­ists (well, ex­cept for the whole gen­der ra­tio and poly thing). This doesn’t mat­ter to me any­more per­son­ally, as I’ve found a long-term part­ner, but that worked out pri­mar­ily be­cause I was in the ma­jor hub, and so had more op­tions to find a good match. But it still seems like a ma­jor benefit of hav­ing a pri­mary hub to me; if (say) MIRI wants new hires to be able to date ra­tio­nal­ists or EAs, it seems like a good idea to have the office near lots of sin­gle ra­tio­nal­ists and EAs. [Other­wise, you might find the only peo­ple you can hire to work in your vol­cano lair are the ones that are already mar­ried.]

  Another benefit of a hub is that you get to have in-per­son con­ver­sa­tions more eas­ily and more spon­ta­neously. I live in a group house that’s or­ga­nized it­self phys­i­cally to try to max­i­mize ‘or­ganic con­nec­tion units’, where peo­ple end up talk­ing or con­nect­ing who oth­er­wise wouldn’t have come into con­tact. The more con­nec­tion needs to be sched­uled, the less of it you get (be­cause taxes lead to dead­weight loss) and also the less serendipity you get (be­cause you only talk to the peo­ple who you know about).

  I think peo­ple are in­fluenced by their sur­round­ings and what peo­ple around them are do­ing; the thing where you know lots of peo­ple who care about X is so­cial proof that you too should care about X and effort put into it isn’t wasted. If ev­ery­one is liv­ing in some ran­dom town and just con­nected to the Craft or Move­ment through the in­ter­net, this in­creases the chance that they dis­en­gage to do some­thing else in­stead.

• Vaniver 4 Oct 2020 20:49 UTC

  17 points

  on: Vaniver’s Shortform

  A challenge in group prefer­ence /​ info ag­gre­ga­tion is dis­t­in­guish­ing be­tween “prefer­ences” and “bids.” For ex­am­ple, I might no­tice that a room is colder than I would like it to be; I might want to share that with the group (so that the de­ci­sion-mak­ing pro­cess has this info that would oth­er­wise be mostly pri­vate), but I also I’m ig­no­rant of other peo­ple’s tem­per­a­ture prefer­ences, and so don’t want to do some­thing unilat­er­al­ist (where I change the tem­per­a­ture my­self) or stake so­cial points on the tem­per­a­ture be­ing changed (in that other peo­ple should be dis­suaded from shar­ing their prefer­ences un­less they agree, or that oth­ers should feel a need to take care of me, or so on).
  
  I’ve seen a lot of solu­tions to this that I don’t like very much, mostly be­cause it feels like this is a short con­cept but most of the sen­tences for it are long. (In part be­cause I think a lot of this ne­go­ti­a­tion hap­pens im­plic­itly, and so mak­ing any of it ex­plicit feels like it nec­es­sar­ily in­volves ramp­ing up the bid strength.)
  
  This also comes up in Cir­cling; there’s lots of times when you might want to ex­press cu­ri­os­ity about some­thing with­out it be­ing in­ter­preted as a de­mand to meet that cu­ri­os­ity. “I’m in­ter­ested in this, but I don’t know if the group is in­ter­ested in this.”
  
  I think my cur­rent strat­egy to try is go­ing to ‘jointly nam­ing my in­di­vi­d­ual de­sire and un­cer­tainty about the ag­gre­gate’, but we’ll see how it goes.

• Vaniver 3 Oct 2020 16:04 UTC

  3 points

  in reply to: Mati_Roy’s comment on: “Zero Sum” is a mis­nomer.

  at­ten­tion is zero-sum: there’s a fix sup­ply (well, as a sim­plifi­ca­tion)

  As the post notes, zero-sum in re­sources is not the same as zero-sum in satis­fac­tion. Even if I can only spend a fixed at­ten­tion bud­get, how I spend it de­ter­mine global satis­fac­tion, not just the dis­tri­bu­tion of satis­fac­tion among play­ers.

• Vaniver 2 Oct 2020 5:13 UTC

  3 points

  on: Vaniver’s Shortform

  Corvee la­bor is when you pay taxes with your time and effort, in­stead of your dol­lars; his­tor­i­cally it’s made sense mostly for smaller so­cieties with­out cash economies. Once you have a cash econ­omy, it doesn’t make very much sense; rather than hav­ing ev­ery­one spend a month a year build­ing roads, bet­ter to have eleven peo­ple fund­ing the twelfth per­son who builds roads, as they can get good at it, and will be the per­son who is best at build­ing roads, in­stead of a po­ten­tially re­sent­ful am­a­teur.
  
  Amer­ica still does this in two ways. The mil­i­tary draft, which was last used in 1973, is still in a state where it could be brought back (the se­lec­tive ser­vice ad­minis­tra­tion still tracks the men who would be drafted if it were re­in­stated), and a similar pro­gram tracks health care work­ers who could be drafted if needed.

  The other is jury duty. Just like one can have pro­fes­sional or vol­un­teer sol­diers in­stead of hav­ing con­scripted sol­diers, one could have pro­fes­sional or vol­un­teer ju­rors. (See this op-ed that makes the case, or this blog post.) As a re­sult, they would be spe­cial­ized and un­der­stand the law, in­stead of be­ing po­ten­tially re­sent­ful am­a­teurs. The pri­mary benefit of a ran­domly se­lected jury—that they will (in ex­pec­ta­tion) rep­re­sent the dis­tri­bu­tion of peo­ple in so­ciety—is lost by the jury se­lec­tion pro­cess, where the lawyers can filter down to a highly non-rep­re­sen­ta­tive sam­ple. For ex­am­ple, in the OJ Simp­son trial, a pool that was 28% black (and pre­sum­ably 50% fe­male) led to a jury that was 75% black and 83% fe­male. Ran­dom se­lec­tion from cer­tified ju­rors seems pos­si­bly more likely to lead to un­bi­ased ju­ries (tho they will definitely be un­rep­re­sen­ta­tive in some ways, in that they’re le­gal pro­fes­sion­als in­stead of pro­fes­sion­als in what­ever ran­domly se­lected field).

  I’m post­ing this here be­cause it seems like the sort of thing that is a good idea that is not my com­par­a­tive ad­van­tage to push for­ward, and nev­er­the­less might be doable with fo­cused effort, and quite plau­si­bly is rather use­ful, and it seems like a sad­der world if peo­ple don’t point out the fruit that might be good to pick even if they them­selves won’t pick them.

• Vaniver 2 Oct 2020 2:28 UTC

  LW: 12 AF: 6

  AF

  on: “Zero Sum” is a mis­nomer.

  “Com­pletely ad­ver­sar­ial” also bet­ter cap­tures the strange fea­ture of zero-sum games where do­ing dam­age to your op­po­nent, by the na­ture of it be­ing zero-sum, nec­es­sar­ily means im­prov­ing your satis­fac­tion, which is a very nar­row class of situ­a­tions.

• Vaniver 26 Sep 2020 16:27 UTC

  LW: 2 AF: 1

  AF

  in reply to: Ajeya Cotra’s comment on: Draft re­port on AI timelines

  We could then back out what a ra­tio­nal firm should be will­ing to in­vest.

  This makes sense, al­tho I note that I ex­pect the fund­ing here to quite plau­si­bly be ‘ir­ra­tional.’ For ex­am­ple, some sub­stan­tial frac­tion of Microsoft’s value cap­tured is go­ing to global de­vel­op­ment in a way that seems un­likely to make sense from Microsoft’s bot­tom line (be­cause Microsoft en­riched one of its own­ers, who then de­cided to de­ploy those riches for global de­vel­op­ment). If build­ing TAI comes out of the ‘al­tru­ism’ or ‘ex­plo­ra­tion’ bud­get in­stead of the ‘we ex­pect this to pay back on sched­ule’ bud­get, you could see more in­vest­ment than that last cat­e­gory would jus­tify.

• Vaniver 25 Sep 2020 19:22 UTC

  36 points

  in reply to: sen’s comment on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  I think NYC was long a ‘sec­ond hub’, and there were a bunch of third-tier hubs, but I think the re­la­tion­ships be­tween the hubs never re­ally worked out to make a happy global com­mu­nity. Here’s a post about some pre­vi­ous con­text. I also sus­pect that the com­mu­nity has never re­ally had enough peo­ple or com­mit­ment to have ‘crit­i­cal mass’ for mul­ti­ple hubs, and this is part of the prob­lem.

  I think there are some sys­tems that have suc­cess­fully figured this out. I am op­ti­mistic about a bunch of cur­rent EA stu­dent groups at top uni­ver­si­ties, many of which I vis­ited on the SSC road trip, where there’s both 1) nat­u­ral re­cruit­ment and 2) nat­u­ral out­flow. If some­one grad­u­ates from Yale and doesn’t stay in New Haven, this is not a sur­prise; if some­one who works as a pro­fes­sional in Austin moves to the Bay Area, this is more of a sur­prise. This does have a suc­ces­sion prob­lem, where it may be the case that a par­tic­u­lar stu­dent or­ga­nizer is great, and once they grad­u­ate the group falls apart, but I think at least one uni­ver­sity has gone through a few ‘gen­er­a­tions’ of or­ga­niz­ers, and there’s prob­a­bly more we can do to sup­port fu­ture or­ga­niz­ers.

  I also think the Mor­mons have figured this out, where ‘Salt Lake’ con­trols a bunch of dis­tri­bu­tion and pub­lish­ing and so on and is definitely the ‘cul­tural cap­i­tal’ of the Mor­mon world. My sense is that in most places, rather than a weekly ser­mon cooked up by the lo­cal pas­tor you get a high-pro­duc­tion val­ues (in all senses) DVD from the cen­tral office. I think our ver­sion of that is pop­u­lar blogs and pod­casts, where a global com­mu­nity can be read­ing SSC and listen­ing to the 80k hours pod­cast and so mostly be in sync with each other, but this only re­ally works for the “ex­cite­ment about cool top­ics” and “grad­u­ally flesh­ing out your world model” di­men­sions, and is not as good for lo­cal com­mu­nity norms or build­ing pair­wise re­la­tion­ships or so on.

  I think a prob­lem here is that while we have lots of fea­tures that are re­li­gion-like, I think we don’t re­ally pri­ori­tize the “cul­tural cen­ter” as­pects, and so there aren’t re­ally peo­ple who want to be ra­tio­nal­ist pas­tors /​ bishops /​ etc.; Eliezer mostly want to work on AI safety in­stead of com­mu­nity-build­ing, my sense is that CFAR mostly wants to work on skil­ling up /​ re­cruit­ing x-risk thinkers in­stead of com­mu­nity-build­ing, and so on. For ex­am­ple, when I look at plans to make sec­ondary hubs that seem likely to ac­tu­ally hap­pen to me, most of them are par­ents try­ing to make good neigh­bor­hoods for them­selves and their kids, where they are ac­tu­ally tak­ing on the ‘bur­den of cul­tural own­er­ship’ or w/​e; I think a lot of orgs that peo­ple hope will be Com­mu­nity orgs are in­stead mostly in­ter­ested in be­ing Craft orgs.

  What links here?

  • Kaj_Sotala's comment on The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem by mingyuan (1 Oct 2020 16:57 UTC; 16 points)

• Vaniver 24 Sep 2020 19:19 UTC

  8 points

  in reply to: johnswentworth’s comment on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  Sure, one could ar­gue that Oak­land ac­tu­ally fits the three desider­ata, be­cause I left out “low crime,” al­tho I don’t think Oak­land is ac­tu­ally cheap. The broader point of “you get what you pay for” holds, I think, and the only way you get some­thing ‘ac­cept­ably cheap’ is de­liber­ately de­cid­ing to not pay for some things you could pay for.

• Vaniver 24 Sep 2020 4:31 UTC

  39 points

  on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  My sense is that co­or­di­na­tion for this is ba­si­cally im­pos­si­ble, be­cause of com­pet­ing ac­cess needs. I am most op­ti­mistic about ver­sions of this that will:
  
  1) Hap­pen even if no one else signs on. [For ex­am­ple, peo­ple mov­ing to small towns within com­mut­ing dis­tance of SF/​Berkeley, where it makes sense for them even if no one else moves to Pinole or Mor­aga or wher­ever.]
  2) Be read­ily ex­ten­si­ble. [If one per­son buys in Pinole, other peo­ple can later buy other houses in Pinole, and slowly shift the bal­ance. Many ra­tio­nal­ist group houses started off as a sin­gle apart­ment in a split house, and slowly took over through or­ganic growth. Build­ing a neigh­bor­hood of small houses in up­state Ver­mont to re­place your group house, if it works, prob­a­bly also means some­one else could build a sub­di­vi­sion for their group house next door.]
  3) Pick a vi­sion and be will­ing to de­liver on it. [You’re not go­ing to find a place that has great weather and cheap prop­erty value and prox­im­ity to great cities; that’s not how effi­cient mar­kets work. In­stead, figure out the few crite­ria that mat­ter most to you, and do what it takes to achieve those crite­ria.]
  
  This is ba­si­cally the only way I see for pro­jects to get out of the plan­ning stage and into the re­al­ity stage; there will be Some Chil­dren Left Be­hind, and also some peo­ple who de­cide that, well, they do re­ally like the sun but lu­me­na­tors will be suffi­cient to make up­state Ver­mont work­able (or what­ever).

  Separately, I note that ‘chance meet­ings among ex­traverts’ seem to be a pretty pow­er­ful fac­tor in shap­ing the his­tory of cul­tures and or­ga­ni­za­tions, and think that there re­ally is a very large benefit to be­ing in a cen­tral hub; I think those hubs have to be­come much worse for it to not be worth it any­more. [The main com­pel­ling rea­son I see for mov­ing away from the hub is in or­der to have chil­dren—sub­urbs ex­ist for a rea­son!--but for peo­ple still try­ing to find part­ners or mean­ingful work, the hubs re­main very im­por­tant.]

• Vaniver 24 Sep 2020 4:05 UTC

  29 points

  on: The ra­tio­nal­ist com­mu­nity’s lo­ca­tion problem

  I think a more at­trac­tive op­tion than a group house is some­thing like a pocket neigh­bor­hood or bau­gruppe; I think a lo­ca­tion be­ing fa­vor­able to that sort of de­vel­op­ment is a ma­jor point in its fa­vor.

• Vaniver 21 Sep 2020 17:11 UTC

  5 points

  in reply to: Vaniver’s comment on: Open & Wel­come Thread—Septem­ber 2020

  To elab­o­rate on this, I think there are two dis­tinct is­sues: “do they have the right norms?” and “do they do norm en­force­ment?”. The sec­ond is nor­mally good in­stead of prob­le­matic, but makes the first much more im­por­tant than it would be oth­er­wise. I see Zack_M_Davis as point­ing out “hey, if we don’t let peo­ple en­force norms be­cause that would make norm­break­ers feel threat­ened, do we even have norms?”, which is a valid point, but which feels some­what ir­rele­vant to the curi ques­tion.

• Vaniver 20 Sep 2020 1:46 UTC

  24 points

  in reply to: Zack_M_Davis’s comment on: Open & Wel­come Thread—Septem­ber 2020

  For what it’s worth, I think a de­ci­sion to ban would stand on just his pur­suit of con­ver­sa­tional norms that re­ward stamina over cor­rect­ness, in a way that I think makes LessWrong worse at in­tel­lec­tual progress. I didn’t check out this page, and it didn’t fac­tor into my sense that curi shouldn’t be on LW.

  I also find it some­what wor­ry­ing that, as I un­der­stand it, the page was a com­bi­na­tion of “quit”, “evaded”, and “lied”, of which ‘quit’ is not wor­ry­ing (I con­sider some­one giv­ing up on a con­ver­sa­tion with curi un­der­stand­able in­stead of shame­ful), and that get­ting wrapped up in the “&c.” in­stead of be­ing the cen­tral ex­am­ple seems like it’s defin­ing away my main crux.

• Vaniver 19 Sep 2020 17:11 UTC

  LW: 4 AF: 2

  AF

  on: Draft re­port on AI timelines

  Part 1 page 15 talks about “spend­ing on com­pu­ta­tion”, and as­sumes spend­ing sat­u­rates at 1% of the GDP of the largest coun­try. This seems po­ten­tially odd to me; quite pos­si­bly the spend­ing will be done by multi­na­tional cor­po­ra­tions that view them­selves as more “global” than “Amer­i­can” or “Bri­tish” or what­ever, and whose for­tunes are more tied to the global econ­omy than to the na­tional econ­omy. At most this gives you a fac­tor of 2-3 dou­blings, but that’s still 4-6 years on a 2-year dou­bling time.

  Over­all I’m not sure how much to be­lieve this hy­poth­e­sis; my main­line pre­dic­tion is that cor­po­ra­tions grow in power and rootless­ness com­pared to na­tion-states, but it also seems likely that bits of the global econ­omy will frac­ture /​ there will be a push to de­cen­tral­iza­tion over cen­tral­iza­tion, where (say) Alpha­bet is more like “global ex­clud­ing China, where Baidu is supreme” than it is “global.” In that world, I think you still see ap­prox­i­mately a 4x in­crease.

  I also don’t have a great sense how we should ex­pect the ‘abil­ity to fund large pro­jects’ to com­pare be­tween the gov­ern­ments of the past and the mega­corps of the fu­ture; it seems quite plau­si­ble to me that Alpha­bet, with­out pres­sure to do welfare spend­ing /​ fund the mil­i­tary /​ etc. could put a much larger frac­tion of its re­sources to­wards build­ing TAI, but also pre­sum­ably this means Alpha­bet has many fewer re­sources than the econ­omy as a whole (be­cause there still will be welfare spend­ing and mil­i­tary fund­ing and so on), and on net this prob­a­bly works out to 1% of to­tal gdp available for megapro­jects.

• Vaniver 19 Sep 2020 16:48 UTC

  LW: 9 AF: 5

  AF

  on: Draft re­port on AI timelines

  Thanks for shar­ing this draft! I’m go­ing to try to make lots of differ­ent com­ments as I go along, rather than one huge com­ment.

  [edit: page 10 calls this the “most im­por­tant thread of fur­ther re­search”; the down­side of writ­ing as I go! For pos­ter­ity’s sake, I’ll leave the com­ment.]

  Pages 8 and 9 of part 1 talk about “effec­tive hori­zon length”, and make the claim:

  Prima fa­cie, I would ex­pect that if we mod­ify an ML prob­lem so that effec­tive hori­zon length is dou­bled (i.e, it takes twice as much data on av­er­age to reach a cer­tain level of con­fi­dence about whether a per­tur­ba­tion to the model im­proved perfor­mance), the to­tal train­ing data re­quired to train a model would also dou­ble. That is, I would ex­pect train­ing data re­quire­ments to scale lin­early with effec­tive hori­zon length as I have defined it.

  I’m cu­ri­ous where ‘lin­early’ came from; my sense is that “effec­tive hori­zon length” is the equiv­a­lent of “op­ti­mal batch size”, which I would have ex­pected to be a weirder func­tion of train­ing data size than ‘lin­ear’. I don’t have a great han­dle on the ML the­ory here, tho, and it might be sub­stan­tially differ­ent be­tween clas­sifi­ca­tion (where I can make batch-of-the-en­velope es­ti­mates for this sort of thing) and RL (where it feels like it’s a com­po­nent of a much trick­ier sys­tem with harder-to-pre­dict con­nec­tions).

  Quite pos­si­bly you talked with some ML ex­perts and their sense was “lin­early”, and it makes sense to roll with that; it also seems quite pos­si­ble that the thing to do here is have un­cer­tainty over func­tional forms. That is, maybe the effec­tive hori­zon scales lin­early, or maybe it scales ex­po­nen­tially, or maybe it scales log­a­r­ith­mi­cally, or in­verse square root, or what­ever. This would help dou­ble-check that the as­sump­tion of lin­ear­ity isn’t do­ing sig­nifi­cant work, and if it is, point to a po­ten­tially promis­ing av­enue of the­o­ret­i­cal ML re­search.

  [As a broader point, I think this ‘func­tional form un­cer­tainty’ is a big deal for my timelines es­ti­mates. A lot of peo­ple (right­fully!) dis­missed the stan­dard RL al­gorithms of 5 years ago for mak­ing AGI be­cause of ex­po­nen­tial train­ing data re­quire­ments, but my sense is that fur­ther al­gorith­mic im­prove­ment is mostly not “it’s 10% faster” but “the base of the ex­po­nent is smaller” or “it’s no longer ex­po­nen­tial.”, which might change whether or not it makes sense to dis­miss it.]

• Vaniver 19 Sep 2020 16:11 UTC

  LW: 7 AF: 4

  AF

  in reply to: ofer’s comment on: Draft re­port on AI timelines

  A sim­ple, well-funded ex­am­ple is au­tonomous ve­hi­cles, which have spent con­sid­er­ably more than the train­ing bud­get of AlphaS­tar, and are not there yet.

  I am aware of other ex­am­ples that do seem to be hap­pen­ing, but I’m not sure what the cut­off for ‘mas­sive’ should be. For ex­am­ple, a ‘call cen­ter bot’ is mod­er­ately valuable (while not nearly as trans­for­ma­tive as au­tonomous ve­hi­cles), and I be­lieve there are many differ­ent com­pa­nies at­tempt­ing to do some­thing like that, al­tho I don’t know how their to­tal ML ex­pen­di­ture com­pared to AlphaS­tar’s. (The com­pany I’m most fa­mil­iar with in this space, Ap­prente, got ac­quired by McDon­alds last year, who I pre­sume is mostly in­ter­ested in the abil­ity to au­to­mate drive-thru or­ders.)

  Another ex­am­ple that seems rele­vant to me is robotic hands (plus image clas­sifi­ca­tion) at suffi­cient level that ware­house pick­ers could be re­placed by robots.