ryan_b

• ryan_b 22 Jan 2020 18:01 UTC

  2 points

  on: Terms & liter­a­ture for pur­posely lossy communication

  Are we think­ing from the trans­mit­ter end, the re­ceiver end, or doesn’t it mat­ter? The ob­vi­ous an­swer seems to me to be filters, speci­fi­cally a band-pass filter.

• ryan_b 17 Jan 2020 19:35 UTC

  9 points

  on: Toward a New Tech­ni­cal Ex­pla­na­tion of Tech­ni­cal Explanation

  I do not un­der­stand Log­i­cal In­duc­tion, and I es­pe­cially don’t un­der­stand the re­la­tion­ship be­tween it and up­dat­ing on ev­i­dence. I feel like I keep view­ing Bayes as a pro­ce­dure sep­a­rate from the agent, and then try­ing to slide LI into that same slot, and it fails be­cause at least LI and prob­a­bly Bayes are wrongly viewed that way.
  
  But this post is what I leaned on to shift from an ut­ter-dark­ness un­der­stand­ing of LI to a heavy-fog one, and re-read­ing it has been very use­ful in that re­gard. Since I am oth­er­wise not a per­son who would be ex­pected to un­der­stand it, I think this speaks very well of the post in gen­eral and of its im­por­tance to the con­ver­sa­tion sur­round­ing LI.

  This also is a good ex­am­ple of the norm of mul­ti­ple lev­els of ex­pla­na­tion: in my lay opinion a good in­tel­lec­tual pipeline needs ex­pla­na­tion stretch­ing from in­tu­ition through for­mal­ism, and this is such a post on one of the most im­por­tant de­vel­op­ments here.

• ryan_b 16 Jan 2020 23:02 UTC

  4 points

  in reply to: Jameson Quinn's comment on: A vot­ing the­ory primer for rationalists

  Con­grat­u­la­tions on finish­ing your doc­torate! I’m very much look­ing for­ward to the next post in the se­quence on multi-win­ner meth­ods, and I’m es­pe­cially the met­ric you men­tion.

• ryan_b 16 Jan 2020 22:59 UTC

  18 points

  on: A vot­ing the­ory primer for rationalists

  I think this post should be in­cluded in the best posts of 2018 col­lec­tion. It does an ex­cel­lent job of bal­anc­ing sev­eral de­sir­able qual­ities: it is very well writ­ten, be­ing both clear and en­ter­tain­ing; it is in­for­ma­tive and thor­ough; it is in the style of ar­gu­ment which is preferred on LessWrong, by which I mean makes use of both the­ory and in­tu­ition in the ex­pla­na­tion.

  This post adds to the greater con­ver­sa­tion by dis­play­ing ra­tio­nal­ity of the kind we are pur­su­ing di­rected at a big so­cietal prob­lem. A spe­cific ex­am­ple of what I mean that dis­t­in­guishes this post from an overview that any mo­ti­vated poster might write is the in­clu­sion of War­ren Smith’s re­sults; Smith is a math­e­mat­i­cian from an un­re­lated field who has no pub­lished work on the sub­ject. But he had work any­way, and it was good work which the au­thor him­self ex­panded on, and now we get to benefit from it through this post. This puts me very much in mind of the fact that this com­mu­nity was pri­mar­ily founded by an au­to­di­dact who was deeply in­fluenced by a physi­cist writ­ing about prob­a­bil­ity the­ory.

  A word on one of our sa­cred taboos: in the be­gin­ning it was writ­ten that Poli­tics is the Mind­kil­ler, and so it was for years and years. I ex­pect this is our most con­sis­tently and uni­ver­sally en­forced taboo. Yet here we have a high-qual­ity and very well re­ceived post about poli­tics, and of the ~70 com­ments only one ap­pears to have been mind­kil­led. This post has great value on the strength of be­ing an ex­am­ple of how to ad­dress trou­bling ter­ri­tory suc­cess­fully. I ex­pect most read­ers didn’t even con­sider that this was poli­ti­cal ter­ri­tory.

  Even though it is a the­ory primer, it man­ages to be prac­ti­cal and ac­tion­able. Ob­serve how the very method of scor­ing posts for the re­view, quadratic vot­ing, is one that is dis­cussed in the post. Prac­ti­cal im­pli­ca­tions for the man­age­ment of the com­mu­nity weigh heav­ily in my con­sid­er­a­tion of what should be con­sid­ered im­por­tant con­ver­sa­tion within the com­mu­nity.

  Car­ry­ing on from that point into its in­verse, I note that this post in­tro­duced the topic to the com­mu­nity (though there are scat­tered older refer­ences to some of the things it con­tains in com­ments). Fur­ther, as far as I can tell the au­thor wasn’t a long­time com­mu­nity mem­ber be­fore this post and the se­quence that fol­lowed it. The rea­son this mat­ters is that LessWrong can now at­tract and give trac­tion to ex­perts in fields out­side of its origi­nal core ar­eas of in­ter­est. This is not a sig­nal of the qual­ity of the post so much as the post be­ing a sig­nal about LessWrong, so there is a definite sense in which this weighs against its in­clu­sion: the post showed up fully formed rather than be­ing the out­put of our in­tel­lec­tual pipeline.

  I would have liked to see (prob­a­bly against the prefer­ences of most of the com­mu­nity and cer­tainly against the sig­nals the au­thor would have re­ceived as a lurker) the ar­eas where ad­vo­cacy is hap­pen­ing as a spe­cific sec­tion. I found them any­way, be­cause they were con­tained in the dis­clo­sures and threaded through the dis­cus­sion, and click­ing the links, but I sus­pect that many read­ers would have missed them. This is es­pe­cially true for read­ers less poli­ti­cally in­ter­ested than I, which most of them. The ob­vi­ous rea­son is for in­ter­ested peo­ple to be able to find it more eas­ily, which mat­ters a lot to prob­lems like this one. The meta-rea­son is that posts that tread dan­ger­ous ground might benefit from di­rect­ing peo­ple some­where else for ad­vo­cacy speci­fi­cally, kind of like a com­mu­ni­ca­tion-pres­sure re­lease valve. It speaks to the qual­ity of the post this wasn’t even an is­sue here, but for fu­ture posts on similar top­ics in a grow­ing LessWrong I ex­pect it to be.

  Lastly I want to ob­serve the fol­low-up posts in the se­quence are also good, sug­gest­ing that this post was fer­tile ground for more dis­cus­sion. In terms of ad­di­tional fol­low-up: I would like to see this the­ory de­ployed at the level of in­tu­ition build­ing, in a way similar to how we use mar­kets, Pri­soner’s Dilem­mas, and more re­cently con­sid­ered Stag Hunts. I feel like it would be a good, hu­man-achiev­able coun­ter­weight to things like util­ity func­tions and value hand­shakes in our con­ver­sa­tion, and make our dis­cus­sions more ac­tion­able thereby.

• ryan_b 16 Jan 2020 18:56 UTC

  4 points

  on: Open & Wel­come Thread—Jan­uary 2020

  Reflect­ing on mak­ing morally good choices vs. morally bad ones, I no­ticed the thing I lean on the most is not eval­u­at­ing the bad ones. This effec­tively means good choices pay up front in com­pu­ta­tional sav­ings.

  I’m not sure whether this counts as dark arts-ing my­self; on the one hand it is clearly a case of mo­ti­vated stop­ping. On the other hand I have a solid prior that there are many more wrong choices than right ones, which im­plies eval­u­at­ing them fairly would be stupidly ex­pen­sive; that in turn im­plies the don’t-com­pute-evil rule is pretty effi­cient even if it were ar­bi­trar­ily cho­sen.

• ryan_b 14 Jan 2020 22:15 UTC

  4 points

  on: Is back­wards cau­sa­tion nec­es­sar­ily ab­surd?

  I feel that ques­tions like this have a hard time es­cap­ing con­fu­sion be­cause the no­tion of lin­ear time is so deeply as­so­ci­ated with causal­ity already.

  Could you point me to the ar­gu­ments about a high-en­tropy uni­verse be­ing ex­pected to de­crease in en­tropy?

• ryan_b 9 Jan 2020 18:23 UTC

  2 points

  in reply to: quanticle's comment on: What is Life in an Im­moral Maze?

  I think I agree with your in­tu­ition, though I sub­mit that size is re­ally only a proxy here for lev­els of hi­er­ar­chy. We ex­pect more lev­els in a big­ger or­ga­ni­za­tion, is all. I think this gets at the mechanisms for why the kinds of be­hav­iors in Mo­ral Mazes might ap­pear. I have seen sev­eral of the Mo­ral Mazes be­hav­iors play out in the Army, which is one of the largest and most hi­er­ar­chi­cal or­ga­ni­za­tions in ex­is­tence.

  I don’t see why be­ing con­sumed by your job would pre­dict any of the rest of it; pro­gram­mers, lawyers, and sales­men are no­to­ri­ous for spend­ing all of their time on work, and those aren’t man­age­ment po­si­tions. Rather, I ex­pect that all these be­hav­iors ex­ist on con­tinua, and we should see more or less of them de­pend­ing on how strongly peo­ple are re­spond­ing to the in­cen­tives.

  My in­tu­ition is that the re­sults prob­lem largely drives the de­scrip­tion to which you are re­spond­ing. Front line peo­ple and front line man­agers usu­ally have some­thing tan­gible by which to be mea­sured, but once peo­ple en­ter the mid­dle zone of not be­ing di­rectly con­nected to the top line or the bot­tom line re­sults, there’s noth­ing left but sig­nal­ling. So even a 9-5 guy who goes fish­ing is still likely to play poli­tics, avoid rock­ing the boat, pass the blame down­hill, and think that out­comes are de­ter­mined by out­side forces.

  I would be shocked to my core if Mo­ral Mazes be­hav­iors rarely ap­peared un­der such con­di­tions.

• ryan_b 8 Jan 2020 15:29 UTC

  2 points

  in reply to: quanticle's comment on: What is Life in an Im­moral Maze?

  One of the largest in the coun­try. The core or­ga­ni­za­tion is less than a thou­sand, but they have state af­fili­ate or­ga­ni­za­tions and as of re­cently in­ter­na­tional ones as well.

  It is ex­ceed­ingly top-heavy; I want to say it was ap­proach­ing 5% ex­ec­u­tives, not count­ing their im­me­di­ate staff.

  The or­ga­ni­za­tion is func­tion­ally in free-fall now; they are hem­or­rhag­ing peo­ple and money. I ex­pect if it were for-profit this is the part where they would go bankrupt. The tran­si­tion from well-func­tion­ing to free-fall took ~5 years.

• ryan_b 7 Jan 2020 22:11 UTC

  4 points

  in reply to: johnswentworth's comment on: What is Life in an Im­moral Maze?

  When con­sid­er­ing a bar­rier to exit, do they usu­ally in­clude the cost to go some­where else? Quit­ting is free and easy, but get­ting an­other job el­se­where isn’t, es­pe­cially when con­sid­er­ing op­por­tu­nity costs.

• ryan_b 7 Jan 2020 22:07 UTC

  4 points

  in reply to: quanticle's comment on: What is Life in an Im­moral Maze?

  By con­trast this does match my wife’s ex­pe­riences as a se­nior man­ager in a large non-profit. There were re­peated and con­sis­tent mes­sages about be­ing ex­pected to re­spond to emails and calls at all hours as you moved up the hi­er­ar­chy; the perfor­mance met­rics were fixed so ev­ery­one fit within a nar­rower band; ac­tual out­comes of pro­grams did not mat­ter sug­gest­ing that they did was pun­ished (cul­mi­nat­ing in one fas­ci­nat­ing epi­sode where a VP seems to have made up an en­tire pro­gram which de­liv­ered 0.001 of pro­jected rev­enue, re­sult­ing in a rev­enue short­fall of some 25% for the whole or­ga­ni­za­tion, and who was not fired).

• ryan_b 30 Dec 2019 15:22 UTC

  3 points

  in reply to: Kaj_Sotala's comment on: We need to re­visit AI rewrit­ing its source code

  Self mod­ify­ing code has been pos­si­ble but not prac­ti­cal for as long as we have had digi­tal com­put­ers. Now it has toolchains, use cases, and in the near fu­ture tens to hun­dreds of peo­ple will do it as their day job.

  The strong ver­sion of my claim is that I ex­pect to see the same kinds of failure modes we are con­cerned with in AGI pushed down to the level of con­sumer-grade soft­ware, at least in huge ap­pli­ca­tions like so­cial net­works and self-driv­ing cars.

  I think it is now sim­ple and cheap enough for a sin­gle re­search group to do some­thing like:

  • Write a for­mal specification

  • Which em­ploys learn­ing for some sim­ple purpose

  • And em­ploys self-mod­ifi­ca­tion on one or more levels

  Which is to say, it feels like we have enough tool­ing to start do­ing “Hello World” grade self-mod­ifi­ca­tion tests that ac­count for ev­ery level of the stack, in real sys­tems.

We need to re­visit AI rewrit­ing its source code

• ryan_b 26 Dec 2019 13:25 UTC

  4 points

  on: Tech­ni­cal AGI safety re­search out­side AI

  I think sys­tems en­g­ineer­ing is a can­di­date for this, at least as far as the safety and meta sec­tions go.

  There is a pro­gram at MIT for ex­pand­ing sys­tems en­g­ineer­ing to ac­count for post-de­sign vari­a­tions in the en­vi­ron­ment, in­clud­ing spe­cific rea­son­ing about a broader no­tion of safety:

  Sys­tems Eng­ineer­ing Ad­vance­ment Re­search Initiative

  There was also a DARPA pro­gram for speed­ing up the de­liv­ery of new mil­i­tary ve­hi­cles, which seems to have the most di­rect ap­pli­ca­tions to CAIS:

  Sys­tems Eng­ineer­ing and the META Program

  Among other things, sys­tems en­g­ineer­ing has the virtue of mak­ing hard­ware an ex­plicit fea­ture of the model.

• ryan_b 19 Dec 2019 16:15 UTC

  6 points

  on: Prop­a­gat­ing Facts into Aesthetics

  1. I strongly en­dorse this line of think­ing, and I want to see it con­tinue to de­velop. I have a very strong ex­pec­ta­tion that we will see benefits re­ally ac­crue from the ra­tio­nal­ity pro­ject when we have fi­nally hit on ev­ery­thing im­por­tant to hu­mans. Speci­fi­cally, tak­ing the first step in each of prob­a­bil­ity|pur­pose|com­mu­nity|aes­thet­ics|etc will be much more im­pact­ful than puis­sant mas­tery of only prob­a­bil­ity.

  2.

  I am in fact con­fused by this. My an­swer is “yes”, and I don’t know why. Deserts don’t have much in the way of re­sources. Their stark beauty is more like the way a statue is beau­tiful than the way a for­est is beau­tiful.

  I think the key word here is “stark.” The desert en­vi­ron­ment is el­e­gant, be­cause it has fewer things in it. We can see clearly the carv­ing of the wind into the dunes, the sud­den con­trast where sand abuts stone, the end­less gleam­ing of the salt. Con­sider for a mo­ment the differ­ence be­tween look­ing at the for­est and look­ing at the trees: when I zoom out to the for­est level I no­tice the lay of the hills be­neath the trees, the grad­ual change from one kind of tree to an­other, spot the gaps where rivers run or the dirt thins. Deserts smack you in the face with the for­est-level view, be­cause there isn’t an­other one available.

  3. I like the ex­ten­sion to dis­gust. My ex­pe­rience was also with deserts, but in this case my im­pres­sion was that deserts were clean. I found my­self out in the dunes of Kuwait, where there was an abun­dance of flies. I figured they would go for our wa­ter, or per­haps our pro­tein bars. Then I saw they hap­pily landed any­where on the sand, and I thought: wait, what do flies eat?

  So now I think of deserts as beau­tiful and filthy.

  What links here?

  • Prop­a­gat­ing Facts into Aesthetics by Raemon (19 Dec 2019 4:09 UTC; 83 points)

• ryan_b 18 Dec 2019 19:07 UTC

  4 points

  in reply to: johnswentworth's comment on: Is Causal­ity in the Map or the Ter­ri­tory?

  I have a hard time think­ing of that ex­am­ple as a differ­ent causal struc­ture. Rather I think of it as keep­ing the same causal struc­ture, but ab­stract­ing most of it away un­til we reach the level of the knob; then we make the knob con­crete. This cre­ates an af­for­dance.

  Of course when I am in my house I am ap­proach­ing it from the knob-end, so mostly I just as­sume some lay­ers of hid­den de­tail be­hind it.

  Another way to say this is that I tend to view it as com­press­ing causal struc­ture.

• ryan_b 18 Dec 2019 17:48 UTC

  4 points

  on: Is Causal­ity in the Map or the Ter­ri­tory?

  This point might be use­less, but it feels like we are sub­sti­tut­ing sub-maps for the ter­ri­tory here. This ex­am­ple looks to me like:

  Cir­cuits → Map

  Physics → Sub-map

  Real­ity → Territory

  I in­tu­itively feel like a causal sig­na­ture should show up in the sub-map of whichever level you are cur­rently ex­am­in­ing. I am tempted to go as far as say­ing the de­gree to which the sub-map al­lows causal in­fer­ence is effec­tively a mea­sure of how close the lay­ers are on the lad­der of ab­strac­tion. In my head this sounds some­thing like “perfect causal in­fer­ence im­plies the min­i­mum co­her­ent ab­strac­tion dis­tance.”

• ryan_b 18 Dec 2019 15:37 UTC

  2 points

  in reply to: clone of saturn's comment on: Open & Wel­come Thread—De­cem­ber 2019

  That’s the one! My thanks, I was on the verge of mad­ness!

• ryan_b 16 Dec 2019 21:06 UTC

  6 points

  on: Ap­proval Ex­trac­tion Ad­ver­tised as Production

  This part is a lit­tle baf­fling to me:

  For bet­ter or worse that’s never go­ing to be more than a thought ex­per­i­ment. We could never stand it. How about that for coun­ter­in­tu­itive? I can lay out what I know to be the right thing to do, and still not do it. I can make up all sorts of plau­si­ble jus­tifi­ca­tions. It would hurt YC’s brand (at least among the in­nu­mer­ate) if we in­vested in huge num­bers of risky star­tups that flamed out. It might dilute the value of the alumni net­work. Per­haps most con­vinc­ingly, it would be de­mor­al­iz­ing for us to be up to our chins in failure all the time. But I know the real rea­son we’re so con­ser­va­tive is that we just haven’t as­similated the fact of 1000x vari­a­tion in re­turns.
  

  We’ll prob­a­bly never be able to bring our­selves to take risks pro­por­tionate to the re­turns in this busi­ness.

  So I get why Y Com­bi­na­tor can’t do this, but the “we” seems more in­clu­sive here than just the YC team. I think this be­cause in most other in­stances of not know­ing how or be­ing un­able to do some­thing, he trou­bles to sug­gest a way some­one else might be able to.

  If peo­ple are pre­pared to in­vest a lot of money in high fre­quency trad­ing al­gorithms which are fa­mously opaque to the peo­ple pro­vid­ing the money, or into hedge funds which sys­tem­at­i­cally lose to the mar­ket, why wouldn’t some­one be will­ing to in­vest in an even larger num­ber of star­tups than Y Com­bi­na­tor?

  If we fol­low the logic of dump­ing ar­bi­trary tests, it feels like it might be as di­rect as con­figur­ing a few rea­soned rules, us­ing a stan­dard­ized equity offer with stan­dard­ized pa­per­work, and then just slowly tweak the rea­soned rules as the batch out­comes roll in.

• ryan_b 11 Dec 2019 15:53 UTC

  4 points

  in reply to: Tenoke's comment on: Open & Wel­come Thread—De­cem­ber 2019

  The com­ment­ing guidelines al­lows users to set their own norms of com­mu­ni­ca­tion for their own pri­vate posts. This lets us ex­per­i­ment with differ­ent norms to see which work bet­ter, and also al­lows the LessWrong com­mu­nity to di­ver­sify into differ­ent sub­com­mu­ni­ties should there be in­ter­est. It says habryka’s guidelines be­cause that’s who posted this post; if you go back through the other open threads, you will see other peo­ple posted many of them, and differ­ent com­ment­ing guidelines here and there. I think the posts that speak to this the most are:

  [Meta] New mod­er­a­tion tools and mod­er­a­tion guidelines (by habryka)

  Meta-tations on Moder­a­tion: Towards Public Archipelago (by Rae­mon)

• ryan_b 9 Dec 2019 22:29 UTC

  4 points

  on: Open & Wel­come Thread—De­cem­ber 2019

  There’s a post some­where in the ra­tio­nal­sphere that I can’t re­lo­cate for the life of me. Can any­body help?

  The point was com­mu­ni­ca­tion. The ex­am­ple given was the differ­ence be­tween a lec­ture and a ser­mon. The dis­tinc­tion the au­thor made was some­thing like a pro­fes­sor talk­ing to stu­dents in class, each of whom then goes home and does home­work by them­selves, ver­sus a preacher who gives his ser­mon to the con­gre­ga­tion, with the ex­pec­ta­tion that they will break off into groups and dis­cuss the ser­mon among them­selves.

  I have a vague mem­ory that there were graph­ics in­volved.

  I have tried lo­cal search on LessWrong, site search of LessWrong, and brows­ing a few post his­to­ries that seemed like they might be the au­thor based on a vague sense of aes­thetic similar­ity. I was sure it was here, but now I fear it may have been el­se­where or it is hid­den in some other kind of post.