Vladimir_Nesov

• Vladimir_Nesov 9 Apr 2019 16:48 UTC

  10 points

  on: Agent Foun­da­tion Foun­da­tions and the Rocket Align­ment Problem

  At first glance, the cur­rent Agent Foun­da­tions work seems to be for­mal, but it’s not the kind of for­mal where you work in an es­tab­lished set­ting. It’s coun­ter­in­tu­itive that peo­ple can doo­dle in math, but they can. There’s a lot of that in physics and eco­nomics. Pre-for­mal work doesn’t need to lack for­mal­ity, it just doesn’t fol­low a spe­cific set of rules, so it can use math to sketch prob­lems ap­prox­i­mately the same way in­for­mal words sketch prob­lems ap­prox­i­mately.

• Vladimir_Nesov 6 Apr 2019 12:33 UTC

  4 points

  in reply to: Chris_Leong's comment on: Would solv­ing log­i­cal coun­ter­fac­tu­als solve an­throp­ics?

  I con­sider ques­tions of moral­ity or ax­iol­ogy seper­ate from ques­tions of de­ci­sion the­ory.

  The claim is es­sen­tially that speci­fi­ca­tion of an­thropic prin­ci­ples an agent fol­lows be­longs to ax­iol­ogy, not de­ci­sion the­ory. That is, the or­thog­o­nal­ity the­sis ap­plies to the dis­tinc­tion, so that differ­ent agents may fol­low differ­ent an­thropic prin­ci­ples in the same way as differ­ent stuff-max­i­miz­ers may max­i­mize differ­ent kinds of stuff. Some things dis­cussed un­der the um­brella of “an­throp­ics” seem rele­vant to de­ci­sion the­ory, such as be­ing able to func­tion with most an­thropic prin­ci­ples, but not, say, choice be­tween SIA and SSA.

  (I some­what dis­agree with the claim, as struc­tur­ing val­ues around in­stances of agents doesn’t seem nat­u­ral, maps/​wor­lds are more ba­sic than agents. But that is dis­agree­ment with em­pha­siz­ing the whole con­cept of an­throp­ics, per­haps even with em­pha­siz­ing agents, not with where to put the con­cepts be­tween ax­iol­ogy and de­ci­sion the­ory.)

• Vladimir_Nesov 1 Apr 2019 8:04 UTC

  11 points

  in reply to: habryka's comment on: Open Thread April 2019

  The is­sue is that GPT2 posts so much it drowns out ev­ery­thing else.

• Vladimir_Nesov 26 Mar 2019 21:11 UTC

  4 points

  in reply to: G Gordon Worley III's comment on: Do you like bul­let points?

  An ex­pla­na­tion com­mu­ni­cates an idea with­out in­sist­ing on its rele­vance to re­al­ity (or some other topic). It’s mod­u­lar. You can then ex­plain its rele­vance to re­al­ity, as an­other idea that reifies the rele­vance. Per­sua­sion is do­ing both at the same time with­out mak­ing it clear. For ex­am­ple, you can ex­plain how to think about a hy­poth­e­sis to see what ob­ser­va­tions it pre­dicts, with­out per­suad­ing that the hy­poth­e­sis is likely.

• Vladimir_Nesov 19 Mar 2019 4:54 UTC

  5 points

  in reply to: Raemon's comment on: Privacy

  Nod. I did ac­tu­ally con­sider a more ac­cu­rate ver­sion of the com­ment that said some­thing like “at least one of us is at least some­what con­fused about some­thing” [...]

  The clar­ifi­ca­tion doesn’t ad­dress what I was talk­ing about, or else dis­agrees with my point, so I don’t see how that can be char­ac­ter­ised with a “Nod”. The con­fu­sion I re­fer to is about what the other means, with the ques­tion of whether any­one is cor­rect about the world ir­rele­vant. And this con­fu­sion is sig­nifi­cant on both sides, oth­er­wise a con­ver­sa­tion doesn’t go off the rails in this way. Pay­ing at­ten­tion to truth is coun­ter­pro­duc­tive when in­tended mean­ing is not yet es­tab­lished, and you seem to be talk­ing about truth, while I was com­ment­ing about mean­ing.

• Vladimir_Nesov 19 Mar 2019 4:24 UTC

  4 points

  in reply to: Raemon's comment on: Karma-Change Notifications

  No an­cient up­dates for the pre­vi­ous week, sev­eral for this week. An al­ter­na­tive to re­mov­ing old no­tifi­ca­tions is to prepend en­tries in the list with re­cency, like “13d” or “8y”, and sort by it.

• Vladimir_Nesov 19 Mar 2019 4:16 UTC

  2 points

  in reply to: Raemon's comment on: Karma-Change Notifications

  The re­ver­sal test is with re­spect to the norm, not with re­spect to ways of han­dling a fixed norm. So imag­ine that the norm is the op­po­site, and see what will hap­pen. Peo­ple will in­vent weird things like gag­ing pop­u­lar­ity based on num­ber of down­votes, or sum of ab­solute val­ues of up­votes and down­votes, when there are not enough down­votes. This will work about as well as what hap­pens with the pre­sent norm. In that con­text, the op­tion of “only up­votes” looks funny and pointless, but we can see that it ac­tu­ally isn’t, be­cause we can look from the point of view of both pos­si­ble norms.

  When an ar­gu­ment goes through in the world of the op­po­site sta­tus quo, we can trans­port it to our world. In this case, we ob­tain the ar­gu­ment that “only down­votes” is not par­tic­u­larly funny and pointless, in­stead it’s about as ser­vice­able (or about as funny and pointless) as “only up­votes”, and both are not very good.

• Vladimir_Nesov 19 Mar 2019 3:59 UTC

  6 points

  in reply to: Raemon's comment on: Privacy

  I have some prob­a­bil­ity on me be­ing the con­fused one here.

  In con­ver­sa­tions like this, both sides are con­fused, that is don’t un­der­stand the other’s point, so “who is the con­fused one” is already an in­cor­rect fram­ing. One of you may be fac­tu­ally cor­rect, but that doesn’t re­ally mat­ter for mak­ing a con­ver­sa­tion work, un­der­stand­ing each other is more rele­vant.

  (In this par­tic­u­lar case, I think both of you are cor­rect and fail to see what the other means, but Jes­sica’s point is harder to fol­low and pat­tern-matches mis­lead­ing things, hence the bal­ance of votes.)

• Vladimir_Nesov 10 Mar 2019 0:32 UTC

  4 points

  in reply to: rohinmshah's comment on: How dan­ger­ous is it to ride a bi­cy­cle with­out a helmet?

  Sure, for vot­ing the effect on de­ci­sion mak­ing is greater. I’m just sus­pi­cious of this whole idea of acausal im­pact, and mod­er­ate ob­ser­va­tions about effect size don’t help with that con­fu­sion. I don’t think it can ap­ply to vot­ing with­out ap­ply­ing to other things, so the quan­ti­ta­tive dis­tinc­tion doesn’t point in a par­tic­u­lar di­rec­tion on cor­rect­ness of the over­all idea.

• Vladimir_Nesov 9 Mar 2019 23:35 UTC

  2 points

  in reply to: TheWakalix's comment on: How dan­ger­ous is it to ride a bi­cy­cle with­out a helmet?

  New in­for­ma­tion ar­gues for a change on the mar­gin, so the new equil­ibrium is differ­ent, though it may not be far away. The ar­gu­ments are not “can­cel­led out”, but they do only have bounded im­pact. Com­pare with char­ity eval­u­a­tion in effec­tive al­tru­ism: if we take the im­pact of cer­tain de­ci­sions as suffi­ciently sig­nifi­cant, it calls for their or­ga­nized study, so that the de­ci­sions are no longer made based on first im­pres­sions. On the other hand, if there is already enough in­fras­truc­ture for mak­ing good de­ci­sions of that type, then sig­nifi­cant changes are un­nec­es­sary.

  In the case of acausal im­pact, large refer­ence classes im­ply that at least that many peo­ple are already af­fected, so if or­ga­nized eval­u­a­tion of such de­ci­sions is fea­si­ble to set up, it’s prob­a­bly already in place with­out any need for the acausal im­pact ar­gu­ment. So ac­tual changes are prob­a­bly in how you pay at­ten­tion to info that’s already available, not in cre­at­ing in­fras­truc­ture for gen­er­at­ing bet­ter info. On the other hand, a source of info about sizes of refer­ence classes may be use­ful.

• Vladimir_Nesov 9 Mar 2019 23:12 UTC

  6 points

  in reply to: rohinmshah's comment on: How dan­ger­ous is it to ride a bi­cy­cle with­out a helmet?

  The ab­solute size of a refer­ence class only gives the prob­lem state­ment for an in­di­vi­d­ual de­ci­sion some al­tru­is­tic/​pa­ter­nal­is­tic tilt, which can fail to change it. Greater rel­a­tive size of a refer­ence class in­creases the de­ci­sion’s rel­a­tive im­por­tance com­pared to other de­ci­sions, which on the mar­gin should pull some effort away from the other de­ci­sions.

  That the effec­tive mul­ti­plier due to acausal co­or­di­na­tion is smaller for non-vot­ing de­ci­sions doesn’t in­form the ques­tion of whether the ar­gu­ment ap­plies to non-vot­ing de­ci­sions. The ar­gu­ment may be ig­nored in the de­ci­sion al­gorithm only if the refer­ence class is always small or about the same size for differ­ent de­ci­sions.

• Vladimir_Nesov 9 Mar 2019 20:03 UTC

  2 points

  in reply to: habryka's comment on: How dan­ger­ous is it to ride a bi­cy­cle with­out a helmet?

  That in­fluences sizes of refer­ence classes, but at some point the sizes cash out in morally rele­vant ob­ject level de­ci­sions.

• Vladimir_Nesov 9 Mar 2019 6:01 UTC

  2 points

  in reply to: habryka's comment on: How dan­ger­ous is it to ride a bi­cy­cle with­out a helmet?

  The mag­ni­tude de­pends on the sizes of refer­ence classes, which differ dra­mat­i­cally. So some per­sonal de­ci­sions are sud­denly much more im­por­tant than oth­ers sim­ply be­cause more peo­ple make them, and so you should al­lo­cate more re­sources on de­cid­ing those things in par­tic­u­lar cor­rectly. Ex­er­cise reg­i­men seems like a high acausal im­pact de­ci­sion. Another differ­ence is that the goal that the per­sonal de­ci­sions pur­sue shifts from what you want to hap­pen to your­self, to what you want to hap­pen to other peo­ple, and this effect in­creases with pop­u­la­tion. (Edited the grand­par­ent to ex­press these points more clearly.)

• Vladimir_Nesov 9 Mar 2019 5:24 UTC

  8 points

  in reply to: Wei_Dai's comment on: How dan­ger­ous is it to ride a bi­cy­cle with­out a helmet?

  In an old post I ar­gued that for acausal co­or­di­na­tion rea­sons it seems as if you should fur­ther mul­ti­ply this value by the num­ber of peo­ple in the refer­ence class of those mak­ing the de­ci­sion the same way (dis­counted by how lit­tle you care about strangers vs. your­self). This makes de­ci­sions about things that only af­fect you per­son­ally de­pend on the rel­a­tive sizes of their refer­ence classes and on to­tal pop­u­la­tion (greater pop­u­la­tion shifts fo­cus of the de­ci­sions fur­ther away from your­self). Your de­ci­sion in­flicts the micro­morts not just on your­self, but on all the peo­ple in the refer­ence class, for the pro­por­tion­ally greater to­tal num­ber of micro­morts that given this con­sid­er­a­tion turn into ac­tual morts very eas­ily.

  The idea doesn’t seem to have taken root, peo­ple talk about this ar­gu­ment mostly in the con­text of vot­ing, where it’s com­fort­ing for the ar­gu­ment to hold, even though it seems to ap­ply in gen­eral, where it de­mands mon­strous re­spon­si­bil­ity for ev­ery tiny lit­tle thing. It’s very sus­pi­cious, but I don’t know how to re­solve the con­fu­sion. Maybe it’s just psy­cholog­i­cally un­re­al­is­tic to fol­low through in al­most all cases where the ar­gu­ment ap­plies, de­spite its nor­ma­tive cor­rect­ness.

• Vladimir_Nesov 6 Mar 2019 1:08 UTC

  2 points

  in reply to: Rob Bensinger's comment on: Karma-Change Notifications

  The main thing I like about the ‘only down­votes’ op­tion is that it’s kind of funny and pointless.

  I feel the same about the ‘only up­votes’ op­tion. Ap­ply­ing the re­ver­sal test, imag­ine that most peo­ple treat the ‘only down­votes’ op­tion se­ri­ously and sug­gest that it should be the de­fault, since it agrees with the usual norms of in-per­son con­ver­sa­tion. Down­votes could even mea­sure pop­u­lar­ity if there was enough vol­ume, in the mean­time the sum of ab­solute val­ues of up­votes and down­votes can play that role.

• Vladimir_Nesov 2 Mar 2019 13:03 UTC

  9 points

  on: Karma-Change Notifications

  I’d ap­pre­ci­ate the fea­ture that re­stricts the no­tifi­ca­tions to the votes on com­ments posted at most X months ago (with X con­figurable in the set­tings). As it is, I’ll mostly get noise, no­tifi­ca­tions for the com­ments posted 8-11 years ago that I’m not cur­rently learn­ing from. (At least I ex­pect this to be the case and the first batch of no­tifi­ca­tions sup­ports this.)

• Vladimir_Nesov 28 Feb 2019 13:16 UTC

  2 points

  on: Rule Thinkers In, Not Out

  The ap­par­ent al­ter­na­tive to the re­li­able vs. New­ton trade­off when you are the thinker is to put ap­pro­pri­ate epistemic sta­tus around the hy­pothe­ses. So you pub­lish the book on Bible codes or all-pow­er­ful Vi­tamin C, but note in the pref­ace that you re­main ag­nos­tic about whether any ver­sion of the main the­sis ap­plies to the real world, pend­ing fur­ther de­vel­op­ment. You build a the­ory to ex­pe­rience how it looks once it’s more de­vel­oped, and pub­lish it be­cause it was sub­stan­tial work, even when upon pub­li­ca­tion you still don’t know if there is a ver­sion of the the­ory that works out.

  Maybe the the­ory is just beau­tiful, and that beauty doesn’t much diminish from its falsity. So call it philo­soph­i­cal fic­tion, not a de­scrip­tion of this world, the sub­stan­tial ac­tivity of de­vel­op­ing the the­ory and com­mu­ni­cat­ing it re­mains the same with­out sac­ri­fic­ing re­li­a­bil­ity of your ideas. There might even be a place for an ed­ifice of such fic­tions that’s similar to math in map­ping out an as­pect of the world that doesn’t con­nect to the phys­i­cal re­al­ity for very long stretches. This doesn’t seem plau­si­ble in the cur­rent prac­tice, but seems pos­si­ble in prin­ci­ple, so even call­ing such ac­tivity “fic­tion” might be mis­lead­ing, it’s more than mere fic­tion.

  I don’t think hy­per­sen­si­tive pat­tern-match­ing does a lot to de­stroy abil­ity to dis­t­in­guish be­tween an idea that you feel like pur­su­ing and an idea that you see as more re­li­ably con­firmed to be ap­pli­ca­ble in the real world. So you can dis­cuss this dis­tinc­tion when com­mu­ni­cat­ing such ideas. Maybe the au­di­ence won’t listen to the dis­tinc­tion you are mak­ing, or won’t listen be­cause you are mak­ing this dis­tinc­tion, but that’s a differ­ent is­sue.

• Vladimir_Nesov 18 Feb 2019 12:53 UTC

  2 points

  AF

  in reply to: Stuart_Armstrong's comment on: Why we need a *the­ory* of hu­man values

  Yes, that’s the al­most fully gen­eral coun­ter­ar­gu­ment: punt all the prob­lems to the wiser ver­sions of our­selves.

  It’s not clear what the rele­vant differ­ence is be­tween then and now, so the ar­gu­ment that it’s more im­por­tant to solve a prob­lem now is as sus­pect as the ar­gu­ment that the prob­lem should be solved later.

  How are we cur­rently in a bet­ter po­si­tion to in­fluence the out­come? If we are, then the rea­son for be­ing in a bet­ter po­si­tion is a more im­por­tant fea­ture of the pre­sent situ­a­tion than ob­ject-level solu­tions that we can pro­duce.

• Vladimir_Nesov 18 Feb 2019 12:44 UTC

  9 points

  in reply to: EulersApprentice's comment on: Limit­ing an AGI’s Con­text Temporally

  It could throw a pa­per­clip max­i­mizer at you.

• Vladimir_Nesov 16 Jan 2019 3:21 UTC

  6 points

  in reply to: dxu's comment on: Open Thread Jan­uary 2019

  So your de­ci­sion doesn’t just de­ter­mine the fu­ture; it also de­ter­mines (with high prob­a­bil­ity) which you “you” are.

  Worse. It doesn’t change who you are, you are the per­son be­ing black­mailed. This you know. What you don’t know is whether you ex­ist (or ever ex­isted). Whether you ever ex­isted is de­ter­mined by your de­ci­sion.

  (The dis­tinc­tion from the quoted sen­tence may mat­ter if you put less value on the wor­lds of peo­ple slightly differ­ent from your­self, so you may pre­fer to en­sure your own ex­is­tence, even in a black­mailed situ­a­tion, over the ex­is­tence of the al­ter­na­tive you who is not black­mailed but who is differ­ent and so less valuable. This of course in­volves un­sta­ble val­ues, but mo­ti­vates de­gree of ex­is­tence phras­ing of the effect of de­ci­sions, over the change in the con­tent of the world phras­ing, since the lat­ter doesn’t let us weigh whole al­ter­na­tive wor­lds differ­ently.)