rossry

• rossry 29 Apr 2019 14:30 UTC

  3 points

  in reply to: Virgil Kurkjian's comment on: Counterspells

  cf. my com­ment cousin to this one; I mi­s­un­der­stood what the term was point­ing at at first, though I stand by my com­plaint that that’s a prob­lem with the term.

• rossry 28 Apr 2019 14:34 UTC

  3 points

  in reply to: Slider's comment on: Counterspells

  Thanks; I le­gi­t­i­mately mi­s­un­der­stood at first read whether “coun­ter­spell” was in­tended to ap­ply to the in­vo­ca­tions thrown out by bad ar­guers or the con­cise and spe­cific dis­til­la­tions the OP is pre­sent­ing for use. On re-read, I agree that it’s sup­posed to be a set of use­ful tools.

  I re­main con­vinced of the spe­cific claim that “coun­ter­spell” is bad jar­gon (though I don’t think it’s good prac­tice to cite my own con­fu­sion too strongly; the in­cen­tives there aren’t great). I agree that MtG″s paradigm where more gen­eral coun­ter­spells are more ex­pen­sive seems like a good fit for think­ing about rhetor­i­cal (and per­haps epistemic) tac­tics, though I re­it­er­ate that that’s not how they work in many other set­tings, and that am­bigu­ous bag­gage is worse than no bag­gage for this sort of thing. The ques­tion of whether iden­ti­fy­ing coun­ter­spells with magic is sup­posed to be a pos­i­tive or nega­tive as­so­ci­a­tion is ad­di­tional gra­tu­itous con­fu­sion—I think your claim that the magic metaphor im­plies they don’t work is wrong, but I’m not 85% sure.

• rossry 28 Apr 2019 4:40 UTC

  16 points

  on: Counterspells

  Strong ap­proval of the over­all goal of the post, but here’s a se­man­tic crit­i­cism:

  In ac­cor­dance with the Ra­tion­al­ist tra­di­tion that re­quires ev­ery­thing to have a nerdy sci-fi or fan­tasy name

  I parse this as an in-joke (and ap­pre­ci­ate it as such), but I do think that reg­u­larly mint­ing new jar­gon that’s likely to carry sub­stan­tial, con­flict­ing(!) con­tex­tual bag­gage (not all of it ap­pro­pri­ate[1]) is...a bad norm for an epistemic com­mu­nity to have.

  I also think that the deeper tra­di­tion of jar­gon-forg­ing (as Eliezer prac­ticed it) in­volved names that sounded nerdy, but _not_ sci-fi or fan­tasy -- _cf._ Know­ing About Bi­ases Can Hurt Peo­ple, which uses “Fully Gen­eral Coun­ter­ar­gu­ment” in much the same way(?) as you’re us­ing “Coun­ter­spell”. “Fully Gen­eral Coun­ter­ar­gu­ment” is slightly more un­wieldy, but apart from that is a bet­ter piece of jar­gon—it’s less loaded and by lean­ing even harder into the im­plicit snark that winks at the fact that the pur­ported counter- isn’t work­ing at all, makes it even more clear that no, this is not a use­ful thing.

  [1] To be­la­bor this point, MtG coun­ter­spells are fully gen­eral (edit: okay, not **fully** gen­eral, and see Slider be­low), and a rea­son­able fit for the term as you’re us­ing it, but D&D (3.5e, at least) coun­ter­spells are based on negat­ing a spell by cast­ing a copy of it, which is not what you mean at all. I don’t ac­tu­ally know what “coun­ter­spells” are in WoD/​Mage, but the risk that they’re even fur­ther afield from your in­ten­tion should be an­other strike against us­ing the already-loaded han­dle.

• rossry 28 Apr 2019 4:23 UTC

  1 point

  in reply to: avturchin's comment on: Open Thread April 2019

  prefer­ence “my de­ci­sions should be mine”—and many peo­ple seems to have it

  Fair. I’m not sure how to for­mal­ize this, though—to my in­tu­ition it seems con­fused in roughly the same way that the con­cept of free will is con­fused. Do you have a way to for­mal­ize what this means?

  (In the ab­sence of a com­pel­ling de­con­fu­sion of what this goal means, I’d be wary of hack­ing epistemics in defense of it.)

  There are “friends” who claim to have the same goals as me, but later turns out that they have hid­den mo­tives.

  Agreed and agreed that there’s a benefit to re­mov­ing their af­for­dance to ex­ploit you. That said, why does this de­serve more at­ten­tion than the in­verse case (there are par­ties you do not trust who later turn out to have be­nign mo­tives)?

• rossry 27 Apr 2019 2:57 UTC

  3 points

  in reply to: avturchin's comment on: Open Thread April 2019

  “give the power over my fi­nal de­ci­sions to small ran­dom events around me” seems like a slightly con­fused con­cept if your prefer­ences are truly in­differ­ent. Can you say more about why you see that as a prob­lem?

  The po­ten­tial ad­ver­sary seems like a more straight­for­ward prob­lem, though one ex­cit­ing pos­si­bil­ity is that light­ness of de­ci­sions lets a po­ten­tial co­op­er­a­tor ma­nipu­late your de­ci­sions on fa­vor of your com­mon in­ter­ests. And pre­sum­ably you already have some sys­tem for fil­ter­ing ac­quain­tances into ad­ver­saries and co­op­er­a­tors. Is the con­cern that your fil­ter­ing is faulty, or some­thing else?

  [Com­mit­ment] even­tu­ally of­ten turn to be win­ning strat­egy, com­pared to the flex­ible strat­egy of con­stant up­dat­ing ex­pected util­ity.

  Some real-world games are re­ducible to the game of Chicken. Com­mit­ment is of­ten a win­ning strat­egy in them. Though I’m not cer­tain that it’s a com­mit­ment to a par­tic­u­lar set of be­liefs about util­ity so much as a more-com­plex de­ci­sion the­ory which sits be­tween util­ity be­liefs and ac­tions.

  In sum­mary, if the ac­quain­tances whose info you up­date on are suffi­ciently un­al­igned with you and your de­ci­sion the­ory always se­lects the ad­di­tion that your pos­te­rior as­signs the high­est util­ity, then your ac­tions will be “over-up­dat­ing on the ev­i­dence” if your be­liefs are prop­erly Bayesian. But I don’t think the best re­sponse is to bias your­self to­wards un­der-up­dat­ing.

• rossry 23 Apr 2019 23:22 UTC

  3 points

  in reply to: clone of saturn's comment on: Mo­ral Weight Doesn’t Ac­crue Linearly

  Do you bite the bul­let that this means the set of things you morally value changes dis­con­tin­u­ously and pre­dictably as things move out of your light cone? (Or is there some way you value things less as they are less “in” your light cone, in some non­bi­nary way?)

• rossry 23 Apr 2019 23:15 UTC

  4 points

  on: On the Na­ture of Pro­gram­ming Languages

  I think it’s from SICP that pro­grams are meant to be read by hu­mans and only in­ci­den­tally for com­put­ers to ex­e­cute; I’ve been try­ing for more than a year now to write a blog post about the fun­da­men­tal premise that, effort-weighted, we al­most never write new pro­grams from scratch, and mostly are en­gaged in trans­mut­ing one work­ing pro­gram into an­other work­ing pro­gram. Pro­grams are not only meant to be read by hu­mans, but ed­ited by hu­mans.

  I think if you start from the ques­tion of how much effort it is to write a new pro­gram on a blank page, most lan­guages will come out look­ing the same, and the differ­ences will look like psy­cholog­i­cal con­structs. If you ask, how­ever, how much effort it is to change an ex­ist­ing piece of a code base to a spe­cific some­thing else, you start to see differ­ences in epistemic struc­ture, where it mat­ters how many of the pos­si­ble mu­ta­tions that a hu­man al­gorithm might try will non-ob­vi­ously make the re­sult­ing pro­gram do some­thing un­ex­pected. And that, as you point out, opens the door to at least some no­tion of uni­ver­sal­ity.

• rossry 23 Apr 2019 22:55 UTC

  3 points

  in reply to: ryan_b's comment on: On the Na­ture of Pro­gram­ming Languages

  C, the most wide­spread gen­eral-pur­pose pro­gram­ming lan­guage, does things that are ex­tremely difficult or im­pos­si­ble in highly ab­stract lan­guages like Haskell or LISP

  Can you give an ex­am­ple? I’m sur­prised by this claim, but I only have deep fa­mil­iar­ity with C of these three. (My pri­mary func­tional lan­guage in­cludes muta­ble con­structs; I don’t know how purely func­tional lan­guages fare with­out them.)

• rossry 9 Apr 2019 23:40 UTC

  1 point

  in reply to: cousin_it's comment on: Open Thread April 2019

  I was try­ing to con­struct a proof along similar lines, so thank you for beat­ing me to it!

  Note that 2 is ac­tu­ally a case of 1, since you can think of the “walls” of the sim­plex as be­ing bets that the uni­verse offers you (at zero odds).

• rossry 9 Apr 2019 11:34 UTC

  1 point

  in reply to: Oscar_Cunningham's comment on: Open Thread April 2019

  (This com­ment isn’t an an­swer to your ques­tion.)

  If I’m un­der­stand­ing prop­erly, you’re try­ing to use the set of bets offered as ev­i­dence to in­fer the com­mon be­liefs of the mar­ket that’s offer­ing them. Yet from a Bayesian per­spec­tive, it seems like you’re as­sign­ing P( X offers bet B | bet B has pos­i­tive ex­pec­ta­tion ) = 0. While that’s liter­ally the state­ment of the Effi­cient Mar­kets Hy­poth­e­sis, pre­sum­ably you—as a Bayesian—don’t ac­tu­ally be­lieve the prob­a­bil­ity to be liter­ally 0.

  Get­ting this right and gen­er­al­iz­ing a bit (pre­sum­ably, you think that P( X offers B | B has ex­pec­ta­tion +ep­silon ) < P( X offers B | B has ex­pec­ta­tion +BIG_E )), should make the mar­ket ev­i­dence more in­for­ma­tive (and cases of ar­bi­trage less di­vide-by-zero, break-your-math con­fus­ing).

• rossry 9 Apr 2019 9:46 UTC

  1 point

  in reply to: Gurkenglas's comment on: Open Thread April 2019

  I’m con­fused what the word “fairly” means in this sen­tence.

  Do you mean that they make a zero-ex­pected-value bet, e.g., 1:1 odds for a fair coin? (Then “fairly” is too strong; non-de­gen­er­ate odds (i.e., not zero on ei­ther side) is the ac­tual re­quired con­di­tion.)

  Do you mean that they bet with­out fraud, such that one will get a pos­i­tive pay­out in one out­come and the other will in the other? (Then I think “fairly” is re­dun­dant, be­cause I would say they haven’t ac­tu­ally bet on the out­come of the coin if the pay­outs don’t cor­re­spond to coin out­comes.)

• rossry 17 Mar 2019 1:53 UTC

  1 point

  in reply to: rossry's comment on: How can we re­spond to info-cas­cades? [Info-cas­cade se­ries]

  A re­lated idea in non-pun­ish­ment of “wrong” re­ports that have in­suffi­cient sup­port (again in the com­mon-prior/​pri­vate-info set­ting) comes from this pa­per [pdf] (pre­sented at the same con­fer­ence), which sug­gests col­lect­ing re­ports from all agents and as­sign­ing re­wards/​pun­ish­ments by as­sum­ing that agents’ re­ports rep­re­sent their pri­vate sig­nal, com­put­ing their pos­te­rior, and scor­ing this as­sumed pos­te­rior. Un­der the model as­sump­tions, this makes it an op­ti­mal strat­egy for agents to truly re­veal their pri­vate sig­nal to the mechanism, while al­low­ing the mechanism to col­lect non-cas­caded base data to make a de­ci­sion.

  In gen­eral, I feel like the aca­demic liter­a­ture on mar­ket de­sign /​ mechanism de­sign has a lot to say about ques­tions of this fla­vor.

• rossry 17 Mar 2019 1:34 UTC

  4 points

  on: How can we re­spond to info-cas­cades? [Info-cas­cade se­ries]

  Ab­stract: Con­sid­er­ing in­for­ma­tion cas­cades (both up­wards and down­wards) as a prob­lem of in­cen­tives, bet­ter in­cen­tive de­sign holds some promise. This aca­demic pa­per sug­gests a model in which mak­ing truth-find­ing re­wards con­tin­gent on reach­ing a cer­tain num­ber of votes pre­vents down-cas­cades, and where an in­formed (self-in­ter­ested) choice of pay­out odds and thresh­old can also pre­vent up-cas­cades in the limit of a large pop­u­la­tion of pre­dic­tors.

  1) cf. av­turchin from the ques­tion about dis­tri­bu­tion across fields, point­ing out that up-cas­cades and down-cas­cades are both rele­vant con­cerns, in many con­texts.

  2) Con­sider in­for­ma­tion cas­cades as re­lated to a prob­lem of in­cen­tives—in the com­ments of the Johni­chols post refer­enced in the for­mal­iza­tion ques­tion, mul­ti­ple com­men­ta­tors point out that the model fails if agents seek to ex­press their marginal opinion, rather than their true (pos­te­rior) be­lief. But in­cen­tives to be right do need to be built into a sys­tem that you’re try­ing to pump en­ergy into, so the ques­tion re­mains of whether a differ­ent in­cen­tive struc­ture could do bet­ter, while still en­courag­ing truth-find­ing.

  3) Up-Cas­caded Wis­dom of the Crowd (Cong and Xiao, work­ing pa­per) con­sid­ers the in­for­ma­tion-ag­gre­ga­tion prob­lem in terms of in­cen­tives, and con­sider the in­cen­tives at play in an all-or-noth­ing crowd­fund­ing model, like ven­ture cap­i­tal or Kick­starter (as­sum­ing that a ‘no’ vote is ir­re­vo­ca­ble like a ‘yes’ vote is) -- ‘yes’ vot­ers win if there is a crit­i­cal mass of other ‘yes’ vot­ers and the propo­si­tion re­solves to ‘yes’; they lose if there is a crit­i­cal mass and the propo­si­tion re­solves to ‘no’; they have 0 loss/​gain if ‘yes’ doesn’t reach a crit­i­cal mass; ‘no’ vot­ers are merely ab­stain­ing from vot­ing ‘yes’.

  Their main re­sult is that if the pay­ment of in­cen­tives is con­di­tioned on the propo­si­tion gain­ing a fixed num­ber of ‘yes’ votes, a pop­u­la­tion of sym­met­ric, com­mon-prior/​pri­vate-info agents will avoid down-cas­cades, as a sin­gle ‘yes’ vote that breaks a down-cas­cade will not be pe­nal­ized for be­ing wrong un­less some later agent in­ten­tion­ally votes ‘yes’ to put the vote over the ‘yes’ thresh­old. (An agent i with nega­tive pri­vate info still should vote no, be­cause if a later agent i’ puts the vote over the ‘yes’ thresh­old based in part on i’s false vote, then i ex­pects to lose on the truth-eval­u­a­tion, since they’ve backed ‘yes’ but be­lieve ‘no’.)

  A fur­ther re­sult from the same pa­per is that if the ac­tor pos­ing the propo­si­tion can set the pay­out odds and the thresh­old in re­sponse to the com­mon prior and known info-dis­tri­bu­tion, then a propo­si­tion-poser at­tempt­ing to min­i­mize down-cas­cades (per­haps be­cause they will cast the first ‘yes’ vote, and so can only hope to win if the vote re­solves to ‘yes’) will be in­cen­tivized to set odds and a thresh­old that co­in­ci­den­tally min­i­mize the chance of up-cas­cades. In the large-pop­u­la­tion limit, the num­ber of cas­cades un­der such an in­cen­tive de­sign goes to 0.

  4) I sus­pect (but will not here prove) that aug­ment­ing Cong and Xiao’s all-or-noth­ing “crowd­fund­ing for ‘yes’” de­sign with a par­allel “crowd­fund­ing for ‘no’” de­sign—i.e., ‘no’ vot­ers win (resp. lose) iff there is a crit­i­cal mass of ‘no’ vot­ers and the propo­si­tion re­solves ‘no’ (resp. ‘yes’) -- can fur­ther strengthen the defenses against up-cas­cades (by mak­ing it pos­si­ble to cast a more in­formed ‘no’ vote con­di­tioned on a later, more-in­formed agent de­cid­ing to put ‘no’ over the thresh­old).

• rossry 17 Mar 2019 1:30 UTC

  0 points

  on: How can we re­spond to info-cas­cades? [Info-cas­cade se­ries]

  [this an­swer was du­pli­cated when I mis­tak­enly copied my com­ment into an an­swer and then moved the com­ment to an an­swer.]

• rossry 17 Mar 2019 0:14 UTC

  1 point

  in reply to: avturchin's comment on: Distri­bu­tion of info-cas­cades across fields? [Info-cas­cade se­ries]

  Why stop at 2? Belief-space is large, and many is­sues ad­mit more than one (+/​-) bit of in­for­ma­tion to cas­cade.

• rossry 2 Mar 2019 2:03 UTC

  1 point

  in reply to: avturchin's comment on: Open Thread Fe­bru­ary 2019

  But how make small changes in the tra­jec­tory of a star? One idea is to im­pact the star with large comets. It is not difficult, as re­mote Oort cloud ob­jects (or wan­der­ing small planets, as they are not part of already es­tab­lished or­bital move­ment of the star) need only small per­tur­ba­tions to start fal­ling down on the cen­tral star, which could be done via nu­clear ex­plo­sions or smaller im­pacts by smaller as­treoids.

  I don’t think this works; con­ser­va­tion of mo­men­tum means that the im­pact is al­most fully coun­ter­acted by the grav­i­ta­tional pull that ac­cel­er­ated the comet to such speed (so that, in the end, the delta-v im­parted to the star is pre­cisely what you im­parted with your nu­clear ex­plo­sions or smaller as­ter­oids).

  Maybe a Shkadov thruster is what you want? (It’s slow go­ing, though; this ar­ti­cle sug­gests 60ly/​200My, ac­count­ing for ac­cel­er­a­tion.)

• rossry 23 Feb 2019 10:35 UTC

  1 point

  in reply to: TheWakalix's comment on: Blackmail

  Win­ner’s Curse doesn’t seem like the right effect to me—it seems more like an or­thog­o­nal­ity/​Good­hart effect, where op­ti­miz­ing for out­ra­geous­ness de­creases the fit­ness w/​r/​t so­cial welfare (on the mar­gin). It’s always in the black­mailer’s in­ter­est to make the out­ra­geous­ness greater, so they’re not (self­ishly) sad when they over­shoot.

• rossry 19 Feb 2019 2:11 UTC

  8 points

  in reply to: ozziegooen's comment on: Can We Place Trust in Post-AGI Fore­cast­ing Eval­u­a­tions?

  I’m still think­ing about what quan­ti­ta­tive es­ti­mates I’d stand be­hind. I think I’d be­lieve that a prize-based com­pet­i­tive pre­dic­tion sys­tem with all eval deferred un­til and con­di­tioned on AGI is <4% to add more than $1mln of value to [just pay some smart par­ti­ci­pants for their best-efforts opinions].

  (If I thought harder about cor­ner-cases, I think I could come up with a stronger state­ment.)

• rossry 18 Feb 2019 23:39 UTC

  3 points

  in reply to: ozziegooen's comment on: Can We Place Trust in Post-AGI Fore­cast­ing Eval­u­a­tions?

  If the rea­son your ques­tions won’t re­solve is that you are dead or that none of your money at all will be use­ful, I think things are a bit differ­ent.

  I’m con­fused; to restate the above, I think that a p% chance that your pre­dic­tions don’t mat­ter (for any rea­son: game rained out, you’re dead, your money isn’t use­ful) is (to first or­der) equiv­a­lent to a p% tax on in­vest­ment in mak­ing bet­ter pre­dic­tions. What do you think is differ­ent?

  one ma­jor ask is that the fore­cast­ers be­lieve the AGI will hap­pen in be­tween, which seems to me like an even big­ger issue

  Sure, that’s an is­sue, but I think that re­quiring par­ti­ci­pants to all as­sume short AGI timelines is tractable in a way that the de­layed/​im­prob­a­ble re­s­olu­tion is­sues are not.

  I can imag­ine a mar­ket with­out re­s­olu­tion is­sues that as­sumes par­ti­ci­pants all be­lieve short AGI timelines could sup­port 12 semi-pro­fes­sional traders sub­si­dized by in­ter­ested stake­hold­ers. I don’t be­lieve that a mar­ket with re­s­olu­tion is­sues as above can elicit se­ri­ous in­vest­ment in get­ting its an­swers right from half that many. (I rec­og­nize that I’m elid­ing my defi­ni­tion of “se­ri­ous in­vest­ment” here.)

• rossry 18 Feb 2019 14:39 UTC

  3 points

  in reply to: ozziegooen's comment on: Can We Place Trust in Post-AGI Fore­cast­ing Eval­u­a­tions?

  I un­der­stand Zvi’s points as be­ing rel­a­tively uni­ver­sal to sys­tems where you want to use re­wards to in­cen­tivize par­ti­ci­pants to work hard to get good an­swers.

  No mat­ter how the pay­outs work, a p% chance that your ques­tions don’t re­solve is (to first or­der) equiv­a­lent to a p% tax on in­vest­ment in mak­ing bet­ter pre­dic­tions, and a years-long tie-up kills iter­a­tive growth and se­lec­tion/​am­plifi­ca­tion cy­cles as well limit­ing the re­turn-on-in­vest­ment-in-gen­eral-pre­dic­tion-skill to a one-shot game. I don’t think these is­sues go away if you re­ward pre­dic­tions differ­ently, since they’re gen­eral fea­tures of the re­la­tion be­tween the up-front in­vest­ment in mak­ing bet­ter pre­dic­tions and the to-come po­ten­tial re­ward for do­ing so well.

  (A coun­ter­point I’ll en­ter­tain is Zvi’s caveat to “quick re­s­olu­tion”—which also caveats “prob­a­ble re­s­olu­tion”—that suffi­cient liquidity can sub­sti­tute for re­s­olu­tion. But boot­strap­ping that liquidity it­self seems like a Hard Prob­lem, so I’d need to fur­ther be con­vinced that it’s tractable here.)