Dagon

• Dagon 22 Mar 2019 23:17 UTC

  6 points

  on: Can a Bayesian agent be in­finitely con­fused?

  This math is ex­actly why we say a ra­tio­nal agent can never as­sign a perfect 1 or 0 to any prob­a­bil­ity es­ti­mate. Do­ing so in a uni­verse which then pre­sents you with coun­terev­i­dence means you’re not ra­tio­nal.

  Which I sup­pose could be termed “in­finitely con­fused”, but that feels like a mix­ing of lev­els. You’re not con­fused about a given prob­a­bil­ity, you’re con­fused about how prob­a­bil­ity works.

  In prac­tice, when a well-cal­ibrated per­son says 100% or 0%, they’re round­ing off from some un­speci­fied-pre­ci­sion es­ti­mate like 99.9% or 0.000000000001.

• Dagon 22 Mar 2019 23:06 UTC

  5 points

  on: The Game The­ory of Blackmail

  This map­ping does not match any ac­tual de­ci­sions in black­mail. First, it’s not a si­mul­ta­neous choice, it’s a branch­ing multi-turn de­ci­sion tree. Se­cond, there are more than 2 ac­tions available at var­i­ous stages. Either of these would make pris­oner’s dilemma anal­y­sis sus­pect, to­gether it be­comes much more like multi-street multi-bet poker than like PD.

  The “vic­tim” first makes choices (or is born into a situ­a­tion) sus­cep­ti­ble to black­mail. The black­mailer learns of this, and has at least 3 choices: pub­lish the in­for­ma­tion, threaten to pub­lish, or bury the in­for­ma­tion. The “vic­tim” in the threaten-to-pub­lish (black­mail) case offers in­cen­tives (which may be the same as the re­quested fee, or may not) to bury rather than pub­lish, and the black­mailer chooses which ac­tion to take. Even leav­ing out true defec­tion cases (ac­cept the money and pub­lish any­way, or kil­ling the black­mailer), this is a fairly com­plex pay­out tree, and the cor­rect choices are spe­cific to the situ­a­tion. In fact, since parts of the pay­out tree are un­known to one or both play­ers, it’s likely that mixed strate­gies come into play, to pre­vent ex­ploita­tion of the un­knowns.

• Dagon 22 Mar 2019 17:00 UTC

  3 points

  on: In­tu­itions on Nega­tive Utilitarianism

  On the “a day in hell can­not be out­weighed” ques­tion, do you have any anal­y­sis of that in­tu­ition? Are you as­sum­ing that you’ll re­mem­ber that day and be bro­ken by it, or is there some other nega­tive value you’re putting on it. Do you eval­u­ate “a day in hell, 1000 years in heaven, then ter­mi­na­tion” differ­ently from “1000 years in heaven, then a day in hell, then ter­mi­na­tion”? How about “a day in hell, mind-re­set to prior state, then 1000 years in heaven”?

  The rea­son I ask is that I’m try­ing to un­der­stand what’s be­ing eval­u­ated. Are you com­par­ing value of in­stan­ta­neous ex­pe­rience in­te­grated over time, or are you com­par­ing value of effect that ex­pe­riences have on your iden­tity?

• Dagon 21 Mar 2019 17:09 UTC

  2 points

  in reply to: Sherrinford's comment on: What so­cieties have ever had le­gal or ac­cepted black­mail?

  Cor­rect. I sus­pect it’s too broad a cat­e­gory, and im­pre­cisely defined (or at least differ­ent from things com­monly mea­sured) to have much ba­sis OTHER than analo­gies and anec­dotes. For­tu­nately, it’s also ir­rele­vant to most of us—no ac­tion is pro­posed or ex­pected as part of this dis­cus­sion/​de­bate.

  [ edit: I’m em­barassed to say that un­til now I hadn’t even looked up the le­gal­ity of it. In the US, black­mail is speci­fi­cally about de­mand­ing pay­ment for not in­form­ing of a vi­o­la­tion of US law, not for any other top­ics of gos­sip. It’s also a rel­a­tively minor offence. Many ju­ris­dic­tions just treat it as a spe­cial case of ex­tor­tion, which is se­vere, but un­clear whether that’s be­cause it’s of­ten vi­o­lent or be­cause it’s bad on it’s own.]

• Dagon 19 Mar 2019 19:39 UTC

  2 points

  in reply to: Tater's comment on: Im­pli­ca­tions of liv­ing within a Simulation

  (Are trig­ger warn­ings still a thing? it oc­curs to me that this topic may in­ter­act badly with suici­dal thoughts. Please take it only as the silly ex­plo­ra­tion of imag­i­na­tion-space that it is).

  I don’t give a lot of weight to the basilisk pos­si­bil­ity—that was some­thing of a throw­away com­ment.

  What I meant is that if the truth is that the simu­la­tion con­trol­ler is speci­fi­cally in­ter­ested in you as an ex­pe­rience-haver within the simu­la­tion, then there is no pos­si­bil­ity to in­ten­tion­ally in­fluence the simu­la­tion. Your per­cep­tions and your cog­ni­tion will be ma­nipu­lated to make you be­lieve what­ever the simu­la­tor things fur­thers their goals. Your uni­verse of per­cep­tion and pos­si­ble ac­tions sim­ply won’t con­tain things that counter their goals.

  And one amus­ing (per­haps only to me) pos­si­bil­ity of mo­ti­va­tion for such a per­sonal simu­la­tion is that my cur­rent life is the worst that the at­tacker can imag­ine within the con­straints that I must be­lieve it’s real. Maybe I’m be­ing tor­tured, as pun­ish­ment for some out­side-uni­verse crime. I find it amus­ing, as I as­sign pos­i­tive value to this mo­ment of ex­pe­rience (which is the only thing I can be sure of), so the basilisk is be­ing thwarted in it’s mis­sion of pun­ish­ing me. And also be­cause it seems lu­dicrously un­likely.

• Dagon 18 Mar 2019 21:53 UTC

  −2 points

  in reply to: Raemon's comment on: Privacy

  [note: this sub­thread is far afield from the ar­ti­cle—LW is about pub­li­ca­tion, not pri­vate thoughts (un­less there’s a sec­tion I don’t know about where only speci­fi­cally in­vited peo­ple can see things) . And LW karma is far from the sanc­tions un­der dis­cus­sion in the rest of the post.]

  Have you con­sid­ered things to re­duce the as­sy­met­ric im­pact of up- and down-votes? Cap karma value at −5? Use down­votes as a di­vi­sor for up­votes (say, score is up­votes /​ (1 + 0.25 * down­votes)) rather than sim­ple sub­trac­tion?

• Dagon 18 Mar 2019 21:15 UTC

  2 points

  in reply to: avturchin's comment on: Im­pli­ca­tions of liv­ing within a Simulation

  In terms of in­for­ma­tional sim­plic­ity, there’s a mea­sure­ment bound at the limits of your mem­ory of per­cep­tions. You can’t dis­t­in­guish be­tween a simu­la­tion of quan­tum or atomic (or even sim­ple me­chan­i­cal) rules, from a tiny simu­la­tion of point-in-time set of mem­o­ries and ex­pe­rience. You liter­ally can­not know the differ­ence (and there may be no differ­ence—see the philo­soph­i­cal zom­bies de­bate) be­tween an NPC and a “real per­son”.

  If it turns out that this is your per­sonal Basilisk pun­ish­ment, and you’re be­ing tor­tured for not bring­ing forth the cre­ator in an­other uni­verse, there is no the­o­ret­i­cal or prac­ti­cal way to know that.

• Dagon 18 Mar 2019 20:35 UTC

  2 points

  on: Im­pli­ca­tions of liv­ing within a Simulation

  Do you di­rectly care about the goals of the simu­la­tor(s)? In the ab­sence of a cre­ator, how do you an­swer about evolu­tion (a goal-less op­ti­miza­tion pro­cess)? Are your goals de­pen­dent on a cre­ator, or in­de­pen­dent?

  You might not share any goals, but still care be­cause you fear in­terfer­ence or ter­mi­na­tion if the simu­la­tion no longer fur­thers the cre­ator’s goals. I ar­gue that this fear is un­founded. Whether you’re a side-effect or a pri­mary rea­son for the simu­la­tion, you’re still part of the en­vi­ron­ment be­ing simu­lated. If the cre­ators are both­er­ing to simu­late this level of de­tail, then they think this level of de­tail is im­por­tant to their goals.

  Any de­tectable change the cre­ators make in the simu­la­tion (like cre­at­ing or al­ter­ing the path of an as­ter­oid, tweak­ing hu­man be­hav­ior, com­mu­ni­cat­ing with some or all sub­jects, etc.) re­duces the simu­la­tion value of the simu­la­tion.

  It’s pos­si­ble they _WANT_ to simu­late a uni­verse with no in­tel­li­gent in­ter­stel­lar life. If so, they’d build the filters into the simu­la­tion, rather than notic­ing a prob­lem and chang­ing the code. They might no­tice a prob­lem, change the code, and ter­mi­nate/​restart the simu­la­tion, but I can’t imag­ine any way to guess the things that will ac­cel­er­ate or pre­vent this from hap­pen­ing. obDou­glasA­dams: “There is an­other the­ory which states that this has already hap­pened.”

• Dagon 18 Mar 2019 20:09 UTC

  5 points

  in reply to: Richard_Kennaway's comment on: Black­mailers are pri­va­teers in the war on hypocrisy

  I don’t know what my price would be, and I hope it’s too high to ever come into play. I like to think there’s no pos­si­ble situ­a­tion I’d turn some­one in, and I’d fa­vor the in­di­vi­d­ual over the mob at any cost to my­self. But that’s not true for the vast ma­jor­ity of hu­mans, and prob­a­bly not for me ei­ther.

  But we’re not talk­ing about heroes (even if I hope I would qual­ify and fear I wouldn’t). We’re talk­ing about the range of hu­man be­hav­ior and mo­ti­va­tion. It’s clear that so­cial pres­sure _is_ enough for some peo­ple to turn oth­ers in. Medals, re­wards, etc. likely in­crease that a lit­tle. Black­mail prob­a­bly de­creases it a lit­tle, as the data-hold­ers can now get paid for keep­ing se­crets rather than do­ing it in spite of in­cen­tives.

• Dagon 18 Mar 2019 17:38 UTC

  2 points

  in reply to: Richard_Kennaway's comment on: Black­mailers are pri­va­teers in the war on hypocrisy

  “com­pared to what?” should always be part of the anal­y­sis. In the ex­am­ples you give (un­just per­se­cu­tion if pri­vate in­for­ma­tion is pub­lished), I be­lieve you’d pre­fer black­mail to pub­li­ca­tion, and pre­fer un­paid silence to black­mail. It’s un­clear what in­tu­itions you have if there’s a so­cial or mon­e­tary re­ward for turn­ing them in. Is black­mail ac­cep­ti­ble, if it’s no more than the value of the fore­gone re­ward?

• Dagon 18 Mar 2019 17:31 UTC

  4 points

  in reply to: Bucky's comment on: What so­cieties have ever had le­gal or ac­cepted black­mail?

  Clar­ified—the hypocrisy is that black­mail is pro­hibited but not en­forced against. We claim that it’s bad, but al­low it most of the time. I could ar­gue that hypocrisy it­self falls into this cat­e­gory (we com­plain about it, but don’t ac­tu­ally pun­ish it) as well, but I didn’t in­tend to.

• Dagon 17 Mar 2019 17:19 UTC

  3 points

  in reply to: TheAncientGeek's comment on: Privacy

  Many laws in­cor­po­rate scal­ing in terms of dam­age thresh­old or mag­ni­tude of sin­gle in­ci­dent. We have very few laws that are ex­plicit about scale in terms of over­all fre­quency or num­ber of par­ti­ci­pants in mul­ti­ple in­ci­dents. City zon­ing may be one ex­am­ple of suc­cess in this area—only al­low­ing so many res­i­dents in an area, with­out spec­i­fy­ing who.

  There are very few crim­i­nal laws such that some­thing is le­gal only when a few peo­ple are do­ing it, and be­comes ille­gal if it’s too pop­u­lar. Much more com­mon to just out­law it and al­low pros­e­cu­tors/​judges lee­way in en­forc­ing it. I’d ar­gue that this choice gets ex­er­cised in ways that are harm­ful, but it does get the job (per­mit­ting low-level in­ci­dence while pre­vent­ing large-scale in­frac­tions) done.

• Dagon 17 Mar 2019 17:13 UTC

  6 points

  on: Has “poli­tics is the mind-kil­ler” been a mind-kil­ler?

  There are some groups with which I en­joy dis­cussing poli­tics, and some which I be­lieve it is effec­tive to do so, as it leads to de­ci­sions I make about where to live, what dona­tions to make, how to vote (though that’s pretty small im­pact), who to pub­li­cly sup­port, etc.

  In all cases, they’re rel­a­tively small groups, with enough face-to-face con­tact that I can es­ti­mate the lev­els of ra­tio­nal­ity and knowl­edge, and tai­lor the dis­cus­sions to what I can learn, more than how I can con­vince them of some­thing (note: many times there _is_ an ad­ver­sar­ial tone to the truth-seek­ing. I think this works well in per­son, and very badly on­line).

  My main res­o­nance with those threads is that poli­tics (at all lev­els, from fam­ily to office to town to na­tion to world) is baked into hu­man­ity, and can­not be ig­nored. And si­mul­ta­neously, the top­ics can’t be ab­stracted or gen­er­al­ized enough to ac­tu­ally dis­cuss dis­pas­sion­ately in al­most any group situ­a­tion. Poli­tics is pris­oner’s dilemma with par­ti­ci­pants (in­clud­ing my­self) whose mo­tives are un­known and in­con­sis­tent, known to be at best par­tially-ra­tio­nal. When I can play with a con­sis­tent group of iden­ti­fied par­ti­ci­pants, I can learn which mechanisms work for each. When I play si­mul­ta­neously with a large group, which statis­ti­cally will in­clude some always-defect, I fall into the defect-for-defense pat­tern.

  I do, in fact get mind-kil­led. Often by what I per­ceive as my cor­re­spon­dents’ mind-kil­ling, so in­ter­nally it feels like I’m tit-for-tat, but it’s not clear that there’s enough band­width to ever get back to a good equil­ibrium.

  So, please do put in­di­vi­d­ual thought into it, and if you have the right groups of peo­ple dis­cuss in those groups. But not here, and prob­a­bly not any­where on­line in large groups. For those places, just rec­og­nize that you’re there (as are most par­ti­ci­pants) to win peo­ple to your side, not to learn.

• Dagon 17 Mar 2019 16:48 UTC

  3 points

  on: What so­cieties have ever had le­gal or ac­cepted black­mail?

  So many (in­clud­ing Robin) are mix­ing up ar­gu­ments about black­mail (threat of re­veal­ing true in­for­ma­tion) with ar­gu­ments about non-black­mail-mo­ti­vated in­ves­ti­gat­ing, re­veal­ing, or con­ceal­ing in­for­ma­tion (gos­sip) in the ab­sence of threat and pay­ment.

  I sus­pect there are few or no ex­am­ples of so­cieties with the mix of per­sonal free­doms, non­vi­o­lent dis­pute re­s­olu­tion, and eco­nomic so­phis­ti­ca­tion (need some form of liquidity for trade) where black­mail is as sig­nifi­cant as to­day.

  In any case, hypocrisy has been around longer than his­tory—to­day’s situ­a­tion, where it (edit: black­mail) is pro­hibited but that is rarely en­forced, is likely the com­mon case.

• Dagon 16 Mar 2019 16:12 UTC

  2 points

  in reply to: TAG's comment on: Privacy

  Some things are ac­cept­able on small quan­tities but un­ac­cept­able in large ones. You don’t want to in­cen­tivise those things.

  This takes some un­pack­ing. For things that are ac­cept­able on small scales and not large ones, should we pro­hibit the scale rather than the act? The sta­tus quo is that black­mail is frowned upon, but not en­forced un­less par­tic­u­larly note­wor­thy. That bugs me from a rule-im­ple­men­ta­tion stand­point, but may be ideal in a prac­ti­cal sense.

• Dagon 15 Mar 2019 20:42 UTC

  4 points

  on: Privacy

  There’s an­other largely-un­ad­dressed el­e­ment to the de­bate: un­der­ly­ing free­doms of trans­ac­tion and of in­for­ma­tion-han­dling. All of the ar­gu­ments about black­mail are about it as an in­cen­tive for some­thing—why are we not de­bat­ing the things them­selves? Ar­gu­ments against gos­sip and in­ves­ti­ga­tion are not nec­es­sar­ily ar­gu­ments against black­mail.

  Be­fore ad­dress­ing the in­cen­tives, you should seek clar­ity/​agree­ment on what be­hav­iors you’re try­ing to en­courage and pre­vent. I still have heard very few ex­am­ples of things that are ac­cept­able with­out money in­volve­ment (in­ves­ti­gat­ing and pub­lish­ing some­one for spite or so­cial one-ups) and be­come un­ac­cept­able only be­cause of the black­mail.

• Dagon 15 Mar 2019 14:45 UTC

  2 points

  on: Mys­ter­ies, iden­tity, and prefer­ences over non-rewards

  Leav­ing aside the ques­tion of why you be­lieve that your prefer­ences don’t re­duce to he­do­nism (when con­sid­er­ing the pos­si­bil­ity of prefer­ence to iden­tify as some­one who’s prefer­ences don’t re­duce to he­do­nism)...

  One par­tial solu­tion is to rec­og­nize that I am not atomic. Parts of my mind have goals and knowl­edge that differ from other parts—it’s not a crisp sep­a­ra­tion, but it’s not a uniform be­lief-mass.

  Which opens the path to an anal­ogy to stan­dard ML prac­tice: sep­a­rat­ing your in­puts into train­ing and test sets (which are in­de­pen­dent) builds way stronger mod­els than putting all of it into train­ing, even though it’s less data in­put to the ac­tual model. I think this does give some in­sight into the prefer­ence to ini­tial ig­no­rance for games and en­ter­tain­ment/​prac­tice mys­ter­ies. I don’t think it re­solves all as­pects of the ques­tion, of course.

• Dagon 14 Mar 2019 16:54 UTC

  4 points

  on: Black­mailers are pri­va­teers in the war on hypocrisy

  I think I’d say “ar­bi­trageur” rather than “pri­va­teer”—they’re not com­bat­ants au­tho­rized to prey on an op­pos­ing state’s com­merce, they’re just notic­ing and fix­ing (by tak­ing a cut of) an in­for­ma­tion-value asym­me­try. In fact, much of the de­bate is similar to other ar­bi­trage pro­hi­bi­tions—peo­ple hate “price goug­ing”, “scalping”, “spec­u­la­tion”, and many other similar things.

  Th­ese are perfectly le­gi­t­i­mate in the­ory, but are based on un­der­ly­ing co­or­di­na­tion failures that cause bad feel­ings, and they tend to cluster with not-OK be­hav­iors (ly­ing, ar­tifi­cial ma­nipu­la­tion, in­terfer­ence with com­peti­tors, un­sanc­tioned vi­o­lence, etc.). It’s perfectly rea­son­able to look at the cluster of be­hav­iors and de­cide to pro­hibit the lot, even though it catches some things that are the­o­ret­i­cally ac­cept­able.

  The hypocrisy an­gle is in­ter­est­ing—many peo­ple seem to pre­fer that it’s “pro­hibited, but tol­er­ated at small scale”. I sus­pect we’ll face a lot of these is­sues as hu­man­ity be­comes more densely packed and visi­bly in­ter­con­nected—there are a LOT of free­doms and pri­vate choices that our in­tu­ition says should be al­lowed, but which we rec­og­nize cause mas­sive harm if scaled up. Cur­rently, they’re mostly han­dled by hypocrisy—say­ing it’s al­lowed/​dis­al­lowed, but then en­forc­ing against egre­gious cases. I won­der if there are bet­ter ways.

• Dagon 14 Mar 2019 5:43 UTC

  3 points

  in reply to: jessicata's comment on: Blegg Mode

  In many con­texts, the pri­mary benefit of the sum­mary is brevity and sim­plic­ity, more even than in­for­ma­tion. If you have more time/​band­width/​at­ten­tion, then cer­tainly in­clud­ing more in­for­ma­tion is bet­ter, and even then you should pri­ori­tize in­for­ma­tion by im­por­tance.

  In any case, I ap­pre­ci­ate the re­minder that this is the wrong fo­rum for poli­ti­cally-charged dis­cus­sions. I’m bow­ing out—I’ll read any fur­ther com­ments, but won’t re­spond.

• Dagon 14 Mar 2019 0:15 UTC

  4 points

  in reply to: Raemon's comment on: You Have About Five Words

  I think I was tak­ing “co­or­di­na­tion” in the nar­row sense of in­cent­ing peo­ple to do ac­tions to­ward a rel­a­tively straight­for­ward goal that they may or may not share. In that view, nu­ance is the en­emy of co­or­di­na­tion, and most of the work is sim­plify­ing the in­struc­tions so that it’s OK that there’s not much in­for­ma­tion trans­mit­ted. If the goal is com­mu­ni­ca­tion, rather than near-term ac­tion, you can’t avoid the ne­ces­sity of de­tail.