abramdemski(Abram Demski)

• abramdemski 10 Jul 2020 22:28 UTC

  2 points

  in reply to: Pongo’s comment on: What does it mean to ap­ply de­ci­sion the­ory?

  Yep, that’s a good way to ex­plain it!

• abramdemski 10 Jul 2020 17:13 UTC

  2 points

  on: Clas­sify­ing games like the Pri­soner’s Dilemma

  (Un­der “farmer’s dilemma”:)

  If 2W>X+Y then the so­cially op­ti­mal out­come is Work/​Work, and a means to co­or­di­nate on that out­come would be so­cially use­ful. If 2W<X+Y, the so­cially op­ti­mal out­come is for one player to Work while the other Shirks, but with no ob­vi­ous choice for which one of you it should be.

  (Un­der PD /​ too many cooks:)

  (I’m a lit­tle sur­prised that this is the only case where I’ve wanted to re­name the game de­pend­ing on the so­cial prefer­ence of the out­comes. That said, the only other games where X+Y isn’t forced to be greater or less than 2X are the Farmer’s Dilemma and the Abun­dant Com­mons, and those are the ones I’d most ex­pect to want to split in fu­ture.)

  To my per­sonal taste, “chicken” is the game where 2W<X+Y; I think of chicken as fun­da­men­tally be­ing a game where there isn’t a fair and so­cially op­ti­mal out­come w/​o cor­re­lated ran­dom­ness (ie in cor­re­lated equil­ibria, we have the “stop light” solu­tion where one player gets a sig­nal to go straight, with fair odds of who gets to go). It be­comes “hawk/​dove” when 2W$\ge$X+Y, rep­re­sent­ing the idea that fight­ing can’t in­crease the amount of re­sources (at best the hawk strat­egy doesn’t de­stroy any­thing un­less there’s an­other hawk; re­al­is­ti­cally it burns some re­sources ei­ther way, just a lot more when there are two hawks). How­ever, I re­al­ize this isn’t stan­dard—it seems like a lot of peo­ple con­sider both games to be “chicken”.

  (By the way, I was sur­prised you la­bel­led the sec­tion “farmer’s dilemma”—just from my ex­pe­rience, “chicken” is far more com­mon, and “hawk/​dove” is also quite com­mon, but not as com­mon as “chicken”; whereas I’d never ac­tu­ally heard “farmer’s dilemma” be­fore.)

  [I also want to briefly point out that it’s ques­tion­able to as­sume that the util­ity func­tions of the two play­ers are com­pa­rable, and hence, that so­cial welfare is nec­es­sar­ily cap­tured by the sum of the two pay­offs. Utility func­tions are always “up to a scalar”, so re­ally “sym­met­ric game” should mean some­thing more com­pli­cated … but all that be­ing said, I think you han­dled this the right way in the end, be­cause we don’t want to deal with that ex­tra com­plex­ity in a clas­sifi­ca­tion like this.]

• abramdemski 9 Jul 2020 20:58 UTC

  4 points

  in reply to: jimrandomh’s comment on: The Core Tags [Beta]

  What’s the idea be­hind “mo­ti­va­tions”? I don’t un­der­stand what your pro­posal is.

  Edit: Ah, hadn’t read Habryka’s com­ment yet :p

• abramdemski 9 Jul 2020 20:49 UTC

  2 points

  in reply to: abramdemski’s comment on: The Core Tags [Beta]

  Look­ing at things a bit more, maybe prac­ti­cal vs ra­tio­nal­ity is sup­posed to cover what I want in­stru­men­tal ra­tio­nal­ity vs epistemic ra­tio­nal­ity to cover? But if so I don’t find those de­scrip­tions very in­tu­itive and don’t ex­pect peo­ple to ap­ply them cor­rectly.

• abramdemski 9 Jul 2020 19:48 UTC

  2 points

  in reply to: habryka’s comment on: The New Front­page De­sign & Open­ing Tag Creation!

  Thanks!

• abramdemski 9 Jul 2020 19:02 UTC

  2 points

  on: The Core Tags [Beta]

  I cur­rently would like tags differ­en­ti­at­ing epistemic ra­tio­nal­ity from in­stru­men­tal ra­tio­nal­ity. (I know I can just go ahead and cre­ate them, now, but the over­lap with ex­ist­ing cat­e­gories is suffi­cient that I’d rather talk it out a bit. The main rea­son I want these tags to ex­ist is ac­tu­ally for use with the fil­ter­ing fea­ture, so I want there to be enough con­sen­sus for oth­ers to use them!) For ex­am­ple, my re­cent post about bet­ting is clearly epistemic ra­tio­nal­ity. But with­out cre­at­ing a tag, rn the clos­est I can come is to la­bel it “episte­mol­ogy”—that doesn’t seem right, yeah? It isn’t the­o­riz­ing about epistemics—it’s just a prac­ti­cal thingy about im­prov­ing one’s epistemics.

  World Model­ing /​ World Op­ti­miza­tion seem like a nod to epistemic/​in­stru­men­tal clusters, but again, don’t ac­tu­ally seem to be the thing I want to tag with—World Model­ing sounds like ob­ject-level world mod­el­ing, not prac­ti­cal but meta-level ad­vice about how to world-model bet­ter.

  Thoughts?

• abramdemski 9 Jul 2020 18:45 UTC

  2 points

  on: The New Front­page De­sign & Open­ing Tag Creation!

  I cre­ated a tag, and added a de­scrip­tion, but now I want to edit that de­scrip­tion. How do I do that?

• abramdemski 9 Jul 2020 17:07 UTC

  LW: 2 AF: 2

  AF

  in reply to: gwern’s comment on: How “hon­est” is GPT-3?

  Thanks for run­ning the test, much ap­pre­ci­ated! (Also, hilar­i­ous.)

• abramdemski 9 Jul 2020 17:06 UTC

  12 points

  on: The New Front­page De­sign & Open­ing Tag Creation!

  run­ning some ex­per­i­ments to make the new de­sign feel a bit less droopy (while the bright white back­ground gave me a bit of eye-strain that I am glad to be rid off in this new de­sign, the grey back­ground does feel a bit de­press­ing and I’ve been ex­per­i­ment­ing with whether we can im­prove that some­how by maybe us­ing more images or gra­di­ents or some kind of tex­ture, but I don’t know yet where that will go.

  Have you con­sid­ered a yel­low­ish or sepia tone? Like, cream-col­ored. I men­tally as­so­ci­ate that kind of slight off-white with high qual­ity writ­ing pa­per, so it feels ap­pro­pri­ate to me. (For the ex­act same rea­son; re­duc­ing eye strain while writ­ing.)

• abramdemski 8 Jul 2020 21:10 UTC

  2 points

  in reply to: George’s comment on: Have gen­eral de­com­posers been for­mal­ized?

  I think Quinn means fac­tored cog­ni­tion, which is quite differ­ent from au­toen­coders/​em­bed­dings/​PCA.

• abramdemski 8 Jul 2020 21:06 UTC

  2 points

  in reply to: Donald Hobson’s comment on: on wellunderstoodness

  The idea (as I un­der­stood it from the post) isn’t to max­i­mize P(goal) con­strained by P(goal)<1-$ϵ$. The idea is to se­lect ac­tions in a way which max­i­mizes P(goal|ac­tion), un­less mul­ti­ple ac­tions achieve P(goal|ac­tion)>1-$ϵ$, in which case you choose be­tween those ran­domly. Or the same but for poli­cies rather than ac­tions. Or the same but for plans rather than ac­tions. Some­thing in that gen­eral area.

  One gen­eral con­cern is tiling—it seems very pos­si­ble that such a sys­tem would build a suc­ces­sor-agent with the limi­ta­tions taken off, since this is an easy way to achieve its ob­jec­tive, es­pe­cially in an en­vi­ron­ment in which one AGI has (ap­par­ently) already been con­structed.

  Another prob­lem with this solu­tion to AI safety is that we can’t ap­ply it to, say, a big neu­ral net­work. The NN may in­ter­nally perform search-based plan­ning to achieve high perfor­mance on a task, but we don’t have ac­cess to that, to pre­vent it from “try­ing too hard”.

  We could, of course, ap­ply it to NN train­ing, mak­ing sure to stop our train­ing be­fore the NN be­comes so ca­pa­ble that we have safety con­cerns. But the big prob­lem there is know­ing where to stop. If you train the NN enough to be highly ca­pa­ble, it may already be im­ple­ment­ing the kind of in­ter­nal search that we would be con­cerned about. But this sug­gests your stop­ping point has to be be­fore NNs could de­velop in­ter­nal search. Un­for­tu­nately we’ve already seen NNs learn to search (granted, with some prod­ding to do so—I be­lieve there have been other ex­am­ples, but I didn’t find what I re­mem­bered). So this con­straint has already been vi­o­lated, and it would pre­sum­ably be re­ally difficult to con­vince peo­ple to reel things back in.

• abramdemski 8 Jul 2020 20:33 UTC

  6 points

  in reply to: D0TheMath’s comment on: How “hon­est” is GPT-3?

  Yeah, but similar puns aren’t made by peo­ple—GPT-3 is that bad at puns. So I won­der how well it would ex­plain puns that are made by peo­ple (ie, ac­tual puns).

What does it mean to ap­ply de­ci­sion the­ory?

[Question] How “hon­est” is GPT-3?

• abramdemski 3 Jul 2020 16:35 UTC

  2 points

  in reply to: jimmy’s comment on: Noise on the Channel

  I think it’s worth mak­ing a dis­tinc­tion be­tween “noise” and “low band­width chan­nel”. Your first ex­am­ples of “a literal noisy room” or “peo­ple get­ting dis­tracted by shiny ob­jects pass­ing by” fit the idea of “noise” well. Your last two ex­am­ples of “in­fer­en­tial dis­tance” and “land mines” don’t, IMO.

  “Noise” is when the use­ful in­for­ma­tion is get­ting crowded out by ran­dom in­for­ma­tion in the chan­nel, but land mines aren’t ran­dom. If you tell some­one their idea is stupid and then you can’t con­tinue tel­ling them why be­cause they’re flip­ping out at you, that’s not a ran­dom oc­cur­rence. Even if such things aren’t triv­ially pre­dictable in more sub­tle cases, it’s still a pre­dictable pos­si­bil­ity and you can gen­er­ally feel out when such things are safe to say or when you must tread a bit more care­fully.

  I ed­ited my post to in­sert this dis­tinc­tion. You’re to­tally right that I’m re­ally fo­cus­ing on band­width and call­ing it low-noise. But I dis­agree about the de­gree of the dis­tinc­tion you’re mak­ing. In the case of the already-stan­dard us­age of “sig­nal/​noise ra­tio”, there’s no worry over whether the “noise” is re­ally ran­dom. Twit­ter in­jects ad­ver­tise­ments reg­u­larly, not ran­domly, but they still dilute the qual­ity of the feed in the same way. Similarly, con­ver­sa­tional land mines are func­tion­ally similar to dis­trac­tions. First, be­cause they tend to de­rail lines of thought. But sec­ond, and more fre­quently, in the way they in­fluence con­ver­sa­tion when they’re merely a threat loom­ing on the bor­der of the con­ver­sa­tion rather than a cer­tainty. We avoid deep top­ics both be­cause they’re more likely to trig­ger defen­sive­ness and be­cause they aren’t so valuable (and in­deed may even be harm­ful) if they’re in­ter­rupted. In­deed, I’m clus­ter­ing them to­gether be­cause the two are some­what ex­change­able: a touchy sub­ject can be­come quite ap­proach­able if you have a lot of qual­ity time to feel it out and deal with mi­s­un­der­stand­ings/​defen­sive­ness (or any of the other helpful vari­ables I men­tioned).

• abramdemski 2 Jul 2020 18:54 UTC

  4 points

  in reply to: MarcelloV’s comment on: Bet­ting with Manda­tory Post-Mortem

  Yeah, pre-mortem is an­other name for pre-hind­sight, and mur­phyjitsu is just the idea of al­ter­nat­ing be­tween mak­ing pre-mortems and fix­ing your plans to pre­vent what­ever prob­lem you en­vi­sioned in the pre-mortem.

• abramdemski 2 Jul 2020 15:59 UTC

  2 points

  in reply to: TurnTrout’s comment on: Noise on the Channel

  This seems to also ap­ply to other cases where some amount of dis­trac­tion ac­tu­ally benefits the con­ver­sa­tion.

Noise on the Channel

• abramdemski 1 Jul 2020 17:05 UTC

  7 points

  in reply to: jollybard’s comment on: The Parable of Pre­dict-O-Matic

  I agree that it’s broadly rele­vant to the par­tial-agency se­quence, but I’m cu­ri­ous what par­tic­u­lar rem­i­nis­cence you’re see­ing here.

  I would say that “if the free en­ergy prin­ci­ple is a good way of look­ing at things” then it is the solu­tion to at least one of the rid­dles I’m think­ing about here. How­ever, I haven’t so far been very con­vinced.

  I haven’t looked into the tech­ni­cal de­tails of Fris­ton’s work very much my­self. How­ever, a very math­e­mat­i­cally so­phis­ti­cated friend has tried go­ing through Fris­ton pa­pers on a cou­ple of oc­ca­sions and found them rid­dled with math­e­mat­i­cal er­rors. This does not, of course, mean that ev­ery ver­sion of free-en­ergy/​pre­dic­tive-pro­cess­ing ideas is wrong, but it does make me hes­i­tant to take pur­ported re­sults at face value.

• abramdemski 1 Jul 2020 16:35 UTC

  4 points

  in reply to: Davidmanheim’s comment on: Bet­ting with Manda­tory Post-Mortem

  Sorta, but you might have 50:50 odds with a very large spread (both peo­ple are very con­fi­dent in their side) or with a very small spread. So it might be helpful to record that.