TurnTrout

A Ker­nel of Truth: In­sights from ‘A Friendly Ap­proach to Func­tional Anal­y­sis’

[Question] Re­search on re­pur­pos­ing filter prod­ucts for masks?

• TurnTrout 1 Apr 2020 12:37 UTC

  8 points

  on: April Fools: An­nounc­ing LessWrong 3.0 – Now in VR!

  Amaz­ing call. In these scary times, it’s com­fort­ing to be re­minded just how smart the LessWrong mod team is.

• TurnTrout 1 Apr 2020 12:29 UTC

  LW: 9 AF: 5

  AF

  on: How spe­cial are hu­man brains among an­i­mal brains?

  Hu­mans seem to have much higher de­grees of con­scious­ness and agency than other an­i­mals, and this may have emerged from our ca­pac­i­ties for lan­guage. He­len Kel­ler (who was deaf and blind since in­fancy, and only started learn­ing lan­guage when she was 6) gave an au­to­bi­o­graph­i­cal ac­count of how she was driven by blind im­pe­tuses un­til she learned the mean­ings of the words “I” and “me”

  This is fas­ci­nat­ing, but there’s a bit of a po­ten­tial con­founder in that she was six years old. I’m anec­do­tally aware of sev­eral peo­ple who feel they weren’t re­ally con­scious be­fore a cer­tain age.

• TurnTrout 28 Mar 2020 13:14 UTC

  2 points

  in reply to: Pattern's comment on: TurnTrout’s short­form feed

  Don’t have much of an opinion—I haven’t rigor­ously stud­ied in­finites­i­mals yet. I usu­ally just think of in­finite /​ in­finitely small quan­tities as be­ing pro­duced by limit­ing pro­cesses. For ex­am­ple, the in­ter­sec­tion of all the $ϵ$-balls around a real num­ber is just that num­ber (un­der the stan­dard topol­ogy), which set has 0 mea­sure and is, in a sense, “in­finitely small”.

• TurnTrout 27 Mar 2020 13:30 UTC

  2 points

  on: TurnTrout’s short­form feed

  To pro­long my medicine stores by 200%, I’ve mixed in similar-look­ing iron sup­ple­ment place­bos with my real med­i­ca­tion. (To be clear, noth­ing se­ri­ous hap­pens to me if I miss days)

• TurnTrout 27 Mar 2020 13:04 UTC

  LW: 2 AF: 1

  AF

  in reply to: michaelcohen's comment on: How im­por­tant are MDPs for AGI (Safety)?

  The point of this point is mostly to claim that it’s not a hugely use­ful frame­work for think­ing about RL.

  Even though I agree it’s un­re­al­is­tic, MDPs are still eas­ier to prove things in and I still think that they can give us im­por­tant in­sights. for ex­am­ple, if I had started with more com­plex en­vi­ron­ments when I was in­ves­ti­gat­ing in­stru­men­tal con­ver­gence, I would’ve spent a ton of ex­tra time grap­pling with the the­o­rems for lit­tle per­ceived benefit. that is, the MDP frame­work let me more eas­ily cut to the core in­sights. some­times it’s worth think­ing about more gen­eral com­putable en­vi­ron­ments, but prob­a­bly not always.

• TurnTrout 26 Mar 2020 23:51 UTC

  4 points

  on: TurnTrout’s short­form feed

  It seems to me that Zeno’s para­doxes lev­er­age in­cor­rect, naïve no­tions of time and com­pu­ta­tion. We ex­ist in the world, and we might sup­pose that that the world is be­ing com­puted in some way. If time is con­tin­u­ous, then the com­puter might need to do some pretty weird things to de­ter­mine our lo­ca­tion at an in­finite num­ber of in­ter­me­di­ate times. How­ever, even if that were the case, we would never no­tice it – we ex­ist within time and we would not ob­serve the ex­ter­nal be­hav­ior of the sys­tem which is com­put­ing us, nor its run­time.

• TurnTrout 26 Mar 2020 2:33 UTC

  3 points

  in reply to: Amritam Gamaya's comment on: ODE to Joy: In­sights from ‘A First Course in Or­di­nary Differ­en­tial Equa­tions’

  Thank you for this, that’s very helpful.

• TurnTrout 25 Mar 2020 23:20 UTC

  2 points

  in reply to: Pattern's comment on: ODE to Joy: In­sights from ‘A First Course in Or­di­nary Differ­en­tial Equa­tions’

  Coun­terex­am­ple: $\sin x$ is an­a­lytic but its deriva­tives don’t satisfy your pro­posed con­di­tion for be­ing an­a­lytic.

ODE to Joy: In­sights from ‘A First Course in Or­di­nary Differ­en­tial Equa­tions’

• TurnTrout 21 Mar 2020 13:37 UTC

  LW: 4 AF: 3

  AF

  in reply to: Дмитрий Зеленский's comment on: The hu­man side of interaction

  why do we even be­lieve that hu­man val­ues are good?

  Be­cause they con­sti­tute, by defi­ni­tion, our good­ness crite­rion? It’s not like we have two sep­a­rate mod­ules—one for “hu­man val­ues”, and one for “is this good?”. (ETA or are you point­ing out how our val­ues might shift over time as we re­flect on our meta-ethics?)

  Per­haps the typ­i­cal hu­man be­havi­our am­plified by pos­si­bil­ities of a su­per-in­tel­li­gence would ac­tu­ally de­stroy the uni­verse.

  If I un­der­stand cor­rectly, this is “are hu­man be­hav­iors catas­trophic?”—not “are hu­man val­ues catas­trophic?”.

• TurnTrout 19 Mar 2020 20:24 UTC

  7 points

  on: TurnTrout’s short­form feed

  Broca’s area han­dles syn­tax, while Wer­nicke’s area han­dles the se­man­tic side of lan­guage pro­cess­ing. Sub­jects with dam­age to the lat­ter can speak in syn­tac­ti­cally fluent jar­gon-filled sen­tences (fluent apha­sia) – and they can’t even tell their ut­ter­ances don’t make sense, be­cause they can’t even make sense of the words leav­ing their own mouth!

  It seems like GPT2 : Broca’s area :: ??? : Wer­nicke’s area. Are there any cog psych/​AI the­o­ries on this?

• TurnTrout 18 Mar 2020 18:36 UTC

  LW: 4 AF: 2

  AF

  on: TurnTrout’s short­form feed

  Very rough idea

  In 2018, I started think­ing about cor­rigi­bil­ity as “be­ing the kind of agent lots of agents would be happy to have ac­ti­vated”. This seems re­ally close to a more am­bi­tious ver­sion of what AUP tries to do (not be catas­trophic for most agents).

  I won­der if you could build an agent that rewrites it­self /​ makes an agent which would tai­lor the AU land­scape to­wards its cre­ators’ in­ter­ests, un­der a wide dis­tri­bu­tion of cre­ator agent goals/​ra­tio­nal­ities/​ca­pa­bil­ities. And maybe you then get a kind of gen­er­al­iza­tion, where most sim­ple al­gorithms which solve this solve am­bi­tious AI al­ign­ment in full gen­er­al­ity.

• TurnTrout 18 Mar 2020 18:33 UTC

  LW: 4 AF: 3

  AF

  on: [AN #91]: Con­cepts, im­ple­men­ta­tions, prob­lems, and a bench­mark for im­pact measurement

  I think this is prob­a­bly go­ing to do some­thing quite differ­ent from the con­cep­tual ver­sion of AUP, be­cause im­pact (as defined in this se­quence) oc­curs only when the agent’s be­liefs change, which doesn’t hap­pen for op­ti­mal agents in de­ter­minis­tic en­vi­ron­ments. The cur­rent im­ple­men­ta­tion of AUP tries to get around this us­ing prox­ies for power (but these can be gamed) or by defin­ing “dumber” be­liefs that power is mea­sured rel­a­tive to (but this fails to lev­er­age the AI sys­tem’s un­der­stand­ing of the world).

  Although the point is more eas­ily made in the de­ter­minis­tic en­vi­ron­ments, im­pact doesn’t hap­pen in ex­pec­ta­tion for op­ti­mal agents in stochas­tic en­vi­ron­ments, ei­ther. This is by con­ser­va­tion of ex­pected AU (this is the point I was mak­ing in The Gears of Im­pact).

  Similar things can be said about power gain – when we think an agent is gain­ing power… gain­ing power com­pared to what? The agent “always had” that power, in a sense – the only thing that hap­pens is that we re­al­ize it.

  This line of ar­gu­ment makes me more pes­simistic about there be­ing a clean for­mal­iza­tion of “don’t gain power”. I do think that the for­mal­iza­tion of power is cor­rect, but I sus­pect peo­ple are do­ing some­thing heuris­tic and pos­si­bly kludgy when we think about some­one else gain­ing power.

• TurnTrout 18 Mar 2020 18:33 UTC

  LW: 2 AF: 1

  AF

  in reply to: rohinmshah's comment on: At­tain­able Utility Preser­va­tion: Scal­ing to Superhuman

  I think this is prob­a­bly go­ing to do some­thing quite differ­ent from the con­cep­tual ver­sion of AUP, be­cause im­pact (as defined in this se­quence) oc­curs only when the agent’s be­liefs change, which doesn’t hap­pen for op­ti­mal agents in de­ter­minis­tic en­vi­ron­ments. The cur­rent im­ple­men­ta­tion of AUP tries to get around this us­ing prox­ies for power (but these can be gamed) or by defin­ing “dumber” be­liefs that power is mea­sured rel­a­tive to (but this fails to lev­er­age the AI sys­tem’s un­der­stand­ing of the world).

  For the benefit of fu­ture read­ers, I replied to this in the newslet­ter’s com­ments.

• TurnTrout 15 Mar 2020 2:28 UTC

  2 points

  on: Coron­avirus Open Thread

  My room­mate tested pos­i­tive for type A flu. Does this mean he is un­likely to have COVID?

• TurnTrout 14 Mar 2020 18:48 UTC

  2 points

  in reply to: DanielFilan's comment on: Coron­avirus Open Thread

  Doesn’t di­rectly an­swer this con­cern, but: I just called the Cry­on­ics In­sti­tute, and they said that CI and Sus­pended An­i­ma­tion both plan to con­tinue offer­ing ser­vices dur­ing the pan­demic.

• TurnTrout 11 Mar 2020 19:22 UTC

  2 points

  in reply to: Filipe Marchesini's comment on: Wel­come to Less Wrong!

  Wel­come :)

• TurnTrout 9 Mar 2020 5:25 UTC

  10 points

  on: Coron­avirus Open Thread

  Should poly peo­ple con­sider stop­ping in­ti­mate con­tact (hugs+) at some point? The net­work struc­ture of polyamorous re­la­tion­ships might make peo­ple par­tic­u­larly vuln­er­a­ble.