johnswentworth

• johnswentworth 1 Jun 2020 15:00 UTC

  4 points

  in reply to: jessicata’s comment on: The Pre­sump­tu­ous Philoso­pher, self-lo­cat­ing in­for­ma­tion, and Solomonoff induction

  Prob­lem is, there isn’t nec­es­sar­ily a mod­u­lar pro­ce­dure used to iden­tify your­self. It may just be some sort of hard-coded in­dex. A Solomonoff in­duc­tor will rea­son over all pos­si­ble such in­dices by rea­son­ing over all pro­grams, and throw out any which turn out to not be con­sis­tent with the data. But that be­hav­ior is pack­aged with the in­duc­tor, which is not it­self a pro­gram.

• johnswentworth 1 Jun 2020 13:59 UTC

  2 points

  in reply to: jessicata’s comment on: The Pre­sump­tu­ous Philoso­pher, self-lo­cat­ing in­for­ma­tion, and Solomonoff induction

  I’m about 80% on board with that ar­gu­ment.

  The main loop­hole I see is that num­ber-of-em­bed­ded-agents may not be de­cid­able. That would make a lot of sense, since em­bed­ded-agent-de­tec­tors are ex­actly the sort of thing which would help cir­cum­vent di­ag­o­nal­iza­tion bar­ri­ers. That does run into the sec­ond part of your ar­gu­ment, but no­tice that there’s no rea­son we need to de­tect all the agents us­ing a sin­gle pro­gram in or­der for the main prob­lem setup to work. They can be ad­dressed one-by-one, by ad-hoc pro­grams, each en­cod­ing one of the hy­pothe­ses (world model, agent lo­ca­tion).

  (Per­son­ally, though, I don’t ex­pect num­ber-of-em­bed­ded-agents to be un­de­cid­able, at least for en­vi­ron­ments with some kind of pri­vate ran­dom bit sources.)

• johnswentworth 1 Jun 2020 2:36 UTC

  6 points

  in reply to: jessicata’s comment on: The Pre­sump­tu­ous Philoso­pher, self-lo­cat­ing in­for­ma­tion, and Solomonoff induction

  Assert­ing that there are n peo­ple takes at least K(n) bits, so large uni­verse sizes have to get less likely at some point.

  The prob­lem setup doesn’t nec­es­sar­ily re­quire as­sert­ing the ex­is­tence of n peo­ple. It just re­quires set­ting up a uni­verse in which n peo­ple hap­pen to ex­ist. That could take con­sid­er­ably less than K(n) bits, if per­son-de­tec­tion is it­self fairly ex­pen­sive. We could even in­dex di­rectly to the Solomonoff in­duc­tor’s in­put data with­out at­tempt­ing to rec­og­nize any agents; that would cir­cum­vent the K(num­ber of peo­ple) is­sue.

• johnswentworth 1 Jun 2020 2:31 UTC

  12 points

  on: The Pre­sump­tu­ous Philoso­pher, self-lo­cat­ing in­for­ma­tion, and Solomonoff induction

  “Okay, so you’re say­ing the ac­tual hy­pothe­ses that pre­dict my ob­ser­va­tions, which I should as­sign prob­a­bil­ity to ac­cord­ing to their com­plex­ity, are things like ‘T1 and I’m per­son #1’ or ‘T2 and I’m per­son #10^10’?” says the Solomonoff in­duc­tor.

  “Ex­actly.”

  “But I’m still con­fused. Be­cause it still re­quires in­for­ma­tion to say that I’m per­son #1 or per­son #10^10. Even if we as­sume that it’s equally easy to spec­ify where a per­son is in both the­o­ries, it just plain old takes more bits to say 10^10 than it does to say 1.”

  I think this sec­tion is con­fused about how the ques­tion “T1 or T2” gets en­coded for a Solomonoff in­duc­tor.

  Given the first chunk in the quote above, we don’t have two world mod­els; we have one world model for each per­son in T1, plus one world model for each per­son in T2. Our mod­els are (T1 & per­son 1), (T1 & per­son 2), …, (T2 & per­son 1), …. To de­cide whether we’re in T1 or T2, our Solomonoff in­duc­tor will com­pare the to­tal prob­a­bil­ity of all the T1 hy­pothe­ses to the to­tal prob­a­bil­ity of all the T2 hy­pothe­ses.

  As­sum­ing T1 and T2 have ex­actly the same com­plex­ity, then pre­sum­ably (T1 & per­son N) should have roughly the same com­plex­ity as (T2 & per­son N). That is not nec­es­sar­ily the case; T1/​T2 may con­tain in­for­ma­tion which makes en­cod­ing some num­bers cheaper/​more ex­pen­sive. But it does seem like a rea­son­able ap­prox­i­ma­tion for build­ing in­tu­ition.

  Any­way, point is, “it just plain old takes more bits to say 10^10 than it does to say 1” isn’t rele­vant here. There’s no par­tic­u­lar rea­son to com­pare the two hy­pothe­ses (T1 & per­son 1) vs (T2 & per­son 10^10); that is not the cor­rect for­mal­iza­tion of the T1 vs T2 ques­tion.

• johnswentworth 28 May 2020 21:24 UTC

  2 points

  in reply to: Daniel Kokotajlo’s comment on: Spec­u­la­tions on the Fu­ture of Fic­tion Writing

  I was un­der the im­pres­sion that movie pro­duc­ers DO hire ex­perts for this sort of thing. At the very least, I know they hire sci­ence con­sul­tants for sci­en­tific ac­cu­racy prob­lems; I as­sume they of­ten do the same for his­tor­i­cal ac­cu­racy.

Spec­u­la­tions on the Fu­ture of Fic­tion Writing

• johnswentworth 26 May 2020 23:12 UTC

  2 points

  in reply to: George’s comment on: Your ab­strac­tion isn’t wrong, it’s just re­ally bad

  Let me try an­other ex­pla­na­tion.

  The main point is: given a sys­tem, we don’t ac­tu­ally have that many de­grees of free­dom in what ab­strac­tions to use in or­der to rea­son about the sys­tem. That’s a core com­po­nent of my re­search: the un­der­ly­ing struc­ture of a sys­tem forces cer­tain ab­strac­tion-choices; choos­ing other ab­strac­tions would force us to carry around lots of ex­tra data.

  How­ever, if we have the op­por­tu­nity to de­sign a sys­tem, then we can choose what ab­strac­tion we want and then choose the sys­tem struc­ture to match that ab­strac­tion. The num­ber of de­grees of free­dom ex­pands dra­mat­i­cally.

  In pro­gram­ming, we get to de­sign very large chunks of the sys­tem; in math and the sci­ences, less so. It’s not a hard di­vid­ing line—there are de­sign prob­lems in the sci­ences and there are prob­lem con­straints in pro­gram­ming—but it’s still a ma­jor differ­ence.

  In gen­eral, we should ex­pect that look­ing for bet­ter ab­strac­tions is much more rele­vant to de­sign prob­lems, sim­ply be­cause the pos­si­bil­ity space is so much larger. For prob­lems where the sys­tem struc­ture is given, the struc­ture it­self dic­tates the ab­strac­tion choice. Peo­ple do still screw up and pick “wrong” ab­strac­tions for a given sys­tem, but since the space of choices is rel­a­tively small, it takes a lot less ex­plo­ra­tion to con­verge to pretty good choices over time.

• johnswentworth 26 May 2020 22:03 UTC

  10 points

  on: Your ab­strac­tion isn’t wrong, it’s just re­ally bad

  There is a ma­jor differ­ence be­tween pro­gram­ming and math/​sci­ence with re­spect to ab­strac­tion: in pro­gram­ming, we don’t just get to choose the ab­strac­tion, we get to de­sign the sys­tem to match that ab­strac­tion. In math and the sci­ences, we don’t get to choose the struc­ture of the un­der­ly­ing sys­tem; the only choice we have is in how to model it.

  Given a fun­da­men­tal differ­ence that large, we should ex­pect that many in­tu­itions about ab­strac­tion-qual­ity in pro­gram­ming will not gen­er­al­ize to math and the sci­ences, and I think that is the case for the core ar­gu­ment of this post.

  The main is­sue is that re­al­ity has struc­ture (es­pe­cially causal struc­ture), and we don’t get to choose that struc­ture. In pro­gram­ming, ab­strac­tion is a so­cial con­ve­nience to a much greater ex­tent; we can de­sign the sys­tems to match the cho­sen ab­strac­tions. But if we choose a poor ab­strac­tion in e.g. physics or biol­ogy, we will find that we need to carry around tons of data in or­der to make ac­cu­rate pre­dic­tions. For in­stance, the ab­strac­tion of a “cell” in biol­ogy is use­ful mainly be­cause the in­side of the cell is largely iso­lated from the out­side; in­ter­ac­tion be­tween the two takes place only through a rel­a­tively small num­ber of defined chem­i­cal/​phys­i­cal chan­nels. It’s like a phys­i­cal em­bod­i­ment of func­tion scope; we can make pre­dic­tions about out­side-the-cell be­hav­ior with­out hav­ing to track all the de­tails of what hap­pens in­side the cell.

  To draw a proper anal­ogy be­tween ab­strac­tion-choice in biol­ogy and pro­gram­ming: imag­ine that you were perform­ing re­verse com­pila­tion. You take in as­sem­bly code, and at­tempt to provide equiv­a­lent, max­i­mally-hu­man-read­able code in some other lan­guage. That’s ba­si­cally the right anal­ogy for ab­strac­tion-choice in biol­ogy.

  Pic­ture that, and hope­fully it’s clear that there are far fewer de­grees of free­dom in the choice of ab­strac­tion, com­pared to nor­mal pro­gram­ming prob­lems. That’s why peo­ple in math/​sci­ence don’t ex­per­i­ment with al­ter­na­tive ab­strac­tions very of­ten com­pared to pro­gram­ming: there just aren’t that many op­tions which make any sense at all. That’s not to say that progress isn’t made from time to time; Feyn­man’s for­mu­la­tion of quan­tum me­chan­ics was a big step for­ward. But there’s not a whole con­tinuum of similarly-de­cent for­mu­la­tions of quan­tum me­chan­ics like there is a con­tinuum of similarly-de­cent pro­gram­ming lan­guages; the ab­strac­tion choice is much more con­strained.

• johnswentworth 26 May 2020 0:53 UTC

  9 points

  in reply to: Raemon’s comment on: Bak­ing is Not a Ritual

  I un­der­stood “rit­ual” here as not just a black­box pro­cess, but a black­box pro­cess which has un­der­gone cul­tural se­lec­tion—i.e. metic knowl­edge. If we “treat bak­ing as a rit­ual” in that sense, it would mean care­fully fol­low­ing some pro­ce­dure ac­quired from some­one else, on the as­sump­tion that some parts are re­ally im­por­tant and we don’t have a good way to tell which.

• johnswentworth 25 May 2020 16:12 UTC

  4 points

  in reply to: Purplehermann’s comment on: Homeosta­sis and “Root Causes” in Aging

  Telomere short­en­ing is an in­ter­est­ing case. (I’m go­ing to give my cur­rent un­der­stand­ing here with­out try­ing to dig up refer­ences, so take it all with a grain of salt.)

  It’s clearly a plau­si­ble root cause—it’s a change which could stick around on long enough timescales to ac­count for ag­ing. On the other hand, it is pos­si­ble for telomeres to turn over: telomerase is ac­tive in stem cells, so telomere length should at least not be an is­sue for cell types which reg­u­larly turn over—the telomeres turn over with the cells, which are ul­ti­mately re­placed from the stem cells. For long-lived cells, there’s a stronger case that telomere short­en­ing could be an is­sue.

  Telomeres do get shorter with age, BUT they get shorter even in cell types which turn over reg­u­larly. That’s a bit of a red flag—ei­ther the telomeres aren’t be­ing fully re­placed by telomerase in the stem cells (in which case the stem cells ought to die a lot sooner), or some other mechanism is mak­ing them short be­sides ac­cu­mu­lated loss over life­time. The al­ter­na­tive mechanism which jumps out to me is: DNA dam­age, and ox­ida­tive dam­age in par­tic­u­lar, has been ob­served to rapidly shorten telomeres. DNA dam­age and ox­ida­tive dam­age rates are gen­er­ally ob­served to be much higher in aged cells of most types, so that would ex­plain why telomeres are shorter in older or­ganisms.

  In terms of ac­tual ex­per­i­ments, telomerase-boost­ers have been ex­per­i­mented with a fair bit, and my un­der­stand­ing is that they don’t have much effect on age-re­lated dis­eases (though of course there’s the usual pile of low-N stud­ies which find barely-sig­nifi­cant and blatantly p-hacked re­sults).

  Other things will even­tu­ally be cov­ered later in this se­quence.

• johnswentworth 24 May 2020 16:39 UTC

  3 points

  in reply to: willbradshaw’s comment on: High­lights of Com­par­a­tive and Evolu­tion­ary Aging

  Thanks, that’s the right ques­tion to ask and some great info on it.

• johnswentworth 24 May 2020 16:37 UTC

  3 points

  in reply to: willbradshaw’s comment on: High­lights of Com­par­a­tive and Evolu­tion­ary Aging

  Yeah, to be clear, semipoli­ti­cal fluff is of­ten valuable, and I agree that that’s likely the case here. But I don’t ex­pect LWers to find any­thing new or in­ter­est­ing in that part of the book, nor is any­thing in­ter­est­ing there about how ag­ing works. It’s for a differ­ent au­di­ence and a differ­ent pur­pose.

• johnswentworth 23 May 2020 21:20 UTC

  7 points

  in reply to: Purplehermann’s comment on: High­lights of Com­par­a­tive and Evolu­tion­ary Aging

  Fol­low­ing the cita­tion on wikipe­dia, sounds like that’s the longest one has lived in cap­tivity. Re­mem­ber, it’s not that they’re im­mor­tal, it’s just that their chance-of-dy­ing-per-unit-time stays flat; that still im­plies that the num­ber of sur­vivors drops off ex­po­nen­tially over time.

• johnswentworth 23 May 2020 16:46 UTC

  6 points

  in reply to: jmh’s comment on: High­lights of Com­par­a­tive and Evolu­tion­ary Aging

  Yup, read it a few months ago. Mini-re­view:

  • The core mechanisms they talk about make sense, in par­tic­u­lar trans­posons as root cause, and the pic­ture of stres­sors com­pet­ing for sir­tuin ac­tivity.

  • The re­view of ag­ing in yeast was a high­light. That’s a great ex­am­ple where the mechanisms were pretty clearly nailed down, and Sin­clair was one of the peo­ple who figured it out.

  • I’m much more skep­ti­cal of the par­tic­u­lar treat­ments dis­cussed, and es­pe­cially the pro­posal that NAD boost­ers ame­lio­rate age-re­lated dis­eases by boost­ing sir­tuin ac­tivity speci­fi­cally. NAD is a fairly gen­eral-pur­pose molecule, there’s a ton of other things it could be do­ing, and I don’t re­call a demon­stra­tion that sir­tuin ac­tivity me­di­ates their effects. (If there is a me­di­a­tion ex­per­i­ment, then that part is much stronger.)

  • Sin­clair in gen­eral seems to fit the “great ex­per­i­men­tal­ist, mediocre the­o­rist” mold; the blab­ber about in­for­ma­tion loss and ag­ing be­ing epi­ge­netic was thor­oughly con­fused. He has a bad case of ob­ses­sion-with-his-fa­vorite-gene-class (namely the sir­tu­ins). In this case that gene class seems to be ad­ja­cent to an ac­tual root cause (trans­posons), but Sin­clair him­self doesn’t seem to have the con­cep­tual frame­work in place to or­ga­nize that knowl­edge.

  • The en­tire sec­ond half of the book is semipoli­ti­cal fluff.

• johnswentworth 23 May 2020 2:00 UTC

  4 points

  on: Post-crash mar­ket effi­ciency 1696-2020

  Where did you get the data for this? I’m par­tic­u­larly im­pressed with the older price se­ries, I imag­ine that must have come from spe­cial­ized sources...

• johnswentworth 22 May 2020 17:04 UTC

  2 points

  in reply to: Roko’s comment on: Pro­ject Pro­posal: Gears of Aging

  See High­lights of Com­par­a­tive and Evolu­tion­ary Aging.

High­lights of Com­par­a­tive and Evolu­tion­ary Aging

• johnswentworth 21 May 2020 16:21 UTC

  2 points

  in reply to: Florian Dietz’s comment on: Point­ing to a Flower

  I’d dis­t­in­guish be­tween “there isn’t an ob­jec­tively cor­rect an­swer about what the flower is” and “there isn’t an ob­jec­tively cor­rect al­gorithm to an­swer the ques­tion”. There are cases where the OP method’s an­swer isn’t unique, and ex­am­ples of this so far (in the other com­ments) mostly match cases where hu­man in­tu­ition breaks down—i.e. the things hu­mans con­sider edge cases are also things the al­gorithm con­sid­ers edge cases. So it can still be the cor­rect al­gorithm, even though in some cases the “cor­rect” an­swer is am­bigu­ous.

• johnswentworth 20 May 2020 20:29 UTC

  9 points

  on: A Prob­lem With Patternism

  Let’s start by set­ting aside the whole mind-up­load­ing prob­lem, and look at some­thing more pro­saic: what makes “me” at noon to­day the same as “me” at 3:00 am one year ago? In fact, let’s set aside what makes this true from my per­spec­tive; what makes you think these two bags of chem­i­cals are the “same per­son”? When you see your mother, and then see her again later on, why do you think of her as the same per­son?

  This is ba­si­cally the same prob­lem as Point­ing to a Flower, ex­cept we’ve dragged in a bunch of new in­tu­itions by mak­ing it about hu­mans in­stead of flow­ers. (It’s also the same ques­tion Yud­kowsky uses in his post on cry­on­ics in the se­quences, al­though I can’t find a link at the mo­ment.)

  The an­swer from the flower post does a fine job of say­ing what makes me the same or­ganism as be­fore: draw a bound­ary around my body in space­time, it’s a lo­cal min­i­mum in terms of sum­mary data re­quired to make pre­dic­tions far away. Non­triv­ial, but quan­tifi­able.

  But for things like mind up­load­ing, we want to go fur­ther than that. Sure, we could simu­late my en­tire body, but it seems like what makes “me” doesn’t re­quire all that. After all, I’m still me if I lose all my limbs and torso and get at­tached to some sci-fi life sup­port ma­chine. “Me” ap­par­ently does not just mean my body. In prac­tice, I think hu­mans use “me”, “you”, etc with sev­eral differ­ent refer­ents de­pend­ing on con­text, and the body is one of them. The refer­ent rele­vant for mind-up­load­ing pur­poses is, pre­sum­ably, the mind—what­ever that means.

  There’s a few more steps be­fore I’m ready to tackle the refer­ent of “my mind”, but I’m pretty con­fi­dent that the first step is ba­si­cally the same as for the flower, and I’m also pretty con­fi­dent that it is crisply quan­tifi­able.

  As to the con­nec­tion with pat­tern match­ing, I’m pretty sure that the flower-ap­proach is roughly equiv­a­lent to what a Bayesian learner would learn by look­ing for pat­terns in data. But that’s a post for an­other time.

  What links here?

  • Bob Jacobs's comment on A Prob­lem With Patternism by Bob Jacobs (20 May 2020 16:45 UTC; 1 point)

• johnswentworth 20 May 2020 18:18 UTC

  4 points

  on: How to learn from a stronger ra­tio­nal­ist in daily life?

  So­cratic Duck­ing fits the crite­ria; it’s a good way to get a feel for what kinds of ques­tions some­one else asks their­self.