25Hour

• 25Hour 14 Jan 2021 19:52 UTC

  1 point

  in reply to: jbash’s comment on: Thoughts on Ia­son Gabriel’s Ar­tifi­cial In­tel­li­gence, Values, and Alignment

  I’d definitely agree with this. Human institutions are very bad at making a lot of extremely crucial decisions; the Stanislav Petrov story, the Holocaust, and the prison system are all pretty good examples of cases where the institutions humans have created have (1) been invested with a ton of power, human and technological, and (2) made really terrible decisions with that power which either could have or did cause untold suffering.

  Which I guess is mostly a longer way of saying +1.

• 25Hour 15 Jan 2021 21:44 UTC

  LW: 3 AF: 1

  AF

  in reply to: Xita’s comment on: Thoughts on Ia­son Gabriel’s Ar­tifi­cial In­tel­li­gence, Values, and Alignment

  I’d be concerned that our instincts toward vengeance in particular would result in extremely poor outcomes if you give humans near-unlimited power (which is mostly granted by being put in charge of an otherwise-sovereign AGI); one potential example is the AGI-controller sending a murderer to an artificial, semi-eternal version of Hell as punishment for his crimes. I believe there’s a Black Mirror episode exploring this. In a hypothetical AGI good outcome, this cannot occur.

  The idea of a committee of ordinary humans, ems, and semi-aligned AI which are required to agree in order to perform actions does, in principle, avoid this failure mode. Though worth pointing out that this also requires AI-alignment to be solved—if the AGI is effectively a whatever-maximizer with the requirement that it gets agreement from the ems and humans before acting, it will acquire that agreement by whatever means, which brings up the question of whether the participation of the humans and ems is at all meaningful in this scenario.

  The obvious retort is that the AGI would be a tool-ai without the desire to maximize any specific property of the world beyond the degree to which it obeys the humans and ems in the loop; the usual objections to the idea of tool-ais as an AI alignment solution apply here.

  Another possible retort, which you bring up, is that the AGI needs to (instead of maximizing a specific property of the world) understand the balance between various competing human values, which is (as I understand it) another way of saying it needs to be capable of understanding and implementing coherent extrapolated volition. Which is fine, though if you posit this then CEV simply becomes the value to maximize instead; which brings us back to the objection that if it wants to maximize a thing conditional on getting consent from a human, it will figure out a way to do that. Which turns “AI governs the world, in a fashion overseen by humans and ems” into “AI governs the world.”

Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

• 25Hour 14 Aug 2021 19:44 UTC

  2 points

  in reply to: Josh Jacobson’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  All fair points! That said, I think extended lifespan is a very reasonable thing to expect, since IIRC from longevity research that caloric restriction extends lifespan (from animal studies); this seems like a very natural extrapolation from that.

• 25Hour 14 Aug 2021 20:30 UTC

  4 points

  in reply to: ChristianKl’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  Can you give any examples of that happening, where a drug reduces lifespan but not by causing any specific fatal effect?

• 25Hour 14 Aug 2021 20:35 UTC

  2 points

  in reply to: Josh Jacobson’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  Also, good point about highlighting the uncertainty; I’ve added a disclaimer to that effect at the beginning of the section.

• 25Hour 14 Aug 2021 20:40 UTC

  5 points

  in reply to: noggin-scratcher’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  Yup! It’s branded as “Topamax”, but I’ve heard that some users refer to it as “Stupamax” because of the brain fog effect. It doesn’t sound awesome.

  Also, it sounded like it increases probability of getting a kidney stone by a lot, though I’d need to track down the reference. All told, feels like one of the worse options out there.

  As far as I understand it, “combination” drugs don’t really do anything together that each component doesn’t do alone. For example, bupropion causes weight loss if you take it alone; it just causes more when you pair it with naltrexone, which also causes weight loss.

• 25Hour 14 Aug 2021 20:49 UTC

  2 points

  in reply to: gilch’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  Oh, I’m guessing based on purely correlational studies, with all the uncertainty and fuzziness that implies. Added a disclaimer to the relevant section to this effect, since it’s worth calling out.

  That said, I’d be shocked if the whole effect was due to confounders, since there are so many negative conditions comorbid with obesity, along with the existence of some animal studies also pointing in the direction of improved lifespan with caloric restriction.

  Unfortunately, we don’t have the ability to run controlled studies over a human lifespan, so we end up needing to do correlational studies and control for what we can. It seems like a bad idea to simply throw up our hands in complete epistemic helplessness and say that we don’t know anything for sure; we need to act in the presence of incomplete information.

  Also, re: the specific point of

  Diet might fix these, while drugs might not.

  Keep in mind that these drugs cause weight loss by way of causing dietary changes.

• 25Hour 14 Aug 2021 21:15 UTC

  3 points

  in reply to: ChristianKl’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  I suppose that’s reasonable, though i will point out that this is a fully-general argument against taking any drugs long-term at all.

• 25Hour 14 Aug 2021 22:52 UTC

  2 points

  in reply to: romeostevensit’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  Oh, Wellbutrin (bupropion) is totally a thing you can use for weight loss, and is even found in Contrave (one of the drugs I listed) for that reason. Lesser effect, though, since its weight loss effects are additive with naltrexone.

  Berberine is one I hadn’t heard of before; unfortunately I can’t find any articles discussing its use in weight loss.

• 25Hour 17 Aug 2021 22:21 UTC

  3 points

  in reply to: JenniferRM’s comment on: Per­haps vastly more peo­ple should be on FDA-ap­proved weight loss medication

  This is an interesting argument! I certainly acknowledge that if you can become non-obese via purely dietary means, that is best.

  I wonder whether your analogy holds in the circumstance where dietary means have been attempted and failed, as often happens judging by the truly staggering number of posts online on this very topic—whether becoming non-obese via medication constitutes a short-term win outweighed by long-term detriments, and whether the effects of the pills turn out to be more harmful than the original obesity it was meant to treat.

  But it’s not totally clear to me that you have attempted to make an affirmative case for this being true, as opposed to suggesting it as a pure hypothetical.

• 25Hour 21 Aug 2021 4:00 UTC

  8 points

  on: Covid 8/​19: Crack­ing the Booster

  I look forward to Thursdays specifically for these updates.

• 25Hour 4 Feb 2022 21:23 UTC

  3 points

  on: Money-gen­er­at­ing en­vi­ron­ments vs. wealth-build­ing en­vi­ron­ments (or “my thoughts on the stock mar­ket”)

  I’d like to see the intuition expanded upon here:

  And yet when I write that, I start asking myself “but what is a dollar if not an investment that is only worth what someone else is willing to trade for it” and then “wait, what if a stock is a better investment than a dollar” and then “no no no no no investing on top of investing is like double risk”

  Is it double risk? We’re going from a situation where we’re talking to a widget producer and saying “yes I would like to exchange a dollar for a widget” to a situation where we’re saying “I would like to exchange a fractional share of Microsoft for a widget.” Seems basically analogous.

  Now, obviously in our society all transactions are denominated in dollars, and you have to do the conversion to dollars beforehand because no retailer is actually able to accept shares-of-stock at the counter, but the fact that purchases have to be converted to dollars beforehand doesn’t imply you’re taking on the risk of that currency increasing or decreasing in value if you don’t hold any of it at baseline.

  And I guess if you accept this, the question is what defines a “better” or “worse” investment. It sounds like you’re making an assessment that trading risk for money is fundamentally not worthwhile above a certain savings amount; I suppose that’s fair; it just means that to maintain a specific retirement withdrawal rate you have to have a bunch more money saved up pre-retirement (in expectation) than someone who doesn’t, though having done that you also face less risk of ruin from the stock market crashing.

  I’m wondering if that mindset can be trivially extended to “but actually the really foolproof asset is freeze-dried meals, since 1 meal=1 meal, as opposed to one dollar which could equal any number of fractional meals in the future”.

• 25Hour 4 Feb 2022 21:36 UTC

  3 points

  in reply to: Nicole Dieker’s comment on: Money-gen­er­at­ing en­vi­ron­ments vs. wealth-build­ing en­vi­ron­ments (or “my thoughts on the stock mar­ket”)

  I think you’re totally right that to the extent that the stock market is a zero-sum game retail traders will lose almost every time, since the big players on the other end will always have more information and power to leverage that information than retail.

  I think a lot of the relevance of this comment depends on your view of stock-market-as-casino vs stock-market-as-generator-of-wealth-at-several-steps-removed. I take the view that it’s mostly the latter; widget maker IPOs, accepts money from big institutional IPO investor and buys capital with it in exchange for proceeds, IPO investor (effectively after several intermediate trades) sells that share of proceeds-generated-from-capital to retail trader. The capital is still doing stuff for people! It’s just exactly what it’s doing is totally opaque to almost everyone.

• 25Hour 2 Mar 2022 6:12 UTC

  19 points

  on: Have You Tried Hiring Peo­ple?

  > Give Terrence Tao 500 000$ to work on AI alignement six months a year, letting him free to research crazy Navier-Stokes/​Halting problem links the rest of his time… If money really isn’t a problem, this kind of thing should be easy to do.

  Literally that idea has been proposed multiple times before that I know of, and probably many more times many years ago before I was around.

  What was the response? (I mean, obviously it was “not interested”, otherwise it would’ve happened by now, but why?)

• 25Hour 3 Mar 2022 0:43 UTC

  11 points

  in reply to: recursing’s comment on: Have You Tried Hiring Peo­ple?

  From a rando outsider’s perspective, MIRI has not made any public indication that they are funding-constrained, particularly given that their donation page says explicitly that:

  We’re not running a formal fundraiser this year but are participating in end-of-year matching events, including Giving Tuesday.

  Which more or less sounds like “we don’t need any more money but if you want to give us some that’s cool”

• 25Hour 3 Mar 2022 0:56 UTC

  23 points

  in reply to: Eliezer Yudkowsky’s comment on: Late 2021 MIRI Con­ver­sa­tions: AMA /​ Discussion

  I’m not sure whether the unspoken context of this comment is “We tried to hire Terry Tao and he declined, citing lack of interest in AI alignment” vs “we assume, based on not having been contacted by Terry Tao, that he is not interested in AI alignment.”

  If the latter: the implicit assumption seems to be that if Terry Tao would find AI alignment to be an interesting project, we should strongly expect him to both know about it and have approached MIRI regarding it, neither which seems particularly likely given the low public profile of both AI alignment in general and MIRI in particular.

  If the former: bummer.

• 25Hour 11 Mar 2022 21:36 UTC

  13 points

  in reply to: MathiasKB’s comment on: Beyond micromarriages

  but it’s such a good pun!

• 25Hour 7 Jun 2022 11:37 UTC

  7 points

  4

  on: We will be around in 30 years

  You ask elsewhere for commenters to sit down and think for 5 minutes about why an agi might fail. This seems beside the point, since averting human exctinction doesn’t require averting one possible attack from an agi. It involves averting every single one of them, because if even one succeeds everyone dies.

  In this it’s similar to human security—“why might a hacker fail” is not an interesting question to system designers, because the hacker gets as many attempts as he wants. For what attempts might look like, i think other posts have provided some reasonable guesses.

  I also note that there already exist (non-intelligent) distributed computer systems entirely beyond the ability of any motivated human individual, government or organization to shut down. I refer, of course, to cryptocurrencies, which have this property as an explicit goal of their design.

  So. Imagine that an AGI distributes itself among human computer systems in the same way as bitcoin mining software is today. Then it starts executing on someone’s list of doomsday ideas, probably in a way secretive enough to be deniable.

  Who’s gonna shut it down? And what would such an action even look like?

  (A possible suggestion is “everyone realizes their best interest is in coordinating shutting down their computers so that the AGI lacks a substrate to run on”. To which i would suggest considering the last three years’ worth of response to an obvious, threatening, global enemy that’s not even sentient and will not attempt to defend itself.)

• 25Hour 7 Jun 2022 12:05 UTC

  3 points

  1

  in reply to: mukashi’s comment on: We will be around in 30 years

  “If you think this is a simplistic or distorted version of what EY is saying, you are not paying attention. If you think that EY is merely saying that an AGI can kill a big fraction of humans in accident and so on but there will be survivors, you are not paying attention.”

  Not sure why this functions as a rebuttal to anything i’m saying.