MichaelDickens

• MichaelDickens 15 Nov 2025 6:08 UTC

  4 points

  2

  in reply to: habryka’s comment on: Gra­di­en­tDissen­ter’s Shortform

  Yeah I pretty much agree with what you’re saying. But I think I misunderstood your comment before mine, and the thing you’re talking about was not captured by the model I wrote in my last comment; so I have some more thinking to do.

  I didn’t mean “can be trusted to take AI risk seriously” as “indeterminate trustworthiness but cares about x-risk”, more like “the conjunction of trustworthy + cares about x-risk”.

• MichaelDickens 15 Nov 2025 5:32 UTC

  4 points

  0

  in reply to: habryka’s comment on: Gra­di­en­tDissen­ter’s Shortform

  Like, the most important thing to estimate when evaluating a political candidate is their trustworthiness and integrity! It’s the thing that would flip the sign on whether supporting someone is good or bad for the world.

  I agree that this is an important thing that deserved more consideration in Eric’s analysis (I wrote a note about it on Oct 22 but then I forgot to include it in my post yesterday). But I don’t think it’s too hard to put into a model (although it’s hard to find the right numbers to use). The model I wrote down in my note is

  • 30% chance Bores would oppose an AI pause /​ strong AI regulations (b/​c it’s too “anti-innovation” or something)

  • 40% chance Bores would support strong regulations

  • 30% chance he would vote for strong regulations but not advocate for them

  • 90% chance Bores would support weak/​moderate AI regulations

  My guess is that ²⁄₃ of the EV comes from strong regulations and ¹⁄₃ from weak regulations (which I just came up with a justification for earlier today but it’s too complicated to fit in this comment), so these considerations reduce the EV to 37% (i.e., roughly divide EV by 3).

  FWIW I wouldn’t say “trustworthiness” is the most important thing, more like “can be trusted to take AI risk seriously”, and my model is more about the latter. (A trustworthy politician who is honest about the fact that they don’t care about AI safety will not be getting any donations from me.)

• MichaelDickens 14 Nov 2025 16:15 UTC

  13 points

  8

  in reply to: Mo Putera’s comment on: Mo Putera’s Shortform

  An important part of my model of college admissions—which unfortunately I didn’t acquire until after I was done applying for colleges—is to consider what type of person becomes a college admissions officer. What percentage of admissions officers majored in math? (Is it possibly as high as 1%? I doubt it.) What percentage of admissions officers understand the significance of something like “solved the Mizohata-Takeuchi conjecture”? What percentage have a vague (or even explicit) disdain for anything math-flavored?

  On my model, it is not surprising that admissions officers would fail to appreciate a math prodigy.

  Administrators overriding an acceptance does seem like a remarkable failure. I can’t say I’m surprised, but it’s a much worse indictment of those universities, I think.

Epistemic Spot Check: Ex­pected Value of Donat­ing to Alex Bores’s Con­gres­sional Campaign

• MichaelDickens 10 Nov 2025 19:24 UTC

  2 points

  0

  on: Con­sider donat­ing to Alex Bores, au­thor of the RAISE Act

  This means that an extra $300,000 would better position the campaign such that Alex Bores would be able to net an extra 1000 votes in expectation, which (as per my earlier estimate) has a 1.6% chance of counterfactually winning him the election. That would translate to $190,000 for a 1% increase in his chance of winning

  Sanity check: in 2023–2024, median new/​incumbent House member raised $2.4 million/​$2.1 million respectively (source), and median incumbent in a “toss-up” race raised $7.9 million.

  The median race raised ~10x more than the amount needed for a 1% move according to the model in OP. Is that reasonable? It sounds basically reasonable to me.

• MichaelDickens 7 Nov 2025 23:21 UTC

  6 points

  0

  in reply to: gjm’s comment on: De­bunk­ing “When Prophecy Fails”

  So it seems like Kelly’s critique is kinda self-defeating. If Dorothy Martin’s little UFO cult isn’t really an example of the mechanism Festinger popularized, in the way Kelly describes, then Festinger and his colleagues themselves are an even better example of it.

  I think there is a huge difference between the two situations, for several reasons:

  1. a large number of people sticking with a doomsday cult that’s been falsified, vs. a small number of authors fabricating evidence to advance their careers

  2. a trivially-verifiable prediction that’s falsified, vs. a prediction that involves a lot of research and data and is hard to check (I’m assuming the debunking is correct but I don’t really know because I haven’t looked at any of the evidence presented)

  3. as I understand, Festinger et al. did actually change their minds in response to evidence, but then they published findings they knew were false because it was in their personal interest to do so

• MichaelDickens 5 Nov 2025 15:41 UTC

  2 points

  3

  on: MichaelDick­ens’s Shortform

  Why does Eliezer dislike the paperclip maximizer thought experiment?

  Numerous times I have seen him correct people about it and say it wasn’t originally about a totalizing paperclip factory, it was about an AI that wants to make little squiggly lines for inscrutable reasons. Why does the distinction matter? Both scenarios are about an AI that does something very different from what you want and ends up killing you.

  My guess, although I’m not sure about this, is that the paperclip factory is an AI that did as instructed, but its instructions were bad and it killed everyone. Whereas the squiggly line thing is about AI not doing what you want. And perhaps the paperclip factory scenario could mislead people into believing that all you have to do is make sure the AI understands what you want.

  FWIW I always figured the paperclip maximizer would know that people don’t want it to turn the lightcone into paperclips, but it would do it anyway, so I still thought it was a reasonable example of the same principle as the squiggly-lines AI. But I can see how that conclusion requires two steps of reasoning whereas the squiggly-lines scenario only requires one step. Or perhaps the thing that Eliezer thinks is wrong with the paperclip-maximizer scenario is something else entirely.

• MichaelDickens 5 Nov 2025 15:36 UTC

  5 points

  0

  in reply to: Wei Dai’s comment on: Wei Dai’s Shortform

  people being more likely to click on or read shortforms due to less perceived effort of reading (since they’re often shorter and less formal)

  And because you can read them without loading a new page. I think that’s a big factor for me.

• MichaelDickens 4 Nov 2025 16:51 UTC

  6 points

  2

  in reply to: Raemon’s comment on: The Tale of the Top-Tier Intellect

  maybe they help workshop new analogies that eventually can be refined into If Anyone style books or podcast interviews.

  I think it’s helpful to write arguments multiple times. And I think it’s sensible to write out the argument in a “you-shape” and then refine it and try to make it more appealing to a broader range of people.

  This kind of post might also give fuel for someone else to make basically the same argument, but in a different style, which could end up being helpful.

Things I’ve Be­come More Con­fi­dent About

• MichaelDickens 1 Nov 2025 18:45 UTC

  4 points

  2

  in reply to: fx’s comment on: Su­pervillain Monologues Are Unrealistic

  we can never know our own true reason for doing something

  I read Sam Harris’s book on free will which I think is what you’re referring to, but I don’t recall him saying anything like that. If he did, I presume he meant something like “you don’t know which set of physical inputs to your neurons caused your neurons to fire in a way that caused your behavior”, which doesn’t mean you can’t have a belief about whether someone’s motivation is (say) religious or geopolitical.

• MichaelDickens 29 Oct 2025 16:21 UTC

  40 points

  22

  in reply to: Cleo Nardo’s comment on: straw­berry calm’s Shortform

  I think you could approximately define philosophy as “the set of problems that are left over after you take all the problems that can be formally studied using known methods and put them into their own fields.” Once a problem becomes well-understood, it ceases to be considered philosophy. For example, logic, physics, and (more recently) neuroscience used to be philosophy, but now they’re not, because we know how to formally study them.

  So I believe Wei Dai is right that philosophy is exceptionally difficult—and this is true almost by definition, because if we know how to make progress on a problem, then we don’t call it “philosophy”.

  For example, I don’t think it makes sense to say that philosophy of science is a type of science, because it exists outside of science. Philosophy of science is about laying the foundations of science, and you can’t do that using science itself.

  I think the most important philosophical problems with respect to AI are ethics and metaethics because those are essential for deciding what an ASI should do, but I don’t think we have a good enough understanding of ethics/​metaethics to know how to get meaningful work on them out of AI assistants.

• MichaelDickens 29 Oct 2025 16:04 UTC

  2 points

  0

  in reply to: David James’s comment on: David James’s Shortform

  What does economics-as-moral-foundation mean?

• MichaelDickens 26 Oct 2025 4:09 UTC

  4 points

  0

  in reply to: Raemon’s comment on: Foyle’s Shortform

  He mainly used analogies from IABED. Off the top of my head I recall him talking about

  • predicting where the molecules go when you heat an ice cube

  • LLMs are grown; the companies aren’t building crops, they are building farm equipment that grows crops (I don’t remember this one from IABED)

  • we know ASI could build a self-replicating solar-powered factory because those already exist (i.e. grass)

  • leaded gasoline as a case study of scientists/​companies making something bad and being in denial about it, even to their own detriment

  • many people thought nuclear war was inevitable, but it didn’t happen, largely because the people in charge would be personally harmed by it

• MichaelDickens 25 Oct 2025 4:29 UTC

  2 points

  2

  in reply to: kave’s comment on: kave’s Shortform

  I’m talking about my perception of the standards for a quick take vs. post. I don’t know if my perception is accurate

• MichaelDickens 25 Oct 2025 3:48 UTC

  4 points

  2

  in reply to: Tomás B.’s comment on: kave’s Shortform

  My perception is that it’s not exactly about goodness, it’s more like, a post must conform to certain standards*. In the same way that a scientific paper must meet certain standards to get published in a peer-reviewed journal, but a non-publishable paper could still present novel and valuable scientific findings.

  *and, even though I’ve been reading LW since 2012, I’m still not clear on what those standards are or how to meet them

• MichaelDickens 22 Oct 2025 21:41 UTC

  7 points

  4

  on: Con­sider donat­ing to Alex Bores, au­thor of the RAISE Act

  On a meta level, I think this post is a paragon of how to reason about the cost-effectiveness of a highly uncertain decision, and I would love to see more posts like this one.

• MichaelDickens 20 Oct 2025 21:01 UTC

  9 points

  −9

  on: Con­sider donat­ing to Alex Bores, au­thor of the RAISE Act

  I don’t know Bores personally. I looked through some of his communications and social media, most of it seemed reasonable (I noticed his Twitter has an unusually small amount of mud-slinging). I did see one thread with some troubling comments:

  This bill [SB 53] recognizes that in order to win the AI race, our AI needs to be both safe and trustworthy.

  In this case, pro-safety is the pro-innovation position.

  [...]

  As a New Yorker, I have to point out that SB53 includes a cloud compute cluster & @GavinNewsom said in his signing memo “The future happens [in CA] first”

  ...but @KathyHochul established EmpireAI in April 2024. So, thanks to our Gov’s vision, the future actually happens in NY 😉

  Why I find this troubling:

  • Bores seems to want to race to build AI. Racing shortens timelines and decreases safety.

  • “pro-safety is the pro-innovation position” seems false? If AI companies maximize profit by being safe, then they’d do it without regulation, so why would we need regulation? If they don’t maximize profit by being safe, then pro-safety is not (maximally) pro-innovation.

  • I think our best hope for survival is that governments become sufficiently aware of the danger of AI that they agree to ban frontier AI development until we can figure out how to make it safe. If Bores is indeed pro-innovation on AI, then he would presumably oppose such a ban. My guess is the average Democrat would be basically fine with banning frontier AI if the political winds shifted that way, but Bores would have a more strongly-held stance, in which case he would be worse than the average Democrat (but still probably better than the average Republican).

  • He calls out New York directing funding to a new AI research lab as if that’s a good thing, which I don’t think it is. (I don’t actually know what EmpireAI is doing, I looked at their website but it doesn’t really say anything, it says they only fund “responsible” research, but I really don’t trust them to know what qualifies as responsible.)

  Politicans are often pressured to say those sorts of things, so perhaps he would still support an AI pause if it became politically feasible. So these comments aren’t overwhelmingly troubling. But they’re troubling.

  If those quotes accurately reflect his stance on AI innovation and arms races, then he might still be better than the average Democrat if the increased chance of getting weak-to-moderate AI safety regulations outweighs the decreased chance of getting strong regulations, but it’s unclear to me.

  I will note that this was the only worrying comment I saw from Bores, although I didn’t find many comments on AI safety.

• MichaelDickens 20 Oct 2025 18:03 UTC

  11 points

  0

  on: Bub­ble, Bub­ble, Toil and Trouble

  I think the strongest case for AI stocks being overpriced is to ignore any specific facts about how AI works and take the outside view on historical market behavior. I don’t see a good argument being made in the quotes above so I will try to make a version of it.

  I’m going based on memory instead of looking up sources, I’m pretty sure I’m wrong about the exact details of the claims below but I believe they’re approximately true.

  • The Mag 7 have a P/​E of 32. Historically, when companies had a P/​E of 32, their average future returns were much worse than average (my guess would be 0–3%).

  • A study looking at the performance of the #1 market cap company found that the top company almost always went on to underperform, which is an argument against buying Nvidia in particular.

  • Nvidia as a % of the total world market cap is currently the largest any company has ever been in history. When things go outside the normal historical range, that generally suggests the price is unsustainable.

  • Historically, the market has systematically over-extrapolated earnings/​revenue growth. Companies with excellent earnings growth for years 1–3 tend to have merely above-average earnings growth for years 4–6, but they’re usually priced as if they’re going to continue to have excellent earnings growth. Although that’s just an average, and companies with very high market expectations still exceed expectations something like 40% of the time.

  • Stocks and industries tend to exhibit 3–5 year mean reversion, i.e. stocks that perform particularly well for 3–5 years on average underperform the market over the following year.

  (These are five different perspectives on the same general market phenomenon, so they’re not really five independent pieces of evidence.)

  ---

  On the outside view, I think there’s pretty good reason to believe that AI stocks are overpriced. However, on the inside view, the market sort of still doesn’t seem like it appreciates how big a deal AGI could be. So on balance I’m pretty uncertain.

• MichaelDickens 18 Oct 2025 0:46 UTC

  4 points

  0

  in reply to: habryka’s comment on: Mikhail Samin’s Shortform

  Importantly, AFAICT some Horizon fellows are actively working against x-risk (pulling the rope backwards, not sideways). So Horizon’s sign of impact is unclear to me. For a lot of people, “tech policy going well” means “regulations that don’t impede tech companies’ growth”.