Ben Pace

• Ben PaceJun 12, 2025, 5:38 AM

  4 points

  0

  in reply to: Zack_M_Davis’s comment on: β-re­dex’s Shortform

  Yep. From my perspective it’s pretty common to not understand or be able to easily empathize with minds that you interact with regularly (e.g. see Different Worlds by Scott). Especially in certain levels of depth, there are many people whose psychology confuses me, or people for whom I’ve puzzled over for a long time before understanding their basic attitude/​mood in most interactions. I’d happily generate some examples if you find this surprising?

• Ben PaceJun 12, 2025, 1:09 AM

  2 points

  0

  in reply to: Said Achmiz’s comment on: β-re­dex’s Shortform

  One answer is that people who genuinely have a hard time understanding other ppl are selected for heavily.

  To be clear I regularly have a hard time understanding other people, including on the internet, I don’t think it’s that unusual a state of affairs to be explaining.

• Ben PaceJun 12, 2025, 12:57 AM

  2 points

  0

  in reply to: MondSemmel’s comment on: Mikhail Samin’s Shortform

  I make the case here for stopping based on deontological rather than consequentialist reasons.

• Ben PaceJun 12, 2025, 12:51 AM

  2 points

  0

  in reply to: Said Achmiz’s comment on: β-re­dex’s Shortform

  A couple of notes:

  1. I think the position I most want to defend is that you can post something you know will get a lot of engagement, and it can also be true. I think when someone writes something like “I literally don’t understand why people choose to lie regularly in their daily lives” they might be aware that it could be a toxoplasma-kind of scissor-statement, and also they still don’t understand why people do it so much.

  2. I think it’s a very common experience of writers that they have a very poor ability to predict what gets a lot of attention and what does not. I think this is true of Aella and her tweets, in that she is often confident and wrong, or very surprised. So I don’t expect that all of her popular twits are expected to be.

• Ben PaceJun 11, 2025, 11:13 PM

  24 points

  23

  in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform

  I too believe we should ultimately judge things based on their consequences. I believe that having deontological lines against certain actions is something that leads humans to make decisions with better consequences, partly because we are bounded agents that cannot well-compute the consequences of all of our actions.

  For instance, I think you would agree that it would be wrong to kill someone in order to prevent more deaths, today here in the Western world. Like, if an assassin is going to kill two people, but says if you kill one then he won’t kill the other, if you kill that person you should still be prosecuted for murder. It is actually good to not cross these lines even if the local consequentialist argument seems to check out. I make the same sort of argument for being first in the race toward an extinction-level event. Building an extinction-machine is wrong, and arguing you’ll be slightly more likely to pull back first does not stop it from being something you should not do.
  
  I think when you look back at a civilization that raced to the precipice and committed auto-genocide, and ask where the lines in the sand should’ve been drawn, the most natural one will be “building the extinction machine, and competing to be first to do so”. So it is wrong to cross this line, even for locally net positive tradeoffs.

  What links here?

  • Ben Pace's comment on Mikhail Samin’s Shortform by Mikhail Samin (Jun 12, 2025, 12:57 AM; 2 points)

• Ben PaceJun 11, 2025, 11:03 PM

  4 points

  −6

  in reply to: Said Achmiz’s comment on: β-re­dex’s Shortform

  Actually, I confidently believe she (and other autistic rationalists) who ask these questions often are earnestly confused, and do not easily empathize with the other minds on these topics. I think there’s still selection effects on why such people’s comments are selected for, but it is not conscious “engagement-bait”.

• Ben PaceJun 11, 2025, 2:41 AM

  2 points

  −4

  in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform

  I’m not saying that it’s implausible that the consequences might seem better. I’m stating it’s still morally wrong to race toward causing a likely extinction-level event as that’s a pretty schelling place for a deontological lines against action.

• Ben PaceJun 11, 2025, 1:18 AM

  5 points

  0

  in reply to: sunwillrise’s comment on: Mikhail Samin’s Shortform

  I am not quite sure what an overheating gun refers to, I am guessing the idea is that it has some chance of going off without being fired.

  Anyhow, if that’s accurate, it’s acceptable to decide to be the person holding an overheating gun, but it’s not acceptable to (for example) accept a contract to assassinate someone so that you get to have the overheating gun, or to promise to kill slightly fewer people with the gun than the next guy. Like, I understand consequentially fewer deaths happen, but our society has deontological lines against committing murder even given consequentialist arguments, which are good. You’re not allowed to commit murder even if you have a good reason.

• Ben PaceJun 10, 2025, 6:37 PM

  1 point

  0

  in reply to: MondSemmel’s comment on: Mikhail Samin’s Shortform

  I just want to clarify that Anthropic doesn’t have the social authority of a governmental firing squad to kill people.

• Ben PaceJun 10, 2025, 6:35 PM

  6 points

  4

  in reply to: sunwillrise’s comment on: Mikhail Samin’s Shortform

  Just because someone else is going to kill me, doesn’t mean we don’t have an important societal norm against murder. You’re not allowed to kill old people just because they’ve only got a few years left, or kill people with terminal diseases.

• Ben PaceJun 10, 2025, 4:17 PM

  3 points

  3

  in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform

  If you are killing me and my friends because you think it better that you do the killing than someone else, then actually I will still ask you to stop, because I draw a hard line around killing me and my friends. Naturally, I have a similar line around developing tech that will likely kill me and my friends.

Lighthaven Se­quences Read­ing Group #37 (Tues­day 6/​10)

Lighthaven Se­quences Read­ing Group #36.5 (Tues­day 6/​3) [pa­tio11 meetup edi­tion]

• Ben PaceMay 30, 2025, 11:26 PM

  7 points

  0

  in reply to: Czynski’s comment on: Czyn­ski’s Shortform

  I heard someone (who I respect) say why they don’t post on LessWrong more. They said when they talk about their thoughts and ideas with their friends, the friends won’t question their basic frame /​ sanity, and so won’t undermine their trust in themself. They said that this is acceptable on LessWrong, which is a more uncomfortable experience.

  Ever since then I’ve tried a bit harder to make sure to question my friends’ basic frames and sanity, so that I’m not encouraging self-blinding in the people around me, and so they will know that they’re welcome to do the same to me.

• Ben PaceMay 30, 2025, 5:06 AM

  9 points

  4

  on: CFAR is run­ning an ex­per­i­men­tal mini-work­shop (June 2-6, Berkeley CA)!

  Yay! I’m happy to see this happening. I would strongly consider going if I wasn’t running LessOnline immediately before it (after which I will both be exhausted AND on call for festival-season work).

• Ben PaceMay 29, 2025, 2:53 PM

  29 points

  17

  in reply to: Joseph Miller’s comment on: Zac Hat­field Dodds’s Shortform

  I’d guess it looks more stable to investors. Unlike if you have a bunch of EAs on the OpenAI board and they confusingly try to fire the CEO for as unimportant a crime as lying; that’s quite hard for investors to predict.

Lighthaven Se­quences Read­ing Group #36 (Tues­day 5/​27)

• Ben PaceMay 26, 2025, 3:48 AM

  LW: 15 AF: 6

  10

  AF

  in reply to: ryan_greenblatt’s comment on: ryan_green­blatt’s Shortform

  I think the main thing I want to convey is that I think you’re saying that LWers (of which I am one) have a very low opinion of the integrity of people at Anthropic, but what I’m actually saying that their integrity is no match for the forces that they are being tested with.

  I don’t need to be able to predict a lot of fine details about individuals’ decision-making in order to be able to have good estimates of these two quantities, and comparing them is the second-most question relating to whether it’s good to work on capabilities at Anthropic. (The first one is a basic ethical question about working on a potentially extinction-causing technology that is not much related to the details of which capabilities company you’re working on.)

• Ben PaceMay 25, 2025, 8:05 PM

  LW: 2 AF: 2

  0

  AF

  in reply to: ryan_greenblatt’s comment on: ryan_green­blatt’s Shortform

  What’s an example decision or two where you would want to ask yourself whether they should get more or less open-ended power? I’m not sure what you’re thinking of.

• Ben PaceMay 25, 2025, 7:06 PM

  LW: 11 AF: 5

  9

  AF

  in reply to: ryan_greenblatt’s comment on: ryan_green­blatt’s Shortform

  Not the main thrust of the thread, but for what it’s worth, I find it somewhat anti-helpful to flatten things into a single variable of “how much you trust Anthropic leadership to make decisions which are good from your perspective”, and then ask how optimistic/​pessimistic you are about this variable.

  I think I am much more optimistic about Anthropic leadership on many axis relative to an overall survey of the US population or Western population – I expect them to be more libertarian, more in favor of free speech, more pro economic growth, more literate, more self-aware, higher IQ, and a bunch of things.

  I am more pessimistic about their ability to withstand the pressures of a trillion dollar industry to shape their incentives than the people who are at Anthropic.

  I believe the people working there are siloing themselves intellectually into an institution facing incredible financial incentives for certain bottom lines like “rapid AI progress is inevitable” and “it’s reasonably likely we can solve alignment” and “beating China in the race is a top priority”, and aren’t allowed to talk to outsiders about most details of their work, and this is a key reason that I expect them to screw up their decision-making.

  I am optimistic about their relative-ability to have a sensible conversation about the next 5 years and what alignment failures look like, relative to most people on earth. This is not the standard I require to expect people to not do ML training runs that lead to human extinction, but nonetheless I predict they will do relatively quite well on this axis.

  I don’t have a single variable here, I have a much more complicated model than this. It looks to me that collapsing questions of trust about people or groups into a single varibale of how optimistic I am about them making decisions which are good from my values has been a common question-substitution in the Effective Altruism scene, where I think people have been repeatedly hoodwinked by sociopaths due to not moving toward a more detailed model that predicts exactly where and when someone will make good vs bad decisions.