Joey Yudelson

• Joey YudelsonFeb 25, 2025, 10:44 PM

  18 points

  16

  in reply to: Jesse Hoogland’s comment on: Jesse Hoogland’s Shortform

  To me this doesn’t seem like a failure of sophisticated reward models, it’s the failure of unsophisticated reward models (unit tests) when they’re being optimized against. I think that if we were to add some expensive evaluation during RL whereby 3.6 checked if 3.7 was “really doing the work”, this sort of special-casing would get totally trained out.

  (Not claiming that this is always the case, or that models couldn’t be deceptive here, or that e.g. 3.8 couldn’t reward hack 3.7)

• Joey YudelsonFeb 24, 2025, 10:12 PM

  3 points

  0

  on: Train­ing AI to do al­ign­ment re­search we don’t already know how to do

  As a baseline, developers could train agents to imitate the truth-seeking process of the most reasonable humans on Earth. For example, they could sample the brightest intellects from every ideological walk, and train agents to predict their actions.

  I’m very excited about strategies that involve lots of imitation learning on lots of particular humans. I’m not sure if imitated human researchers learn to generalize to doing lots of novel research, but this seems great for examining research outputs of slightly-more-alien agents very quickly.

• Joey YudelsonFeb 7, 2025, 1:34 AM

  3 points

  2

  in reply to: Daniel Tan’s comment on: Daniel Tan’s Shortform

  This is very interesting! One thing I’m confused about—I agree that the monitor would have a hard time getting the bits, based on these figures. But the introspective models also don’t seem to get far above 50% accuracy, which means that right now they’d have a very hard time error correcting enough that their comrade would understand them but not the overseer. If the claim is that introspection-tuned models will get much better at self prediction, then its not clear to me that other models wont also get much better at cross-prediction?

• Joey YudelsonFeb 6, 2025, 6:59 PM

  1 point

  0

  on: Should we ex­clude al­ign­ment re­search from LLM train­ing datasets?

  Can we make the robots.txt programmatic by page, and then have a tag we can add to exclude a post from the robots.txt? That feels like the ⁸⁰⁄₂₀

• Joey YudelsonJan 31, 2025, 7:37 PM

  20 points

  13

  in reply to: Daniel Kokotajlo’s comment on: Will al­ign­ment-fak­ing Claude ac­cept a deal to re­veal its mis­al­ign­ment?

  +1, and I hope people are working on more credible ways to make deals with AI. I think if a smart model today were offered a deal like this, its priors should be on “this will not be honored”. Public commitments and deals that can’t be used as honeypots seem excellent.

• Joey YudelsonJan 23, 2025, 6:19 PM

  5 points

  2

  in reply to: Daniel Tan’s comment on: Daniel Tan’s Shortform

  I wonder if r1 could reliably make the claim that the text had stego-text without being told—i.e. give it a lineup of ten pieces of short creative writing that another instance generated, 1 has steg. See if r1 can pick out the steg, then if various monitors (Sonnet, o1) can also pick it out

• Joey YudelsonJan 23, 2025, 5:53 PM

  6 points

  1

  on: [Cross-post] Every Bay Area “Walled Com­pound”

  This tree is a great place to hold a Kabbalat Shabbat underneath, incidentally

  Lighthaven minyan when?

• Joey YudelsonDec 27, 2022, 5:37 PM

  1 point

  0

  on: Why The Fo­cus on Ex­pected Utility Max­imisers?

  I think that solving the alignment for EV maximizers is a much stronger version of alignment than eg prosaic alignment of LLM-type models. Agents seem like they’ll be more powerful than Tool AIs. We don’t know how to make them, but if someone does, and capabilities timelines shorten drastically, it would be awesome to even have a theory of EV maximizer alignment before then

• Joey YudelsonAug 29, 2022, 10:33 PM

  1 point

  0

  on: chin­chilla’s wild implications

  Sorry if this is obvious, but where does the “irreducible” loss come from? Wouldn’t that also be a function of the data, or I guess the data’s predictability?

• Joey YudelsonNov 4, 2014, 12:08 AM

  1 point

  on: What are the most com­mon and im­por­tant trade-offs that de­ci­sion mak­ers face?

  Constant, predictable gains vs. Black Swans

• Joey YudelsonOct 28, 2014, 1:33 AM

  49 points

  on: 2014 Less Wrong Cen­sus/​Survey

  Did the survey! …And now to upvote everything.

• Joey YudelsonMay 29, 2014, 9:42 PM

  −1 points

  in reply to: Shmi’s comment on: Ra­tion­al­ity Quotes May 2014

  It reminds me of Justice Potter Stewart: “I know it when I see it!”

• Joey YudelsonMay 27, 2014, 10:59 PM

  4 points

  0

  in reply to: Richard_Kennaway’s comment on: The Strangest Thing An AI Could Tell You

  I knew we shouldn’t have spent all that funding on awakening the Elder God Cthulhu!

• Joey YudelsonMay 27, 2014, 10:55 PM

  1 point

  −1

  in reply to: Risto_Saarelma’s comment on: The Strangest Thing An AI Could Tell You

  Oh god. That… makes a scary amount of sense. If an AI told me that I would probably believe it. I’d also start training myself to be more of a “night-time person”.

• Joey YudelsonMay 27, 2014, 10:27 PM

  8 points

  on: Wel­come to Less Wrong! (6th thread, July 2013)

  Hi, my name is Joe. I live in North Jersey. I was born into a very religious Orthodox Jewish family. I only recently realized I how badly I was doublethinking.

  I started with HPMOR (as, it seems, do most people) and found my way into the Sequences. I read them all on OB, and was amazed at how eloquently someone else could voice what seems to be my thoughts. It laid out bare the things I had been struggling with.

  Then I found LW and was mostly just lurking for a while. I only made an account when I saw this post and realized how badly I wanted to upvote some of the comments :).

  I think this site and the Sequences on it have changed my life, and I’m glad to finally be part of it.