zchuang

• zchuang 18 Mar 2025 10:32 UTC

  1 point

  0

  on: Claude Son­net 3.7 (of­ten) knows when it’s in al­ign­ment evaluations

  Can you share the difference between the times it is aware (I know 33% is the floor) and the times where Claude is not aware/​sandbagging?

zchuang’s Shortform

• zchuang 18 Mar 2025 10:29 UTC

  7 points

  −2

  on: zchuang’s Shortform

  The Claude plays pokemon discourse makes me feel out of touch because I don’t understand how this is not an update people had with chess playing LLMs. If anything, I think the data contamination is worse for pokemon because of gamefaq and other guides online.

  I think people are mistaking benchmarks and trailheads. Maths benchmarks are important because they theoretically speed up AI research and benchmark to useful intelligence. Claude playing pokemon doesn’t tell you much because it’s not a map on intelligence nor of generalisation.

• zchuang 7 Mar 2025 4:13 UTC

  9 points

  0

  in reply to: gwern’s comment on: China Hawks are Man­u­fac­tur­ing an AI Arms Race

  Sorry, but why did you connect apathy and lying-flat [sic] with fast-follower culture. Lying flat or tang ping is about youth apathy and nihilism about the job market and existential angst about life in the 21st century. It’s not about corporate or industrial culture from the top end or specific technological political economy strategies.

• zchuang 4 Mar 2025 5:52 UTC

  19 points

  0

  in reply to: Daniel Kokotajlo’s comment on: Daniel Koko­ta­jlo’s Shortform

  I don’t know if this is helpful but as someone who was quite good at competitive Pokemon during their teenage years and also still keeps up with nuzlocking type things for fun, I would note that Pokemon’s game design is made to be a low context intensity RPG especially in early generations where the linearity is pushed to allow kids to do it.
  
  If your point holds true on agency, I think the more important pinch points will be Lavender Town and Sabrina because those require backtracking through the storyline to get things.

  I think mid-late game GSC would also be important to try because there are huge level gaps and transitions in the storyline that would make it hard to progress.

• zchuang 8 Jul 2023 23:20 UTC

  6 points

  0

  in reply to: RussellThor’s comment on: What Does LessWrong/​EA Think of Hu­man In­tel­li­gence Aug­men­ta­tion as of mid-2023?

  Sorry but aren’t we in a fast takeoff world at the point of WBE. What’s the disjunctive world of no recursive self-improvement and WBE?

• zchuang 6 Jun 2023 6:23 UTC

  2 points

  0

  in reply to: Jayson_Virissimo’s comment on: State­ment on AI Ex­tinc­tion—Signed by AGI Labs, Top Aca­demics, and Many Other Notable Figures

  He posted on a twitter a request to talk to people who feel strongly here.

• zchuang 10 May 2023 16:46 UTC

  3 points

  0

  in reply to: Daniel Kokotajlo’s comment on: Clar­ify­ing and pre­dict­ing AGI

  Yeah, re-reading I realise I was unclear. Given your claim: “by the time we get to 2000 in that, such AGIs will be automating huge portions of AI R&D,”. I’m asking the following:

  1. Is the 2000 mark predicated on automation of things we can’t envision now (finding secret sauce to singularity) or is it predicated off pushing existing things like AI R&D finds better compute or is it a combination of both?

  2. What’s the empirical on the ground representative modal action you’re seeing at 2025 from either your vignette (e.g. I found the diplomacy AI super important for grokking what short timelines were to me). I guess it’s more asking what you see as the divergence between you and Richard at 2025 that’s represented by the difference of 25 and 100.

  Hopefully that made the questions clearer.

• zchuang 10 May 2023 14:59 UTC

  3 points

  0

  in reply to: Daniel Kokotajlo’s comment on: Clar­ify­ing and pre­dict­ing AGI

  Sorry for a slightly dumb question but in your part of the table you set 2000 as the year before singularity and your explanation is that 2000-second tasks jump to singularity. Is your model of fast take-off then contingent on there being more special sauce for intelligence being somewhat redundant as a crux because recursive self-improvement is just much more effective. I’m having trouble envisioning a 2000-second task + more scaling and tuning --> singularity.

  Additional question is what your model of falsification is for let’s say 25-second task vs. 100-second task in 2025 because it seems like reading your old vignettes you really nailed the diplomacy AI part.

  Also slightly pedantic but there’s a typo on 2029 on Richard’s guess.

“AI Risk Dis­cus­sions” web­site: Ex­plor­ing in­ter­views from 97 AI Researchers