Corin Katzke

Sce­nario plan­ning for AI x-risk

AISN#15: China and the US take ac­tion to reg­u­late AI, re­sults from a tour­na­ment fore­cast­ing AI risk, up­dates on xAI’s plan, and Meta re­leases its open-source and com­mer­cially available Llama 2

In­ves­ti­gat­ing the role of agency in AI x-risk

AISN #16: White House Se­cures Vol­un­tary Com­mit­ments from Lead­ing AI Labs and Les­sons from Oppenheimer

• Corin Katzke 13 Feb 2024 17:50 UTC

  5 points

  0

  in reply to: Zac Hatfield-Dodds’s comment on: Sce­nario plan­ning for AI x-risk

  Thank you for reading and responding to it! For what it’s worth, some of these ideas got rolling during your “AI safety under uncertainty” workshop at EAG Boston.

• Corin Katzke 13 Feb 2024 17:47 UTC

  5 points

  2

  in reply to: Zac Hatfield-Dodds’s comment on: Sce­nario plan­ning for AI x-risk

  Yep, another good point, and in principle I agree. A couple of caveats, though:
  
  First, it’s not clear to me that experts would agree on enough dynamics to make these clusters predicatively reliable. There might be agreement on the dynamics between scaling laws and timelines (and that’s a nice insight!) — but the Killian et al. paper considered 14 variables, which (for example) would be 91 pairwise dynamics to agree on. I’d at least like some data on whether conditional forecasts converge. I think FRI is doing some work on that.

  Second, the Grace et al. paper suggested that expert forecasts exhibited framing effects. So, even if experts did agree on underlying dynamics, those agreements might not be able to be reliably elicited. But maybe conditional forecasts are less susceptible to framing effects.

• Corin Katzke 13 Feb 2024 17:30 UTC

  3 points

  0

  in reply to: Zac Hatfield-Dodds’s comment on: Sce­nario plan­ning for AI x-risk

  Thanks for the clarification! I didn’t mean to imply that Anthropic hasn’t been thinking about the full spectrum of risk — only that “misuse” and “autonomy and replication” are the two categories of catastrophic risk explicitly listed in the RSP.
  
  If I do think of a good way to evaluate accident risks before deployment, I’ll definitely let you know. (I might actually pitch my team to work on this.)

• Corin Katzke 13 Feb 2024 17:22 UTC

  3 points

  0

  in reply to: Zac Hatfield-Dodds’s comment on: Sce­nario plan­ning for AI x-risk

  Yep, fair enough. I agree that an MTBF of millions of years is an alternative sustainable theory of victory.
  
  Could you expand on “the challenge is almost entirely in getting to an acceptably low rate”? It’s not clear to me that that’s true. For example, it seems plausible that at some point nuclear risk was at an acceptably low rate (maybe post-fall of the USSR? I’m niether an expert nor old enough to remember) conditional on a further downward trend — but we didn’t get a further downward trend.

• Corin Katzke 3 Dec 2023 0:19 UTC

  1 point

  0

  on: Re­spon­si­ble Scal­ing Poli­cies Are Risk Man­age­ment Done Wrong

  It’s called “responsible scaling”. In its own name, it conveys the idea that not further scaling those systems as a risk mitigation measure is not an option.

  That seems like an uncharitable reading of “responsible scaling.” Strictly speaking, the only thing that name implies is that it is possible to scale responsibly. It could be more charitably interpreted as “we will only scale when it is responsible to do so.” Regardless of whether Anthropic is getting the criteria for “responsibility” right, it does seem like their RSP leaves open the possibility of not scaling.