Chris_Leong

• Chris_Leong 21 Jul 2024 13:39 UTC

  2 points

  0

  in reply to: habryka’s comment on: Towards more co­op­er­a­tive AI safety strategies

  I would be keen to see the memo if you’re comfortable sharing it privately.

[Question] Have peo­ple given up on iter­ated dis­til­la­tion and am­plifi­ca­tion?

• Chris_Leong 17 Jul 2024 2:46 UTC

  2 points

  −1

  in reply to: GideonF’s comment on: Against Aschen­bren­ner: How ‘Si­tu­a­tional Aware­ness’ con­structs a nar­ra­tive that un­der­mines safety and threat­ens humanity

  The definition of macrosecuritisation failure includes within it ‘failure to combat the existential threat’.

  
  This seems like a poorly chosen definition that’s simply going to confuse any discussion of the issue.

  The track record of national security has been (as shown) very poor, so I don’t know why pessimism around macrosecuritisation should make you endorse it.

  If neither macrosecuritisation or a pause a likely to occur, what’s the alternative if not Aschenbrenner?
  
  (To clarify, I’m suggesting outreach to the national security folks, not necessarily an AI Manhattan project, but I’m expecting the former to more or less inevitably lead to the latter).

• Chris_Leong 16 Jul 2024 7:24 UTC

  5 points

  1

  on: Against Aschen­bren­ner: How ‘Si­tu­a­tional Aware­ness’ con­structs a nar­ra­tive that un­der­mines safety and threat­ens humanity

  Some bits of this felt a bit too much like an appeal to authority to me.

  I am also wary about attempts that try to abstract the situation too much. In many ways, AI is an unusual technology. Maybe it’s my own reading comprehension, but I came away from this without really understanding why you think human securitisation is feasible for AI.

  Thus, if national securitization narratives winning out leads to macrosecuritization failure as Sears (2023) seems to show, definitionally it is dangerous for our ability to deal with the threat.

  Definitionally? You can’t prove facts about the world by definition.

  For what it’s worth, I’m increasingly coming over to Leopold’s side. There seems to be far too much political opposition to any sensible action and I don’t know how we can overcome this without conducting more outreach to the national security folks. One reason I’ve shifted more this direction is that Trump is looking likely to win the nomination and JD Vance is his running mate. JD Vance seems to think that AI safety is all about wokenes. The national security folks might have enough pull with the Republicans to make a difference here.

• Chris_Leong 13 Jul 2024 12:57 UTC

  14 points

  7

  on: Robin Han­son AI X-Risk De­bate — High­lights and Analysis

  Cool to see you doing this!

• Chris_Leong 9 Jul 2024 13:54 UTC

  2 points

  0

  on: Es­say com­pe­ti­tion on the Au­toma­tion of Wis­dom and Philos­o­phy — $25k in prizes

  Do you only take single-author submissions?

• Chris_Leong 9 Jul 2024 0:52 UTC

  3 points

  0

  on: ZuVillage Ge­or­gia – Mis­sion Statement

  I can’t see any information about the cost listed anywhere.

• Chris_Leong 6 Jul 2024 0:24 UTC

  LW: 2 AF: 1

  0

  AF

  in reply to: Gordon Seidoh Worley’s comment on: Find­ing the Wis­dom to Build Safe AI

  You mean it might still Goodhart to what we think they might say? Ideally, the actual people would be involved in the process.

• Chris_Leong 5 Jul 2024 3:30 UTC

  14 points

  7

  on: Con­sider the hum­ble rock (or: why the dumb thing kills you)

  I think that for a long time the alignment community was betting on recursive self-improvement happening pretty early in which there wouldn’t a gap between AI being able to develop bioweapons and developing AGI.

  Over time, people have updated towards thinking that the path towards AGI will be slower and more gradual, and I think people were either hoping that we could either tank whatever harms happen in this period or that governments might deal with it.
  
  Now that some of these harms are basically imminent, I think the community has updated towards being more worried about some of the less weird harms like misinformation or bioweapons, and I agree with this, but I don’t want to take this too far either, as I mostly just see this work as buying time for us to solve alignment.

• Chris_Leong 5 Jul 2024 3:22 UTC

  2 points

  0

  on: In­tro­duc­tion to French AI Policy

  Do you have any more information on what the National Center for AI Evaluation will be doing?

• Chris_Leong 4 Jul 2024 23:02 UTC

  LW: 6 AF: 3

  3

  AF

  on: Find­ing the Wis­dom to Build Safe AI

  My intuition is that the best way to build wise AI would be to train imitation learning agents on people who we consider to be wise. If we trained imitations of people with a variety of perspectives, we could then simulate discussions between them and try to figure out the best discussion formats between such agents. This could likely get us reasonably far.
  
  The reason why I say imitation learning is because that would give us something that we could treat as an optimisation target which is what we require for training ML systems.

• Chris_Leong 1 Jul 2024 4:57 UTC

  4 points

  0

  in reply to: mesaoptimizer’s comment on: mesaop­ti­mizer’s Shortform

  I think I’ve been (slowly) making progress.
  
  I think we would be able to make progress on this if people seriously wanted to make progress, but understandably it’s not the highest priority.

• Chris_Leong 19 Jun 2024 18:33 UTC

  0 points

  −3

  in reply to: mesaoptimizer’s comment on: Ilya Sutskever cre­ated a new AGI startup

  Paywalled. Would be fantastic if someone with access could summarise the most important bits.

• Chris_Leong 19 Jun 2024 6:19 UTC

  2 points

  0

  in reply to: Elo’s comment on: Kenshō

  What do you mean by “pure clear seeing”?

• Chris_Leong 11 Jun 2024 5:28 UTC

  2 points

  0

  on: DPO/​PPO-RLHF on LLMs in­cen­tivizes syco­phancy, ex­ag­ger­a­tion and de­cep­tive hal­lu­ci­na­tion, but not mis­al­igned powerseeking

  Seems like at some point we’ll need to train on outputs too complex for humans to evaluate, then we’ll end up using training methods based on outcomes in some simulation.

• Chris_Leong 9 Jun 2024 9:30 UTC

  2 points

  0

  in reply to: jacobjacob’s comment on: Two easy things that maybe Just Work to im­prove AI discourse

  I’d participate.

• Chris_Leong 6 Jun 2024 23:16 UTC

  2 points

  0

  in reply to: cousin_it’s comment on: I’m open for pro­jects (sort of)

  I don’t want to universally claim that we should always work with consistent universes, but I think we should have a strong bias toward reformulating problems to be consistent when we can. This involves imagining that the past is different in the counterfactual.

• Chris_Leong 5 Jun 2024 13:47 UTC

  LW: 6 AF: 2

  0

  AF

  on: An­nounc­ing ILIAD — The­o­ret­i­cal AI Align­ment Conference

  How are applications processed? Sometimes applications are processed on a rolling basis, so it’s important to submit as soon as possible. Other times, you just need to apply by the date, so if you’re about to post something big, it makes sense to hold-off your application.

• Chris_Leong 5 Jun 2024 1:26 UTC

  4 points

  7

  in reply to: Thomas Kwa’s comment on: Prometheus’s Shortform

  I would suggest that it’s important to make this distinction to avoid sending people off down the wrong path.

• Chris_Leong 4 Jun 2024 23:05 UTC

  2 points

  0

  in reply to: Thomas Kwa’s comment on: Prometheus’s Shortform

  Why not work on a role related to application though?