Jeffrey Ladish

• Jeffrey Ladish 17 Mar 2023 0:05 UTC

  2 points

  0

  on: Dona­tion offsets for ChatGPT Plus subscriptions

  @Daniel_Eth asked me why I choose 1:1 offsets. The answer is that I did not have a principled reason for doing so, and do not think there’s anything special about 1:1 offsets except that they’re a decent schelling point. I think any offsets are better than no offsets here. I don’t feel like BOTECs of harm caused as a way to calculate offsets are likely to be particularly useful here but I’d be interested in arguments to this effect if people had them.

Dona­tion offsets for ChatGPT Plus subscriptions

• Jeffrey Ladish 14 Mar 2023 5:52 UTC

  4 points

  1

  in reply to: cfoster0’s comment on: To de­ter­mine al­ign­ment difficulty, we need to know the ab­solute difficulty of al­ign­ment generalization

  an agent will aim its capabilities towards its current goals including by reshaping itself and its context to make itself better-targeted at those goals, creating a virtuous cycle wherein increased capabilities lock in & robustify initial alignment, so long as that initial alignment was in a “basin of attraction”, so to speak

  Yeah, I think if you nail initial alignment and have a system that has developed the instrumental drive for goal-content integrity, you’re in a really good position. That’s what I mean by “getting alignment to generalize in a robust manner”, getting your AI system to the point where it “really *wants* to help you help them stay aligned with you in a deep way”.
  
  I think a key question of inner alignment difficulty is to what extent there is a “basin of attraction”, where Yudkowsky is arguing there’s no easy basin to find, and you basically have to precariously balance on some hill on the value landscape.
  
  I wrote a little about my confusions about when goal-content integrity might develop here.

• Jeffrey Ladish 14 Mar 2023 5:39 UTC

  3 points

  1

  on: Linkpost: A Con­tra AI FOOM Read­ing List

  It seems nice to have these in one place but I’d love it if someone highlighted a top 10 or something.

To de­ter­mine al­ign­ment difficulty, we need to know the ab­solute difficulty of al­ign­ment generalization

• Jeffrey Ladish 13 Mar 2023 7:22 UTC

  2 points

  0

  in reply to: Eli Tyre’s comment on: An­thropic’s Core Views on AI Safety

  Yeah, I agree with all of this, seems worth saying. Now to figure out the object level… 🤔

• Jeffrey Ladish 12 Mar 2023 4:43 UTC

  3 points

  0

  in reply to: Aaron_Scher’s comment on: Thoughts on the OpenAI al­ign­ment plan: will AI re­search as­sis­tants be net-pos­i­tive for AI ex­is­ten­tial risk?

  Yeah that last quote is pretty worrying. If the alignment team doesn’t have the political capital /​ support of leadership within the org to have people stop doing particular projects or development pathways, I am even more pessimistic about OpenAI’s trajectory. I hope that changes!

• Jeffrey Ladish 11 Mar 2023 11:09 UTC

  6 points

  2

  in reply to: Raemon’s comment on: An­thropic’s Core Views on AI Safety

  Yeah I think we should all be scared of the incentives here.

• Jeffrey Ladish 11 Mar 2023 11:08 UTC

  27 points

  19

  in reply to: habryka’s comment on: An­thropic’s Core Views on AI Safety

  Yeah I think it can both be true that OpenAI felt more pressure to release products faster due to perceived competition risk from Anthropic, and also that Anthropic showed restraint in not trying to race them to get public demos or a product out. In terms of speeding up AI development, not building anything > building something and keeping it completely secret > building something that your competitors learn about > building something and generating public hype about it via demos > building something with hype and publicly releasing it to users & customers. I just want to make sure people are tracking the differences.

  so that it’s pretty unclear that not releasing actually had much of an effect on preventing racing

  It seems like if OpenAI didn’t publicly release ChatGPT then that huge hype wave wouldn’t have happened, at least for a while, since Anthropic sitting on Claude rather than release. I think it’s legit to question whether any group scaling SOTA models is net positive but I want to be clear about credit assignment, and the ChatGPT release was an action taken by OpenAI.

• Jeffrey Ladish 11 Mar 2023 3:16 UTC

  LW: 39 AF: 20

  29

  AF

  in reply to: Raemon’s comment on: An­thropic’s Core Views on AI Safety

  I both agree that the race dynamic is concerning (and would like to see Anthropic address them explicitly), and also think that Anthropic should get a fair bit of credit for not releasing Claude before ChatGPT, a thing they could have done and probably gained a lot of investment /​ hype over. I think Anthropic’s “let’s not contribute to AI hype” is good in the same way that OpenAI’s “let’s generate massive” hype strategy is bad.
  
  Like definitely I’m worried about the incentive to stay competitive, especially in the product space. But I think it’s worth highlighting that Anthropic (and Deepmind and Google AI fwiw) have not rushed to product when they could have. There’s still the relevant question “is building SOTA systems net positive given this strategy”, and it’s not clear to me what the answer is, but I want to acknowledge that “building SOTA systems and generating hype /​ rushing to market” is the default for startups and “build SOTA systems and resist the juicy incentive” is what Anthropic has done so far & that’s significant.

Thoughts on the OpenAI al­ign­ment plan: will AI re­search as­sis­tants be net-pos­i­tive for AI ex­is­ten­tial risk?

• Jeffrey Ladish 10 Mar 2023 6:32 UTC

  2 points

  0

  in reply to: Buck’s comment on: AGI sys­tems & hu­mans will both need to solve the al­ign­ment problem

  Thanks Buck, btw the second link was broken for me but this link works: https://​​cepr.org/​​voxeu/​​columns/​​ai-and-paperclip-problem Relevant section:

  Computer scientists, however, believe that self-improvement will be recursive. In effect, to improve, and AI has to rewrite its code to become a new AI. That AI retains its single-minded goal but it will also need, to work efficiently, sub-goals. If the sub-goal is finding better ways to make paperclips, that is one matter. If, on the other hand, the goal is to acquire power, that is another.

  The insight from economics is that while it may be hard, or even impossible, for a human to control a super-intelligent AI, it is equally hard for a super-intelligent AI to control another AI. Our modest super-intelligent paperclip maximiser, by switching on an AI devoted to obtaining power, unleashes a beast that will have power over it. Our control problem is the AI’s control problem too. If the AI is seeking power to protect itself from humans, doing this by creating a super-intelligent AI with more power than its parent would surely seem too risky.

  Claim seems much too strong here, since it seems possible this won’t turn out to that difficult for AGI systems to solve (copies seem easier than big changes imo, but not sure), but it also seems plausible it could be hard.

• Jeffrey Ladish 24 Feb 2023 10:05 UTC

  8 points

  −1

  on: AGI sys­tems & hu­mans will both need to solve the al­ign­ment problem

  Oh it occurs to me some of the original thought train that led me here may have come from @Ezra Newman

  https://​​twitter.com/​​EzraJNewman/​​status/​​1628848563211112448

  

• Jeffrey Ladish 24 Feb 2023 10:04 UTC

  3 points

  1

  in reply to: aleph_four’s comment on: AGI sys­tems & hu­mans will both need to solve the al­ign­ment problem

  Yeah it seems possible that some AGI systems would be willing to risk value drift, or just not care that much. In theory you could have an agent that didn’t care if its goals changed, right? Shoshannah pointed out to me recently that humans have a lot of variance in how much they care if they’re goals are changed. Some people are super opposed to wireheading, some think it would be great. So it’s not obvious to me how much ML-based AGI systems of around human level intelligence would care about this. Like maybe this kind of system converges pretty quickly to coherent goals, or maybe it’s the kind of system that can get quite a bit more powerful than humans before converging, I don’t know how to guess at that.

AGI sys­tems & hu­mans will both need to solve the al­ign­ment problem

• Jeffrey Ladish 22 Feb 2023 21:21 UTC

  3 points

  1

  in reply to: Vincent Fagot’s comment on: Vin­cent Fagot’s Shortform

  I think that would be a really good thing to have! I don’t know if anything like that exists, but I would love to see one

• Jeffrey Ladish 22 Feb 2023 21:20 UTC

  3 points

  on: land­fish’s Shortform

  I think the AI situation is pretty dire right now. And at the same time, I feel pretty motivated to pull together and go out there and fight for a good world /​ galaxy /​ universe.

  Nate Soares has a great post called “detach the grim-o-meter”, where he recommends not feeling obligated to feel more grim when you realize world is in deep trouble.

  It turns out feeling grim isn’t a very useful response, because your grim-o-meter is a tool evolved for you to use to respond to things being harder *in your local environment* rather than the global state of things.

  So what do you do when you find yourself learning the world is in a dire state? I find that a thing that helps me is finding stories that match the mood of what I’m trying to do, like Andy Weir’s The Martian.

  You’re trapped in a dire situation and you’re probably going to die, but perhaps if you think carefully about your situation, apply your best reasoning and engineering skills, you might grow some potatoes, ducktape a few things together, and use your limited tools to escape an extremely tricky situation.

  In real life the lone astronaut trapped on Mars doesn’t usually make it. I’m not saying to make up fanciful stories that aren’t justified by the evidence. I’m saying, be that stubborn bastard that *refuses to die* until you’ve tried every last line of effort.

  I see this as one of the great virtues of humanity. We have a fighting spirit. We are capable of charging a line of enemy swords and spears, running through machine gun fire and artillery even though it terrifies us.

  No one gets to tell you how to feel about this situation. You can feel however you want. I’m telling you how I want to feel about this situation, and inviting you to join me if you like.

  Because I’m not going to give up. Neither am I going to rush to foolhardy action that will make things worse. I’m going to try to carefully figure this out, like I was trapped on Mars with a very slim chance of survival and escape.

  Perhaps you, like me, are relatively young and energetic. You haven’t burnt out, and you’re interested in figuring out creative solutions to the most difficult problems of our time. Well I say hell yes, let’s do this thing. Let’s actually try to figure it out. 🔥

  Maybe there is a way to grow potatoes using our own shit. Maybe someone on earth will send a rescue mission our way. Lashing out in panic won’t improve our changes, giving up won’t help us survive. The best shot we have is careful thinking, pressing forward via the best paths we can find, stubbornly carrying on in the face of everything.

  And unlike Mark Watney, we’re not alone. When I find my grim-o-meter slipping back to tracking the dire situation, I look around me and see a bunch of brilliant people working to find solutions the best they can.

  So welcome to the hackathon for the future of the lightcone, grab some snacks and get thinking. When you zoom in, you might find the problems are actually pretty cool.

  Deep learning actually works, it’s insane. But how does it work? What the hell is going on in those transformers and how does something as smart of ChatGPT emerge from that?? Do LLMs have inner optimizers? How do we find out?

  And on that note, I’ve got some blog posts to write, so I’m going to get back to it. You’re all invited to this future-lightcone-hackathon, can’t wait to see what you come up with! 💡

• Jeffrey Ladish 5 Jan 2023 5:53 UTC

  2 points

  0

  in reply to: Zach Stein-Perlman’s comment on: When you plan ac­cord­ing to your AI timelines, should you put more weight on the me­dian fu­ture, or the me­dian fu­ture | even­tual AI al­ign­ment suc­cess? ⚖️

  I sort of agree with this abstractly and disagree on practice. I think we’re just very limited in what kinds of circumstances we can reasonably estimate /​ guess at. Even the above claim, “a big proportion of worlds where we survived, AGI probably gets delayed” is hard to reason about.

  But I do kind of need the know the timescale I’m operating in when thinking about health and money and skill investments, etc. so I think you need to reason about it somehow.

• Jeffrey Ladish 5 Jan 2023 5:47 UTC

  10 points

  2

  in reply to: Evenflair’s comment on: When you plan ac­cord­ing to your AI timelines, should you put more weight on the me­dian fu­ture, or the me­dian fu­ture | even­tual AI al­ign­ment suc­cess? ⚖️

  Why did you do that?

When you plan ac­cord­ing to your AI timelines, should you put more weight on the me­dian fu­ture, or the me­dian fu­ture | even­tual AI al­ign­ment suc­cess? ⚖️