Shoshannah Tekofsky

Naive Hy­pothe­ses on AI Alignment

• Shoshannah Tekofsky 2 Jul 2022 20:47 UTC

  8 points

  0

  in reply to: Rob Bensinger’s comment on: Naive Hy­pothe­ses on AI Alignment

  Thank you! And adding that to my reading list :D

• Shoshannah Tekofsky 3 Jul 2022 8:16 UTC

  1 point

  0

  in reply to: Joe Kwon’s comment on: Naive Hy­pothe­ses on AI Alignment

  Thank you!

  If they are not orthogonal then presumably prosociality and power are inversely related, which is worse?

  In this case, I’m hoping intelligence and prosociality-that-is-robust-to-absolute-power would hopefully be a positive correlation. However, I struggle to think how this might actually be tested… My intuitions may be born from the Stanford Prison experiment, which I think has been refuted since. So maybe we don’t actually have as much data on prosociality in extreme circumstances as I initially intuited. I’m mostly reasoning this out now on the fly by zooming in on where my thoughts may have originally come from.

  That said, it doesn’t very much matter how frequent robust prosociality traits are, as long as they do exist and can be recreated in AGI.

  I’ll DM you my discord :)

• Shoshannah Tekofsky 3 Jul 2022 8:18 UTC

  3 points

  2

  in reply to: Ericf’s comment on: Naive Hy­pothe­ses on AI Alignment

  Are you aware of any research in to this? I struggle to think of any research designs that would make it through an ethics board.

• Shoshannah Tekofsky 3 Jul 2022 8:19 UTC

  1 point

  0

  in reply to: Evan R. Murphy’s comment on: Naive Hy­pothe­ses on AI Alignment

  Thank you! I’ll definitely read that :)

• Shoshannah Tekofsky 3 Jul 2022 14:47 UTC

  4 points

  0

  in reply to: MSRayne’s comment on: Naive Hy­pothe­ses on AI Alignment

  Hmmm, yes and no?

  e.g. many people that care about animal welfare differ on the decisions they would make for those animals. What if the AGI ends up a negative utilitarian and sterilizes us all to save humanity from all future suffering? The missing element would again be to have the AGI aligned with humanity, which brings us back to H4: What’s humanity’s alignment anyway?

• Shoshannah Tekofsky 3 Jul 2022 19:13 UTC

  1 point

  0

  in reply to: jacopo’s comment on: Naive Hy­pothe­ses on AI Alignment

  The premise was intended to contextualize my personal experience of the issue. I did not intend to make a case that everyone should weigh their priorities in the same manner. For my brain specifically, a “hopeless” scenario registers as a Call to Arms where you simply need to drop what else you’re doing, and get to work. In this case, I calculated the age of my children on to all the timelines. I realized either my kids or my grandkids will die from AGI if Eliezer is in any way right. Even a 10% chance of that happening is too high for me, so I’ll pivot to whatever work needs to get done to avoid that. Even if the chance of my work making a difference are very slim, there isn’t anything else worth doing.

• Shoshannah Tekofsky 4 Jul 2022 18:16 UTC

  1 point

  0

  in reply to: jacopo’s comment on: Naive Hy­pothe­ses on AI Alignment

  I think we’re reflecting on the material at different depths. I can’t say I’m far enough along to assess who might be right about our prospects. My point was simply that telling someone with my type of brain “it’s hopeless, we’re all going to die” actually has the effect of me dropping whatever I’m doing, and applying myself to finding a solution anyway.

Re­search Notes: What are we al­ign­ing for?

• Shoshannah Tekofsky 9 Jul 2022 7:16 UTC

  1 point

  0

  in reply to: Ege Erdil’s comment on: Re­search Notes: What are we al­ign­ing for?

  Hmmm, it’s good to know my thesis wasn’t very clear.

  The idea is to train an AI on having our values as its end goals. It doesn’t solve for inner alignment issues, indeed. But say the AI wants to maximize paperclips, then it would be constrained to not damaging our survival etc. while making paperclips.

  I was trying to figure out what set of values we are even trying to give an AGI in the first place and this was my best guess: whatever else you do, optimize the instrumental convergence goals of humanity.

• Shoshannah Tekofsky 9 Jul 2022 7:16 UTC

  3 points

  0

  in reply to: Charlie Steiner’s comment on: Re­search Notes: What are we al­ign­ing for?

  What’s the relevance of the question?

• Shoshannah Tekofsky 9 Jul 2022 7:18 UTC

  1 point

  −3

  in reply to: astridain’s comment on: Re­search Notes: What are we al­ign­ing for?

  The operalationalization would indeed be the next step. I disagree the first step is meaningless without it though. E.g. having some form of self-improvement in the goal set is important as we want to do more than just survive as a species.

• Shoshannah Tekofsky 9 Jul 2022 8:06 UTC

  3 points

  0

  on: Babysit­ting as Par­ent­ing Trial?

  I do wonder sometimes about types of motivation to have kids. I’ve never thought about if I would like parenting, and in practice I find it quite challenging. My parents have four children and were always clear that it is a lot of work and can be quite hard. If anything, they discouraged me from having kids cause of how much work it is and how challenging etc.

  However, my desire to have kids is unrelated to how much direct joy I get from parenting. It’s like an end in itself, regardless of how fun or not-fun it is.

  Maybe that type of motivation is not interesting to reflect on cause you either have it or you don’t? Which is fair. It’s just a different experience from weighing pros and cons etc.

• Shoshannah Tekofsky 9 Jul 2022 11:10 UTC

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Re­search Notes: What are we al­ign­ing for?

  Thank you for your comment!
  

  The distinction between civilization’s goal and goals of individual people is real, but that doesn’t make civilization’s goal unmoored. Rounding it down to some instrumental goals changes it. And that exposes you to goodhart’s curse: if you take something other than actual terminal values of civilization as an optimization target, the outcome looks bad from the point of view of actual terminal values of civilization.

  How do we change humanity’s hypothesized terminal goals by assigning instrumental convergence goals for humanity as terminal goals to the AGI?
  
  Also, I’m trying to think of a goodhart’s curse version of the the Humanity’s Values framework, and can’t think of any obvious cases. I’m not saying it’s waterproof and the AGI can’t misinterpret the goals, but if we presuppose that we find the ideal implementation of these values as goals, and there is no misalignment, then … everything would be ok?

  I think something similar to what you say can be rescued, in the form of the more important terminal values of civilization turning out to be generic, like math, not specific to details of the people who seek to formulate their own values.

  I read the links but don’t understand the terminal values you are pointing to. Could you paraphrase?

Align­ment as Game Design

• Shoshannah Tekofsky 16 Jul 2022 23:31 UTC

  18 points

  3

  on: All AGI safety ques­tions wel­come (es­pe­cially ba­sic ones) [monthly thread]

  Thanks for doing this!

  I was trying to work out how the alignment problem could be framed as a game design problem and I got stuck on this idea of rewards being of different ‘types’. Like, when considering reward hacking, how would one hack the reward of reading a book or exploring a world in a video game? Is there such a thing as ‘types’ of reward in how reward functions are currently created? Or is it that I’m failing to introspect on reward types and they are essentially all the same pain/​pleasure axis attached to different items?

  That last explanation seems hard to resolve with the huge difference in qualia between different motivational sources (like reading a book versus eating food versus hugging a friend… These are not all the same ‘type’ of good, are they?)

  Sorry if my question is a little confused. I was trying to convey my thought process. The core question is really:

  Is there any material on why ‘types’ of reward signals can or can’t exist for AI and what that looks like?

• Shoshannah Tekofsky 17 Jul 2022 19:44 UTC

  2 points

  2

  in reply to: mako yass’s comment on: Align­ment as Game Design

  Thank you for your thoughtful reply!

  Did you check out the list of specification gaming or the article? It’s quite good! Most of the errors are less like missing rungs and more like exploitable mechanics.

  I found that I couldn’t follow through with making those sorts of infinite inescapable playgrounds for humans, I always want the game to lead out, to life, health and purpose...

  But what would that be for AGI? If they escape the reward functions we want them to have, then they are very unlikely to develop a reward function that will be kind or tolerant of humans cause of Instrumental Convergence thesis.

• Shoshannah Tekofsky 17 Jul 2022 19:49 UTC

  1 point

  0

  in reply to: Donald Hobson’s comment on: Align­ment as Game Design

  I was thinking of the structure of Generative Adversarial Networks. Would that not apply in this case? It would involve 2 competing AGI’s in the end though. I’m not sure if they’d just collaborate to set both their reward functions to max, or if that will never happen due to possible game theoretic considerations.

• Shoshannah Tekofsky 17 Jul 2022 20:27 UTC

  1 point

  0

  in reply to: Aprillion (Peter Hozák)’s comment on: All AGI safety ques­tions wel­come (es­pe­cially ba­sic ones) [monthly thread]

  Yes, but that’s not what I meant by my question. It’s more like … do we have a way of applying kinds of reward signals to AI, or can we only apply different amounts of reward signals? My impression is the latter, but humans seem to have the former. So what’s the missing piece?

Cul­ti­vat­ing Valiance