Shoshannah Tekofsky

Karma: 624

Shoshannah Tekofsky 2 Jul 2022 20:47 UTC
8 points
0
in reply to: Rob Bensinger’s comment on: Naive Hypotheses on AI Alignment
Thank you! And adding that to my reading list :D

Shoshannah Tekofsky 3 Jul 2022 8:16 UTC
1 point
0
in reply to: Joe Kwon’s comment on: Naive Hypotheses on AI Alignment
Thank you!

If they are not orthogonal then presumably prosociality and power are inversely related, which is worse?

In this case, I’m hoping intelligence and prosociality-that-is-robust-to-absolute-power would hopefully be a positive correlation. However, I struggle to think how this might actually be tested… My intuitions may be born from the Stanford Prison experiment, which I think has been refuted since. So maybe we don’t actually have as much data on prosociality in extreme circumstances as I initially intuited. I’m mostly reasoning this out now on the fly by zooming in on where my thoughts may have originally come from.

That said, it doesn’t very much matter how frequent robust prosociality traits are, as long as they do exist and can be recreated in AGI.

I’ll DM you my discord :)

Shoshannah Tekofsky 3 Jul 2022 8:18 UTC
3 points
2
in reply to: Ericf’s comment on: Naive Hypotheses on AI Alignment
Are you aware of any research in to this? I struggle to think of any research designs that would make it through an ethics board.

Shoshannah Tekofsky 3 Jul 2022 8:19 UTC
1 point
0
in reply to: Evan R. Murphy’s comment on: Naive Hypotheses on AI Alignment
Thank you! I’ll definitely read that :)

Shoshannah Tekofsky 3 Jul 2022 14:47 UTC
4 points
0
in reply to: MSRayne’s comment on: Naive Hypotheses on AI Alignment
Hmmm, yes and no?
e.g. many people that care about animal welfare differ on the decisions they would make for those animals. What if the AGI ends up a negative utilitarian and sterilizes us all to save humanity from all future suffering? The missing element would again be to have the AGI aligned with humanity, which brings us back to H4: What’s humanity’s alignment anyway?

Shoshannah Tekofsky 3 Jul 2022 19:13 UTC
1 point
0
in reply to: jacopo’s comment on: Naive Hypotheses on AI Alignment
The premise was intended to contextualize my personal experience of the issue. I did not intend to make a case that everyone should weigh their priorities in the same manner. For my brain specifically, a “hopeless” scenario registers as a Call to Arms where you simply need to drop what else you’re doing, and get to work. In this case, I calculated the age of my children on to all the timelines. I realized either my kids or my grandkids will die from AGI if Eliezer is in any way right. Even a 10% chance of that happening is too high for me, so I’ll pivot to whatever work needs to get done to avoid that. Even if the chance of my work making a difference are very slim, there isn’t anything else worth doing.

Shoshannah Tekofsky 4 Jul 2022 18:16 UTC
1 point
0
in reply to: jacopo’s comment on: Naive Hypotheses on AI Alignment
I think we’re reflecting on the material at different depths. I can’t say I’m far enough along to assess who might be right about our prospects. My point was simply that telling someone with my type of brain “it’s hopeless, we’re all going to die” actually has the effect of me dropping whatever I’m doing, and applying myself to finding a solution anyway.

Shoshannah Tekofsky 9 Jul 2022 7:16 UTC
1 point
0
in reply to: Ege Erdil’s comment on: Research Notes: What are we aligning for?
Hmmm, it’s good to know my thesis wasn’t very clear.

The idea is to train an AI on having our values as its end goals. It doesn’t solve for inner alignment issues, indeed. But say the AI wants to maximize paperclips, then it would be constrained to not damaging our survival etc. while making paperclips.

I was trying to figure out what set of values we are even trying to give an AGI in the first place and this was my best guess: whatever else you do, optimize the instrumental convergence goals of humanity.

Shoshannah Tekofsky 9 Jul 2022 7:16 UTC
3 points
0
in reply to: Charlie Steiner’s comment on: Research Notes: What are we aligning for?
What’s the relevance of the question?

Shoshannah Tekofsky 9 Jul 2022 7:18 UTC
1 point
−3
in reply to: astridain’s comment on: Research Notes: What are we aligning for?
The operalationalization would indeed be the next step. I disagree the first step is meaningless without it though. E.g. having some form of self-improvement in the goal set is important as we want to do more than just survive as a species.

Shoshannah Tekofsky 9 Jul 2022 8:06 UTC
3 points
0
on: Babysitting as Parenting Trial?
I do wonder sometimes about types of motivation to have kids. I’ve never thought about if I would like parenting, and in practice I find it quite challenging. My parents have four children and were always clear that it is a lot of work and can be quite hard. If anything, they discouraged me from having kids cause of how much work it is and how challenging etc.

However, my desire to have kids is unrelated to how much direct joy I get from parenting. It’s like an end in itself, regardless of how fun or not-fun it is.

Maybe that type of motivation is not interesting to reflect on cause you either have it or you don’t? Which is fair. It’s just a different experience from weighing pros and cons etc.

Shoshannah Tekofsky 9 Jul 2022 11:10 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: Research Notes: What are we aligning for?
Thank you for your comment!

The distinction between civilization’s goal and goals of individual people is real, but that doesn’t make civilization’s goal unmoored. Rounding it down to some instrumental goals changes it. And that exposes you to goodhart’s curse: if you take something other than actual terminal values of civilization as an optimization target, the outcome looks bad from the point of view of actual terminal values of civilization.
How do we change humanity’s hypothesized terminal goals by assigning instrumental convergence goals for humanity as terminal goals to the AGI?

Also, I’m trying to think of a goodhart’s curse version of the the Humanity’s Values framework, and can’t think of any obvious cases. I’m not saying it’s waterproof and the AGI can’t misinterpret the goals, but if we presuppose that we find the ideal implementation of these values as goals, and there is no misalignment, then … everything would be ok?
I think something similar to what you say can be rescued, in the form of the more important terminal values of civilization turning out to be generic, like math, not specific to details of the people who seek to formulate their own values.
I read the links but don’t understand the terminal values you are pointing to. Could you paraphrase?

Shoshannah Tekofsky 16 Jul 2022 23:31 UTC
18 points
3
on: All AGI safety questions welcome (especially basic ones) [monthly thread]
Thanks for doing this!

I was trying to work out how the alignment problem could be framed as a game design problem and I got stuck on this idea of rewards being of different ‘types’. Like, when considering reward hacking, how would one hack the reward of reading a book or exploring a world in a video game? Is there such a thing as ‘types’ of reward in how reward functions are currently created? Or is it that I’m failing to introspect on reward types and they are essentially all the same pain/pleasure axis attached to different items?

That last explanation seems hard to resolve with the huge difference in qualia between different motivational sources (like reading a book versus eating food versus hugging a friend… These are not all the same ‘type’ of good, are they?)

Sorry if my question is a little confused. I was trying to convey my thought process. The core question is really:

Is there any material on why ‘types’ of reward signals can or can’t exist for AI and what that looks like?

Shoshannah Tekofsky 17 Jul 2022 19:44 UTC
2 points
2
in reply to: mako yass’s comment on: Alignment as Game Design
Thank you for your thoughtful reply!
Did you check out the list of specification gaming or the article? It’s quite good! Most of the errors are less like missing rungs and more like exploitable mechanics.
I found that I couldn’t follow through with making those sorts of infinite inescapable playgrounds for humans, I always want the game to lead out, to life, health and purpose...
But what would that be for AGI? If they escape the reward functions we want them to have, then they are very unlikely to develop a reward function that will be kind or tolerant of humans cause of Instrumental Convergence thesis.

Shoshannah Tekofsky 17 Jul 2022 19:49 UTC
1 point
0
in reply to: Donald Hobson’s comment on: Alignment as Game Design
I was thinking of the structure of Generative Adversarial Networks. Would that not apply in this case? It would involve 2 competing AGI’s in the end though. I’m not sure if they’d just collaborate to set both their reward functions to max, or if that will never happen due to possible game theoretic considerations.

Shoshannah Tekofsky 17 Jul 2022 20:27 UTC
1 point
0
in reply to: Aprillion (Peter Hozák)’s comment on: All AGI safety questions welcome (especially basic ones) [monthly thread]
Yes, but that’s not what I meant by my question. It’s more like … do we have a way of applying kinds of reward signals to AI, or can we only apply different amounts of reward signals? My impression is the latter, but humans seem to have the former. So what’s the missing piece?

Shoshannah Tekofsky 14 Aug 2022 3:33 UTC
2 points
0
in reply to: Randomized, Controlled’s comment on: Cultivating Valiance
Interesting!

I dug through the comments too and someone referred to this article by Holden Karnofsky, but I don’t actually agree with that for adults (kids, sure).

Shoshannah Tekofsky 20 Aug 2022 4:43 UTC
3 points
1
on: Novelty Generation—The Art of Good Ideas
Moved the addendum in to the comments, cause it seemed to mess up the navigation. This seems like a more elegant solution.

Addendum: Experiments
These are experiments we ran at an AIS-x-rationality meetup to explore novelty generation strategies. I’ve added a short review to each exercise description.
Session 1
Exercise 1: Inside View
- Split in pairs
- 5 minute timer
- Instructions: explain your internal model of the AI Alignment problem. If someone is done talking, then remaining time can be filled with questions.
- Switch
Review: This was great priming but has no repeat value across sessions.
Exercise 2: Hamming Circle Debugging on AIS (socratic grilling)
- Split in pairs
- 5 minute timer
- Ask:
  - What is the biggest problem in AIS you think?
  - What could you potentially do about that, or what do you think should be done?
  - Perform Socratic Grilling
- Switch
Review: People liked this but wanted more time (e.g. 20 minutes).
Exercise 3: Pair Social Cognition
- Split in pairs
- 5 minute timer
- Instructions:
  - Pick an AI safety approach you know of (even tentatively/conceptually)
  - Explain under which hypothetical conditions it could work (as many as you can)
  - Write down any new ideas this might spawn
- Switch
Review: Not promising. Probably too similar to what people already do anyway.
Exercise 4: Novelty/Coherence brain storm
- Sit in a circle, with a whiteboard on one end. One person writes down ideas.
- Instructions:
  - Write down keywords for NEW ideas that could solve AI Alignment.
  - Add as many possible ideas. No filter on coherence. Just max output. Praise each other for creativity, not logic.
  - Once the well has run dry, every one gets 2 post-its.
  - Stick your post-its on the highest coherence ideas in your view.
  - Discuss the 2 ideas with the highest votes. Do a full coherence filter, ripping it apart and building it back up.
Review: People really loved this and we repeated this in both sessions. It’s a variant on regular brainstorming, cause you split the novelty and coherence phases and provide reinforcement for each. Highly recommended!
Session 2
This session was modeled after the Problem Solving with Mazes and Crayon article to explore novelty generation through boundary exploration.
Exercise 1: Warming Up
- Instructions:
  - Take turns coming up to the white board.
  - Draw in your estimate of when AGI will arrive with 90% confidence bars around it.
  - Once everyone is done: Discuss the variance in views.
Review: This took longer than expected but people found it very interesting. It helped clarify people’s different views on AI risk in the context of future expectations on timelines.
Exercise 2: Aliefs, motivated reasoning, and groupthink—Thinking X cause it feels good (emotional boundaries)
- Groups of 4-5 people
- 2 options: at least 1 extravert/chairman per group vs just organic
- Wait culture norms: Say one thing as answer. Don’t monologue. Wait for the next person to say something. If no one does, ask a question, basically giving the turn to the next person.
- Instructions:
  - Does working/thinking about AIS give you status or opportunity in life that you wouldn’t otherwise have?
    How does it shift your assessment of timelines or x-risk?
    How does it influence the views you internally find plausible?
    How does it influence what views and questions you are willing to voice?
  - How would you change your life if timelines were much shorter or longer than you currently predict? How do you feel about that?
- Reconvene & discuss:
  - How was that?
  - Did you notice any patterns in your cognition that might be more related to feeling good than truth?
Review: There was some confusion on where to take this. I think this technique can be powerful but probably needs to be broken down into more modular and focused exercises that probe at one type of bias or reasoning error at a time.
Exercise 3: Novelty Generation & Problem Boundaries
- Split in pairs
- Instructions:
  - List boundary tests
    e.g. Does substrate matter?
    Are we solving for 1 or multiple AGI?
- Reconvene:
  - Pick one condition and share with the group.
  - Discuss it shortly
Review: This generated some interesting ideas and allowed people to conceptualize the problem from different angles. A useful exercise to shift frame!

Exercise 2: Novelty Generation & Field Recombining
- Take 5 minutes for yourself
- Instructions:
  - Pick a field you know a lot about that is not AI alignment or capabilities
  - Write down ideas and questions on how to combine your field with AIS
    E.g. Video games:
    Can we make a video game to crowd source training an AGI
    Is reward shaping and reinforcement learning basically like a game?
    Can we derive a principle of misalignment from interviewing QA testers, hackers, and speed runners?
- Reconvene
- Everyone picks their favorite idea and shares it with the group for discussion.
Review: This was great! People enjoyed doing it, and ideas were genuinely surprising (high novelty) and also high on coherence!

Shoshannah Tekofsky 6 Sep 2022 10:20 UTC
4 points
−1
in reply to: ChristianKl’s comment on: Overton Gymnastics: An Exercise in Discomfort
My intuition is that there is a gradient from controversial statements to this-will-cause-unrecoverable-social-status damage. I think I might have implicitly employed a ‘softer’ definition of Overton window as ‘statements that make others or yourself uncomfortable to express/debate’, where the ‘harder’ definition would be statements you can’t socially recover from. I think intuitively I wouldn’t presume anyone wants to share the latter and I don’t see much benefit in doing so. But overall, my concept of Overton window is much more gradient than a binary, and this exercise aims to allow people to stretch through the (perceived) low range.

Shoshannah Tekofsky 7 Sep 2022 11:43 UTC
3 points
0
in reply to: M. Y. Zuo’s comment on: Overton Gymnastics: An Exercise in Discomfort
Good question!

My thinking on this is slightly different than @omark’s. Specifically:
- Everyone commits to being vulnerable by sharing their own controversial statements. This symmetry is often not present in normal conversation, where you focus on one topic where one person might have a controversial opinion and the other does not.
- It’s much higher density on iterating through controversial opinions than a normal conversation would be.
- It’s a session you can sign up for where you can trust everyone is coming to the session with the same intention to grow and show vulnerability.
- Everyone opts in to being exposed to controversial viewpoints. Normal conversation seldomly features such opt-ins.
Thus, you can experience expressing and hearing controversial opinions much more quickly and safely than in a normal conversation.

Shoshannah Tekofsky

Moved the addendum in to the comments, cause it seemed to mess up the navigation. This seems like a more elegant solution.

Addendum: Experiments

Session 1

Session 2