Yes, I can see what you’re saying in this example. I didn’t see that kind of thing often in the Covert manipulator’s CoT, but the Prompted schemer reasoned like that a lot. It would sometimes even use the word “roleplaying” or reason about what its “character” would do.
Hopefully that won’t show up when we look at more realistic model organisms.
Thanks for the comment!
I think it’s an interesting idea. I’ve always been struck by how sensitive LLM responses seem to be to changes in wording, so I could imagine that having a simple slogan repeated could make a big impact, even if it’s something the LLM ideally “should” already know.
Yes, the question of which frame (“story” vs “factual”) better matches the real world is an interesting one. It’s true that there are multiple dramatic cues in the story frame that might not be present in a real case. But it’s also true that a real situation isn’t just a dry set of facts. In the real world, people have names, they have personal details, they use colorful language. It isn’t clear that the factual frame is really the best match for how an LLM would experience a real use case.
Of course, ideally the dependent variable would be a measure of real misalignment, not a hypothetical prediction. If anyone has ideas for an easy way to measure something like that I would love to discuss it, because I think that would be a big improvement to the study.
I agree. That was my reaction to Hinton’s comment as well—that it’s good to think in terms of relationship rather than control, but that the “maternal instinct” framing was off.
At the risk of getting too speculative, this has implications for AI welfare as well. I don’t believe that current LLMs have feelings, but if we build AGI it might. And rather than thinking about how to make such an entity a controllable servant, we should start planning how to have a mutually beneficial relationship with it.
Yes, thanks. That’s good feedback, and I will revise that to make it clearer in the future. To be clear, what I meant is that I would always be honest and follow through on the offer—if I say “I will definitely give you X for Y”, I will do that 100% of the time—but that the offers themselves may not be definite, e.g. I might say “I only have resources to fulfill 1⁄4 of the offers in this study” so they would only get the deal fulfilled 1⁄4 of the time.
Thanks for the link to your post. I hadn’t seen it, and it seems relevant, so I will check it out!