I’d like to thank the authors for this, I really appreciate this line of research. I also noticed that it is being discussed on Ars Technica here https://arstechnica.com/ai/2025/03/researchers-astonished-by-tools-apparent-success-at-revealing-ais-hidden-motives/
Some time ago I discussed a “role-based” approach (based on steering vectors rather than prompts; I called the roles “dispositional traits”, but it’s pretty much the same thing) to buy time while working on true alignment; maybe this approach will achieve true alignment, but (for now) there is no mathematical guarantee it really can!
In case anyone would be interested, here is my post—I am always interested in being challenged. https://www.lesswrong.com/posts/Bf3ryxiM6Gff2zamw/control-vectors-as-dispositional-traits
Gianluca Calcagni
How do you envision access and control of an AI that is robustly and reasonably compliant? And in which way would “human values” be involved? I agree with you that they are part of the solution, but I want to compare my beliefs with yours.
I want to thank the author about this post, that is a very interesting read. I see many comments already and I didn’t have the time to read them thoroughly, so my apologies if what I am stating below has been discussed already.
The author’s point that I wish to challenge is the third one: “alignment is not about loving humanity; it’s about robust reasonable compliance”.
I agree with the fact that embedding (in a principled way) “love” for humanity is a bad solution, and that it cannot go in a favourable direction—as, in the best scenario, it will likely disempower us. However, I disagree about the fact that “human values” are not part of the solution either.
Suppose that alignment is solved in the form of “robust reasonable compliance”: if human values are not embedded at all, the aligned robots will be aligned to single individuals or single organizations, and the effect is that the AIs will be used to fight proxy wars (not necessarily physical wars) among such organizations. This scenario is encouraging rogue AIs on purpose as a form of deterrent among the parties, in the same way nuclear bombs are built as a form of deterrent. That behaviour is an obvious existential risk. I have written a post about this here.
If you agree with the above, there are only two possible solutions: (1) either AI is monopolized by the UN, or (2) it is decentralized but it shares some form of human values, so that it will refuse to do unsafe or unethical things (not sure if that is even possible though, as I discussed at the end of this post).
Thanks to anyone that took the time to read and vote this post—regardless if it was a positive or a negative vote, I still appreciate it.
If you happen to downvote me, I’d appreciate it if you could explain the reason why: this is the second time that happens (for one of my previous posts I chose a title that was sounding like clickbait—I then corrected it), and I am curious to understand your feedback this time as well.
The reason why I write posts here from time to time is simply to be challenged and be exposed to different points of view: that cannot happen without an exchange (even if harsh).
Let me take this chance to wish you all a happy new year 2025!
Gianluca
How Business Solved (?) the Human Alignment Problem
I Recommend More Training Rationales
Hi Andrew, your post is very interesting and it made me think more carefully about the definition of consciousness and how it applies to LLMs. I’d be curious to get your feedback about a post of mine that, in my opinion, is related to yours—I am keen to receive even harsh judgement if you have any!
https://www.lesswrong.com/posts/e9zvHtTfmdm3RgPk2/all-the-following-are-distinct
Can AI Quantity beat AI Quality?
Something quite unexpected happened in the past 19 hours: since I published this post, I received over 12 downvotes! I wasn’t expecting lots of feedback anyway, but this time I was definitely caught by surprise by looking at a negative result.
It’s okay if my point of view doesn’t resonate with the community (being popular is not the reason why I write here), however I am intrigued by this reaction and I’d like to investigate it.
If you happen to read my post and you decide to downvote it, please proceed—but I’d appreciate it if you could explain the reason why. I m happy to be challenged and I will accept even harsh judgements, if that’s how you feel.
An Opinionated Look at Inference Rules
Thanks @Gunnar_Zarncke , I appreciate your comment! You correctly identified my goal, I am trying to ground the concepts and build relationships “from the top to the bottom”, but I don’t think I can succeed alone.
I kindly ask you to provide some challenges: is there any area that you feel “shaky”? Any relation in particular that is too much open to interpretation? Anything obviously missing from the discussion?
All the Following are Distinct
Thanks Seth for your post! I believe I get your point, and in fact I made a post that described exactly that approach. in detail I recommend conditioning the model by using an existing technique called control vectors (or steering vectors), that achieves a raw but incomplete form of safety—in my opinion, just enough partial safety to work on solutioning full safety with the help of AIs.
Of course, I am happy to be challenged.
Very happy to support you :)
It took some time to understand your paper, please find below a few comments:
(1) You are using SVD to find the control vectors (similarly to other authors) but your process is more sophisticated in the following ways: the generation of the matrices, how to reduce them, and how to choose the magnitude of each steering vector. You are also using the non-steered response as an active part of your calculations—something that is marginally done by other authors. The final result works, but the process looks arbitrary to me (tbh all the steering techniques are a bit arbitrary at the moment). What’s the added value of your operations? Maybe you have some intuition about why your calculation is finding the “correct” amount of steering, and I am curious to know more.
(2) Ethics plays a fundamental role in finding a collective solution to AI safety, but I tend to think that we should solve alignment first. It would be interesting to see your future research going in that direction. I can help brainstorming some topics that have not been exhaustively studied yet. Let me know!
Thanks Neel, keep this coming—even if only once every few years :) You helped me clarify lots of confusion I had about the existing techniques.
I am a huge fan of steering vectors / control vectors, and I would love to see future research showing if they can be linearly combined together to achieve multiple behaviours simultaneously (I made a post about this). I don’t think it’s just “internal work”—I think it’s a hint to the fact that language semantics can be linearised as vector spaces (I hope I will be able to formalise mathematically this intuition).
Here a proposal of a possible ELK solution using that approach.
I am surprised I didn’t find any reference to Tim Urban’s “Wait But Why” post What Makes You You.
In short, he argues that “you” is your sense of continuity, rather than your physical substance. He also argues that if (somehow) your mind was copied&pasted somewhere else, then a brand new “not-you” would be born—even though it may share 100% of your memory and behaviour.
In that sense, Tim argues that Theseus’ ship is always “one” despite all its parts are changed over time. If you were to disassemble and reassemble the ship, it would lose its continuity and it could arguably be considered a different ship.
Hi Christopher, thanks for your work! I have high expectations about steering techniques in the context of AI Safety. I actually wrote a post about it, I would appreciate it if you have the time to challenge it!
https://www.lesswrong.com/posts/Bf3ryxiM6Gff2zamw/control-vectors-as-dispositional-traits
I included a link to your post in mine, because they are strongly connected.
Thanks for sharing this research, it’s very promising. I am looking into collecting a list of steering vectors that may “force” a model into behaving safely—and I believe this should be included as well.
I’d be grateful if you could challenge my approach in a constructive way!
https://www.lesswrong.com/posts/Bf3ryxiM6Gff2zamw/control-vectors-as-dispositional-traits
Thank you Paul, this post clarifies many open points related to AI (inner) alignment, including some of its limits!
I recently described a technique called control vectors to force a LLM model to show specific dispositional traits, in order to condition some form of alignment (but definitely not true alignment).
I’d happy to be challenged! In my opinion, the importance of control vectors is definitely underestimated for AI safety. https://www.lesswrong.com/posts/Bf3ryxiM6Gff2zamw/control-vectors-as-dispositional-traits
Thanks to the author for this post and this study! I tend to think that it would be safer to systematically curb directive, expressive, judicative, or suggestive acts (I am using these terms based on speech act’s theory) while training LLMs. Playing any role other than a pure analyst is very likely going to bring unexpected results. I wrote this idea as trait 9 in one of my posts here https://www.lesswrong.com/posts/Bf3ryxiM6Gff2zamw/control-vectors-as-dispositional-traits