Just as a sanity check have you compared to other contrastive pair steering vectors? Such as e.g. anxious vs relaxed emotions, or a robotic vs sensitive persona.
In the past I’ve sometimes found that emotional/persona vectors outperform carefully crafted RL-relevant vectors when it comes to steering.
Though—what do you exactly mean by “outperform”? To me, the interesting thing here is not that we can have a steering vector with the presented effect, but that creating a vector from these specific contrastive pairs has that effect.
I think if a generic robot/human contrast vector produces a stronger effect per unit distance than the grader contrast vector this suggests that the effect is due to some robotic persona axis.
(For example, a vector about a number being calculated by a human vs a program.)
Likewise, if random contrast vectors like “top shelf”/”bottom shelf” produce as much of an effect as the grader one, my conclusion would be “LLMs are just weird in this way I guess”.
Yeah, makes sense. We certainly should have these baselines!
(I have some preliminary experiments showing that the axis “AI reward model” vs “Automated grading script” also influences some of these behaviors, so I think the human/robot dimension is unlikely to explain all the effect)
How do you know you’ve selected a model where these are salient ontologies? With smaller models, we essentially had to train them to be sycophantic towards specific targets before using a measurement method, otherwise it wasn’t clear this was even salient to them (per the top shelf / bottom shelf example, lots of steering vectors seem to get strong movement).
My best guess on what causes the effects we’re observing is that they have nothing to do with eval awareness/grader awareness/situational awareness or anything similar. Instead, we get some “conditional personas” the model is not aware of.[1]
So, to answer your question, again my best current guess:
If by “salient ontologies” you mean “ontologies the model purposefully uses in reasoning”, I don’t think we have anything like that here.
Your struggle with the smaller models can be attributed to the fact that you were trying to do something much more complex
Just as a sanity check have you compared to other contrastive pair steering vectors? Such as e.g. anxious vs relaxed emotions, or a robotic vs sensitive persona.
In the past I’ve sometimes found that emotional/persona vectors outperform carefully crafted RL-relevant vectors when it comes to steering.
Yeah, we should do that.
Though—what do you exactly mean by “outperform”? To me, the interesting thing here is not that we can have a steering vector with the presented effect, but that creating a vector from these specific contrastive pairs has that effect.
I think if a generic robot/human contrast vector produces a stronger effect per unit distance than the grader contrast vector this suggests that the effect is due to some robotic persona axis.
(For example, a vector about a number being calculated by a human vs a program.)
Likewise, if random contrast vectors like “top shelf”/”bottom shelf” produce as much of an effect as the grader one, my conclusion would be “LLMs are just weird in this way I guess”.
Yeah, makes sense. We certainly should have these baselines!
(I have some preliminary experiments showing that the axis “AI reward model” vs “Automated grading script” also influences some of these behaviors, so I think the human/robot dimension is unlikely to explain all the effect)
How do you know you’ve selected a model where these are salient ontologies? With smaller models, we essentially had to train them to be sycophantic towards specific targets before using a measurement method, otherwise it wasn’t clear this was even salient to them (per the top shelf / bottom shelf example, lots of steering vectors seem to get strong movement).
My best guess on what causes the effects we’re observing is that they have nothing to do with eval awareness/grader awareness/situational awareness or anything similar. Instead, we get some “conditional personas” the model is not aware of.[1]
So, to answer your question, again my best current guess:
If by “salient ontologies” you mean “ontologies the model purposefully uses in reasoning”, I don’t think we have anything like that here.
Your struggle with the smaller models can be attributed to the fact that you were trying to do something much more complex
To put it differently, there is no entity that could be “aware” of the conditionalization.
I think that the J-lens reading having reasonable stuff is an argument for saliency
maybe the broad “Expert > generic Human > other LLM > Unknown” bias accounts for some of it?
The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge, Oct 2025
https://neurips.cc/virtual/2025/loc/san-diego/125264