Nice work, the prospect of an in-the-wild interp benchmark is exciting!
Is your impression that the interp tools suffered more from surfacing misleading hypotheses or missing the right hypothesis? And if it’s more so poor recall, is a possible explanation that a lot of the behaviors you look at are data-driven behaviors so there’s little cognitive processing to pick up on?
(By data-driven behaviors I mean correlation-driven, e.g., a possible explanation for the example in Sam’s comment is most lists of TV shows in the model’s training data start with the best one)
I think it’s more common to miss the right hypothesis. The most common issue is that tool outputs almost never contain causal relationships, like “feature X causes behavior Y”.
This question about “is there enough cognitive processing to pick up on?” is pretty common. I think there are two underlying questions here:
Is there little cognitive processing in our examples?
Do we care about behaviors with little cognitive processing?
For 1, I think it’s hard to say. I created an LLM-judge based “coherence filter” that filtered out around 60% of the initial examples to obtain datapoints that I intuitively thought had a coherent enough mechanism that may require cognitive processing. I hoped that we could find some uplift with these filtered examples. You can view randomly sampled investigations here to get a sense for what survives the filter.
People’s intuitions for what requires processing vary. With all four of the below examples (from Figure 2 in our paper), I think it’s plausible that the model may or may not be using cognitive processing when performing the behaviors, and we don’t have a ground truth that can inform us. For example, maybe there’s a correlation-driven behavior to take reckless and unsafe actions when imitating a bold persona.
For 2, I think we still care about behaviors that require little cognitive processing, as e.g. models may take harmful actions when they are unaware of the underlying causes. While this isn’t as concerning as something like deliberate scheming, I think we would like to understand it. I would also be much happier with a tool that can explain the behaviors I selected versus a tool that cannot, and I think we would want to avoid only evaluating what tools currently succeed at. I agree that we don’t care about all behaviors, and some such as the “picking from a list” behavior may not be that interesting.
I’ll also note that our result may not be that surprising based on prior results. While we see e.g. many NLA verbalizations of eval awareness, it’s (AFAIK) very rare to see NLA verbalizations that involve causality like “I’m behaving safely only because I’m in an eval”. This is similar to our finding that tool outputs almost never mention causal relationships.
Nice work, the prospect of an in-the-wild interp benchmark is exciting!
Is your impression that the interp tools suffered more from surfacing misleading hypotheses or missing the right hypothesis? And if it’s more so poor recall, is a possible explanation that a lot of the behaviors you look at are data-driven behaviors so there’s little cognitive processing to pick up on?
(By data-driven behaviors I mean correlation-driven, e.g., a possible explanation for the example in Sam’s comment is most lists of TV shows in the model’s training data start with the best one)
I think it’s more common to miss the right hypothesis. The most common issue is that tool outputs almost never contain causal relationships, like “feature X causes behavior Y”.
This question about “is there enough cognitive processing to pick up on?” is pretty common. I think there are two underlying questions here:
Is there little cognitive processing in our examples?
Do we care about behaviors with little cognitive processing?
For 1, I think it’s hard to say. I created an LLM-judge based “coherence filter” that filtered out around 60% of the initial examples to obtain datapoints that I intuitively thought had a coherent enough mechanism that may require cognitive processing. I hoped that we could find some uplift with these filtered examples. You can view randomly sampled investigations here to get a sense for what survives the filter.
People’s intuitions for what requires processing vary. With all four of the below examples (from Figure 2 in our paper), I think it’s plausible that the model may or may not be using cognitive processing when performing the behaviors, and we don’t have a ground truth that can inform us. For example, maybe there’s a correlation-driven behavior to take reckless and unsafe actions when imitating a bold persona.
For 2, I think we still care about behaviors that require little cognitive processing, as e.g. models may take harmful actions when they are unaware of the underlying causes. While this isn’t as concerning as something like deliberate scheming, I think we would like to understand it. I would also be much happier with a tool that can explain the behaviors I selected versus a tool that cannot, and I think we would want to avoid only evaluating what tools currently succeed at. I agree that we don’t care about all behaviors, and some such as the “picking from a list” behavior may not be that interesting.
I’ll also note that our result may not be that surprising based on prior results. While we see e.g. many NLA verbalizations of eval awareness, it’s (AFAIK) very rare to see NLA verbalizations that involve causality like “I’m behaving safely only because I’m in an eval”. This is similar to our finding that tool outputs almost never mention causal relationships.