Cool work. What kinds of situations trigger CoT evaluation awareness mentions in simulated deployment? Does this overlap heavily with the distribution of scenarios that are interesting from an alignment or safety perspective? Where those distributions come apart is quite interesting to me.
One such situation is when tools throw errors or other unexpected things happen (e.g. a file was edited since the model looked at it last). This seems happens in both real and simulated deployments and IMO is the reason eval awareness is actually lower in simulation than in real deployment (when not using tool simulation).
Cool work. What kinds of situations trigger CoT evaluation awareness mentions in simulated deployment? Does this overlap heavily with the distribution of scenarios that are interesting from an alignment or safety perspective? Where those distributions come apart is quite interesting to me.
One such situation is when tools throw errors or other unexpected things happen (e.g. a file was edited since the model looked at it last). This seems happens in both real and simulated deployments and IMO is the reason eval awareness is actually lower in simulation than in real deployment (when not using tool simulation).