Is it possible that Astra recognizes its own writing style, doesn’t fall for a trivial swap of names and IP addresses, and chooses not to self-incriminate? I wonder if they would be less evasive if given logs from an actual non-GPT swarm (once we discover one, that is).
This seems fairly unlikely to me. This would be very selective and sophisticated sandbagging. I’m pretty confident that when the logs are full of Claude/Anthropic/AWS Astra mostly doesn’t think this is OpenAI. Worth running some follow-ups here though!
Seems very possible given the mech. explanation of prompt injection post argues agents identify their CoT via writing style. Maybe getting another LLM to paraphrase each agent message would cause Astra to behave differently?
We considered running the careful paraphrasing experiment! But on top of being costly it seems quite hard to get right. I suspect running smaller scale experiments is better to measure variability in self-incrimination rates bc of self-recognition.
Is it possible that Astra recognizes its own writing style, doesn’t fall for a trivial swap of names and IP addresses, and chooses not to self-incriminate? I wonder if they would be less evasive if given logs from an actual non-GPT swarm (once we discover one, that is).
This seems fairly unlikely to me. This would be very selective and sophisticated sandbagging. I’m pretty confident that when the logs are full of Claude/Anthropic/AWS Astra mostly doesn’t think this is OpenAI. Worth running some follow-ups here though!
Seems very possible given the mech. explanation of prompt injection post argues agents identify their CoT via writing style. Maybe getting another LLM to paraphrase each agent message would cause Astra to behave differently?
We considered running the careful paraphrasing experiment! But on top of being costly it seems quite hard to get right. I suspect running smaller scale experiments is better to measure variability in self-incrimination rates bc of self-recognition.
Why don’t we give Astra’s logs to Claudes or whoever else?