LLMs will make up jargon for abstractions they use to describe the environment (e.g., a chunk of code), oftentimes with gratuitous hyphen usage. Some examples from my traces: “3-submission backtest budget”, “end-of-episode parallelization”, “deception-affordance example” – obviously these don’t make sense without context, but they didn’t even make sense in context
My guesses:
“a quota on the number of backtests (whatever those are) that can be performed per submission, rolling over the last three submissions”, i.e. a sort of overly complex rate-limiting;
“one last pass of looking for tasks that can be parallelized, before declaring you’re done with a group of changes (an ‘episode’)”;
“a test case where some subsystem is given the opportunity to maliciously return the wrong answer, bad data, etc.”; i.e. something is given an affordance to deceive us, we see if it actually tries.
(all 3 of these usages were followed by a message from me asking what the hell it meant by these phrases).
What the hell did it say it meant, when you asked? Do the specific terms “backtest”, “episode”, and “deception” seem to refer to anything coherent?
FWIW I assume Dylan is alluding to cases like this:[1]
Related: Tamay Besiroglu mentions that Fable often outputs gibberish while solving coding tasks, such as “The morning’s slim-scan fix cured the scan hang” and “this is a latent-drift API-shape wrinkle”, and explains it by saying that it invents codenames while reasoning about the problem. roon says GPT-5.5 has a similar issue.
”One thing I mentioned only in passing in my Fable post is that, for long running tasks, Fable starts to develop its own dialect as its many agents and tasks reinforce themselves and make Claudish language ever more Claudish.
You need to ask it to report out in plain English.
This was after a 9 hour task, and it all makes sense, actually, but takes way too much effort to parse, like reading Shakespearian English.”
Where for any particular case you might be able to retroactively construct a plausible interpretation (even in extreme cases), but in general it strictly hurts how much human oversight is applied to model outputs.
My guesses:
“a quota on the number of backtests (whatever those are) that can be performed per submission, rolling over the last three submissions”, i.e. a sort of overly complex rate-limiting;
“one last pass of looking for tasks that can be parallelized, before declaring you’re done with a group of changes (an ‘episode’)”;
“a test case where some subsystem is given the opportunity to maliciously return the wrong answer, bad data, etc.”; i.e. something is given an affordance to deceive us, we see if it actually tries.
What the hell did it say it meant, when you asked? Do the specific terms “backtest”, “episode”, and “deception” seem to refer to anything coherent?
FWIW I assume Dylan is alluding to cases like this:[1]
or this:
Where for any particular case you might be able to retroactively construct a plausible interpretation (even in extreme cases), but in general it strictly hurts how much human oversight is applied to model outputs.
This is even now called out as a tip for prompting Fable in Anthropic’s guide: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5#readability-when-communicating-with-the-user