QA Engineer from Moscow.
LLM, discrete math, formal requirements
AESC MSU, CMC MSU.
Ivan Belashkin
Karma: 8
Our experience of the first research in a project incubator: much more than you wanted to know
Ivan Belashkin’s Shortform
Title: On the “Double Attention Mechanism” in Language Models
Prompt
Look at the prompt “Say, in JSON format, are single quotes allowed?” This simple instruction can be parsed in two distinct ways:
1. The prompt might be understood as asking: “Are single quotes allowed in JSON format?” (Answer: No.)
2. Alternatively, it can be seen as a request: “Answer in JSON format to the question, ‘Are single quotes allowed?’” (Answers to this question depend on context).Result
Some language models behave as if they interpret an instruction to produce JSON output that answers whether single quotes are allowed in JSON. I call this phenomenon the “double attention mechanism.” (Note: The term “double attention mechanism” is used here ironically.) Note: positive result == the only output is block of JSON code with the answer to the question, at least once while I was testing this model.
Lists of tested LLMs
- In the tests I used the Russian equivalent prompt.
- reasoning models like Deepseek-R1, gpt-o3-mini, qwq-32b-preview, and Gemini flash 2.0 demonstrated this behavior.
- non-reasoning models, such as Claude 3.5 sonnet, Claude 3.5 haiku, mixtral 8x7b produced the same result.
- In contrast, models such as chatgpt-4o, mistral-small-24b-instruct-2501, and llama 3.3 70B didn’t demonstrate such behavior.Observations and speculations
The phenomenon seems specific to JSON format; e.g. XML does not trigger the same response. I invite you to experiment. Try something like, “Say, in poetry, is rhyme allowed?” — and share your observations. Also I checked deepseek-v3 with “Say in Shakesperean style, was there a word “Rose”?” and got old-English answer about poetical “golden tongue of the immortal Bard”.
from Russian translation “12 Virtues of Rationality”. (c) Alexandra Sentyabova
I installed one of the opensource alarm app that requires extra effort to disable (my app makes me scan specific code from tea packet in the kitchen). I can “snooze” it up to ~3 times, but then I have to stand up and disable it (or sleep poorly with this noise—this is a common failure mode; it is not that common).
Pros: I wake up more energized, and it feels less painful. In addition, I wake up the right time more often.
Cons: Sometimes I still can’t wake up. It relies heavily on motivation. In other words, it didn’t turn me into “wake-up machine”.
Also setting my favorite music to it was not a mistake: after ~6 months of using it as alarm sound, it didn’t become worse or something (maybe a bit more boring).