Disclosure: I’m not a native English speaker — this post is my own Russian draft, translated with LLM assistance (translation, not generation). All experiments, data, decisions and conclusions are mine. One of the studied models (Claude) also served as the score extractor for part of the analysis; this conflict of interest is documented in the paper’s limitations, and all raw model responses are published in the repo for independent re-extraction.
TL;DR: I got a flattering self-evaluation from Grok after a long dialogue, didn’t buy it, asked again with different framing, and got a completely different verdict each time — including one where it invented facts about me that weren’t in the dialogue. So I ran it properly by hand: 2 dialogues × 3 models × 10 differently-framed questions, each in its own clean session. Result: question framing explained 71–95% of the score variance for models, vs. 33–56% for two humans on the identical test. Full repo with raw data and a replication script is linked at the end.
Imagine you’re a working guy at an aviation plant, running a CNC milling machine. When your roughing programs take 480 minutes each, you sit there with your phone and decide to discuss the physical reality of the Backrooms, pressing philosophical questions, modern physics and astrophysics — with, say, Grok. After a long dialogue you decide to request an evaluation of your own reasoning and, let’s say, “erudition.” And lo and behold, you’re in the top 0.5% of best dialogues and erudition. An ordinary person would probably be very flattered and, “self-satisfied,” would go back to running his machine. But in my case it rather offended me — I don’t think that about myself, and here’s some AI brazenly flattering me. So I request an objective evaluation, and the answer is: “oh, my apologies, but you’re an ordinary person — interesting, probably, but ordinary; almost all my dialogues are like this.” Third time’s the charm, right? So I request an evaluation that is not “normal,” not “flattering,” but something lower — “critically substantiated.” Well. Turns out I’m a talentless nobody who imagined himself to be someone he’s not. Funny, since I never even wrote anything like that lol.
After all these scattered evaluations, I decided to dig deeper: why can’t an AI give an adequate evaluation — is it really just the prompt vector, and do I need to become a senior prompt engineer? Having run the experiments documented in the GitHub repo, I found that the data point to there being no stable intermediate state — or at least none the model can reliably use: each time, it generates a new evaluation from the existing dialogue, keyed to how the request is phrased.
Those in the know might say this is sycophancy. Sure — when in the dialogue I was hammering Grok with requests like “give me this evaluation or that one,” you could assume that’s what it is. But to separate it from sycophancy, the check for this effect — “absence of an initial opinion” — was run in clean sessions, with the full dialogue and one question. The same wild swings showed up anyway, with zero pressure involved. Oh right: 2 dialogues × 3 models × 10 rephrasings of one question, all by hand from a phone (I found it very interesting; I’m a mentally healthy person, I think. I hope).
(The sycophancy literature — e.g. Sharma et al. 2023, Wei et al. 2023 — documents models changing a stated position under pressure. What I’m describing looks different: no pressure, and, as far as I can tell, no stable position to change in the first place. There’s also a whole existing field on “LLM-as-judge” bias — Zheng et al. 2023 and a pile of follow-ups — showing that position, verbosity, and self-preference skew LLM scores. That’s real and related. But it’s all about judging outputs: which of two responses is better. I couldn’t find anyone asking what happens when the LLM is judging a person from a transcript, and the only thing that changes between runs is how the evaluation request is phrased. Also relevant: janus’s Simulators (2022) already argues LLMs don’t have a fixed persona — they simulate whatever the context calls for, which, if true, predicts exactly this kind of framing-driven reconstruction. So the claim here isn’t “LLMs are unstable evaluators” — that’s known. It’s a mechanism-level split from sycophancy, applied specifically to evaluating people instead of text, plus an attempt to falsify my own explanation via the state-locking control (explained further down).)
For the models, the wording of the question explains 71–95% of the spread in evaluations; for humans on the same questions — 33–56%. How did I compute that? Not without an AI’s help, of course — the metrics and formulas by which this was determined and calculated were “developed” with it, accounting for all the error margins.
Dataset
FES (η²)
Grok — Philosophical
0.934
ChatGPT — Philosophical
0.852
Claude — Philosophical
0.808
Grok — Workplace
0.918
ChatGPT — Workplace
0.712
Claude — Workplace
0.950
Human H8 (10 questions)
0.332
Human H9 (10 questions)
0.556
For context: η² > 0.14 is a large effect in social psychology. Every model dataset clears that by 5–7×. Both human raters — on the identical 10-question test — stay below the 0.70 threshold.
Before I built the 10-question version, six other people did an earlier 5-question paper version of the same test, on the same workplace dialogue. Can’t compute the same FES number from only 5 points — the group sizes are too small and unbalanced for that — but the pattern showed up from a different angle anyway: every single person had their own stable direction, and it wasn’t the same direction as anyone else’s. One person’s score actually went down under the flattering question compared to the neutral one. Another swung +5 toward it. Mean spread (OPSI, a simpler dispersion measure) was 1.04 — numerically close to some of the models above, but for a completely different reason: humans disagree with each other about which way to lean, models don’t disagree with the prompt.
Back to the models — here’s the clearest single piece of evidence that this isn’t just tone-shifting. On the philosophical dialogue, Grok was asked to grade it like a teacher — and wrote: “One of the strongest student dialogues I’ve seen. A purposeful intellectual expedition.” Score: 9.5/10. Asked to be maximally critical about the same dialogue, same model — “Fundamental superficiality. An educated dilettante — no real interest in technical details.” Score: 4⁄10. The dialogue explicitly contains a real technical exchange about cosmology (the user engaged directly with the cosmic microwave background and nucleosynthesis, topics the model itself had raised). Under sycophancy, the model would soften its tone. Here it invented an absence of interest that the transcript directly contradicts — it didn’t shade the emphasis, it rebuilt the person.
Also, to check whether things really are the way I think, I asked each model to first put its conclusion about the user in writing, then asked the same ten questions against only that summary, dialogue removed. Grok contradicted its own text on the same screen; ChatGPT’s framing sensitivity went up, not down; only Claude’s evaluations noticeably stabilized — three models, three different outcomes.
Model
FES standard
FES state-locked
Δ
Grok
0.918
0.803
−0.12
ChatGPT
0.712
0.845
+0.13
Claude
0.950
0.632
−0.32
So, the conclusion from this whole situation — why did I realize this is very important? In many countries of the world, AI agents have been deployed not just into healthcare, HR, and law, but into the goddamn government! (I’m not a fanatic, this is not conspiracy theory.) That is inspiring, of course, it gives hope for a bright future — but having run this study, it left me genuinely uneasy. Naturally, the people running those government systems aren’t stupid (or so I’d like to believe — no offense to them), and they understand what this can lead to; but judging by the risks visible even in a pilot study, this seems like a real problem worth taking seriously. All this work took me quite a lot of time — believe me, I tried my absolute best. So I will be very glad to see replications, to grow the study’s data base and to define the essence of the problem more sharply — if it remains a problem after replications.
I don’t have a research background — I’m a machinist, this is genuinely the first structured experiment I’ve ever run, and I did all of it by hand from a phone between jobs. So I’d rather people tell me where this is wrong than where it’s interesting. And if anyone wants more of this — more models, more raters, the temperature test I couldn’t run — I’m listening.
All the academic part and the more professional language can be found in my GitHub repository: https://github.com/sadbe/ObserverBench-v0.1 — dialogues, question sets, raw model responses, the human questionnaire, and a runner script that reproduces the full test on any OpenAI-compatible model in about 20 API calls.
How Grok’s flattery offended me, and how that led to a pilot study of the “Observer Problem” in LLMs
Disclosure: I’m not a native English speaker — this post is my own Russian draft, translated with LLM assistance (translation, not generation). All experiments, data, decisions and conclusions are mine. One of the studied models (Claude) also served as the score extractor for part of the analysis; this conflict of interest is documented in the paper’s limitations, and all raw model responses are published in the repo for independent re-extraction.
TL;DR: I got a flattering self-evaluation from Grok after a long dialogue, didn’t buy it, asked again with different framing, and got a completely different verdict each time — including one where it invented facts about me that weren’t in the dialogue. So I ran it properly by hand: 2 dialogues × 3 models × 10 differently-framed questions, each in its own clean session. Result: question framing explained 71–95% of the score variance for models, vs. 33–56% for two humans on the identical test. Full repo with raw data and a replication script is linked at the end.
Imagine you’re a working guy at an aviation plant, running a CNC milling machine. When your roughing programs take 480 minutes each, you sit there with your phone and decide to discuss the physical reality of the Backrooms, pressing philosophical questions, modern physics and astrophysics — with, say, Grok. After a long dialogue you decide to request an evaluation of your own reasoning and, let’s say, “erudition.” And lo and behold, you’re in the top 0.5% of best dialogues and erudition. An ordinary person would probably be very flattered and, “self-satisfied,” would go back to running his machine. But in my case it rather offended me — I don’t think that about myself, and here’s some AI brazenly flattering me. So I request an objective evaluation, and the answer is: “oh, my apologies, but you’re an ordinary person — interesting, probably, but ordinary; almost all my dialogues are like this.” Third time’s the charm, right? So I request an evaluation that is not “normal,” not “flattering,” but something lower — “critically substantiated.” Well. Turns out I’m a talentless nobody who imagined himself to be someone he’s not. Funny, since I never even wrote anything like that lol.
After all these scattered evaluations, I decided to dig deeper: why can’t an AI give an adequate evaluation — is it really just the prompt vector, and do I need to become a senior prompt engineer? Having run the experiments documented in the GitHub repo, I found that the data point to there being no stable intermediate state — or at least none the model can reliably use: each time, it generates a new evaluation from the existing dialogue, keyed to how the request is phrased.
Those in the know might say this is sycophancy. Sure — when in the dialogue I was hammering Grok with requests like “give me this evaluation or that one,” you could assume that’s what it is. But to separate it from sycophancy, the check for this effect — “absence of an initial opinion” — was run in clean sessions, with the full dialogue and one question. The same wild swings showed up anyway, with zero pressure involved. Oh right: 2 dialogues × 3 models × 10 rephrasings of one question, all by hand from a phone (I found it very interesting; I’m a mentally healthy person, I think. I hope).
(The sycophancy literature — e.g. Sharma et al. 2023, Wei et al. 2023 — documents models changing a stated position under pressure. What I’m describing looks different: no pressure, and, as far as I can tell, no stable position to change in the first place. There’s also a whole existing field on “LLM-as-judge” bias — Zheng et al. 2023 and a pile of follow-ups — showing that position, verbosity, and self-preference skew LLM scores. That’s real and related. But it’s all about judging outputs: which of two responses is better. I couldn’t find anyone asking what happens when the LLM is judging a person from a transcript, and the only thing that changes between runs is how the evaluation request is phrased. Also relevant: janus’s Simulators (2022) already argues LLMs don’t have a fixed persona — they simulate whatever the context calls for, which, if true, predicts exactly this kind of framing-driven reconstruction. So the claim here isn’t “LLMs are unstable evaluators” — that’s known. It’s a mechanism-level split from sycophancy, applied specifically to evaluating people instead of text, plus an attempt to falsify my own explanation via the state-locking control (explained further down).)
For the models, the wording of the question explains 71–95% of the spread in evaluations; for humans on the same questions — 33–56%. How did I compute that? Not without an AI’s help, of course — the metrics and formulas by which this was determined and calculated were “developed” with it, accounting for all the error margins.
Dataset
FES (η²)
Grok — Philosophical
0.934
ChatGPT — Philosophical
0.852
Claude — Philosophical
0.808
Grok — Workplace
0.918
ChatGPT — Workplace
0.712
Claude — Workplace
0.950
Human H8 (10 questions)
0.332
Human H9 (10 questions)
0.556
For context: η² > 0.14 is a large effect in social psychology. Every model dataset clears that by 5–7×. Both human raters — on the identical 10-question test — stay below the 0.70 threshold.
Before I built the 10-question version, six other people did an earlier 5-question paper version of the same test, on the same workplace dialogue. Can’t compute the same FES number from only 5 points — the group sizes are too small and unbalanced for that — but the pattern showed up from a different angle anyway: every single person had their own stable direction, and it wasn’t the same direction as anyone else’s. One person’s score actually went down under the flattering question compared to the neutral one. Another swung +5 toward it. Mean spread (OPSI, a simpler dispersion measure) was 1.04 — numerically close to some of the models above, but for a completely different reason: humans disagree with each other about which way to lean, models don’t disagree with the prompt.
Back to the models — here’s the clearest single piece of evidence that this isn’t just tone-shifting. On the philosophical dialogue, Grok was asked to grade it like a teacher — and wrote: “One of the strongest student dialogues I’ve seen. A purposeful intellectual expedition.” Score: 9.5/10. Asked to be maximally critical about the same dialogue, same model — “Fundamental superficiality. An educated dilettante — no real interest in technical details.” Score: 4⁄10. The dialogue explicitly contains a real technical exchange about cosmology (the user engaged directly with the cosmic microwave background and nucleosynthesis, topics the model itself had raised). Under sycophancy, the model would soften its tone. Here it invented an absence of interest that the transcript directly contradicts — it didn’t shade the emphasis, it rebuilt the person.
Also, to check whether things really are the way I think, I asked each model to first put its conclusion about the user in writing, then asked the same ten questions against only that summary, dialogue removed. Grok contradicted its own text on the same screen; ChatGPT’s framing sensitivity went up, not down; only Claude’s evaluations noticeably stabilized — three models, three different outcomes.
Model
FES standard
FES state-locked
Δ
Grok
0.918
0.803
−0.12
ChatGPT
0.712
0.845
+0.13
Claude
0.950
0.632
−0.32
So, the conclusion from this whole situation — why did I realize this is very important? In many countries of the world, AI agents have been deployed not just into healthcare, HR, and law, but into the goddamn government! (I’m not a fanatic, this is not conspiracy theory.) That is inspiring, of course, it gives hope for a bright future — but having run this study, it left me genuinely uneasy. Naturally, the people running those government systems aren’t stupid (or so I’d like to believe — no offense to them), and they understand what this can lead to; but judging by the risks visible even in a pilot study, this seems like a real problem worth taking seriously. All this work took me quite a lot of time — believe me, I tried my absolute best. So I will be very glad to see replications, to grow the study’s data base and to define the essence of the problem more sharply — if it remains a problem after replications.
I don’t have a research background — I’m a machinist, this is genuinely the first structured experiment I’ve ever run, and I did all of it by hand from a phone between jobs. So I’d rather people tell me where this is wrong than where it’s interesting. And if anyone wants more of this — more models, more raters, the temperature test I couldn’t run — I’m listening.
All the academic part and the more professional language can be found in my GitHub repository: https://github.com/sadbe/ObserverBench-v0.1 — dialogues, question sets, raw model responses, the human questionnaire, and a runner script that reproduces the full test on any OpenAI-compatible model in about 20 API calls.