I was just dragged through Demons for a book club, so I was amused to read this. At least it means the time I spent reading that wasn’t in vain.
There’s some stuff that feels a little bit weird here. The author says they left in early 2024 and then spent the “following months” reading Dostoevsky and writing this essay. Was the essay a bit older and only got put up? (Has to be relatively recently edited, if it was run through 4.5). Who are the editors alluded to at the very end? Is it supposed to be Tim Hwang? A little bit more transparency would be much appreciated (the disclaimer about Opus 4.5 being used for anonymization was only added on the 24th after some people had pointed out that it sounded rather AI-written.).
Another weirdness: why did Hwang put up another microsite about Demons that’s written by an anonymous author “still working in industry” that has clear LLM-writing patterns at basically the same time? https://shigalyovism.com/. Though this one is much less in-depth.
Can anyone with more experience in the frontier labs/the uniparty give a sanity check for whether this seems like it was written by someone who is who they say they are?
it seems plausible that the piece was written by someone who only has access to public writings. it has some confusions that seem unlikely but not completely inexplicable—for example, the assumption that EA is a major steering force in the uniparty (maybe the author is from Anthropic where this is more true); I also find the description of uniparty views to a bit too homogenous (maybe the author is trying to emphasize how small the apparent differences are compared to the space of all beliefs, or maybe they are an external spectator who is unaware of the details).
Yeah that EA-prevalence assumption also caused me to doubt that the author actually worked at an AI company, it was very dissonant with my experience at least.
I’m skeptical that the author is who they say they are. (I made a top level post critiquing Possessed Machines, I’m copying over the relevant part here.)
1. I think the author is being dishonest about how this piece was written.
There is a lot of AI in the writing of Possessed Machines. The bottom of the webpage states “To conceal stylistic identifiers of the authors, the above text is a sentence-for-sentence rewrite of an original hand-written composition processed via Claude Opus 4.5.” As I wrote in a comment:
Ah, this [statement] was not there when I read the piece (Jan 23). You can see an archived version here in which it doesn’t say that.
I don’t actually believe that this is how the document was made. A few reasons. First, I don’t think this is what a sentence-for-sentence rewrite looks like; I don’t think you get that much of the AI style that this piece has with that^. Second, the stories in the interlude are superrrrr AI-y, not just in sentence-by-sentence style but in other ways. Third, the chapter and part titles seem very AI generated...
The piece has 31 uses of “genuine”/“genuinely” in ~17000 words. One “genuine” every 550 words.
There’s some stuff that feels a little bit weird here. The author says they left in early 2024 and then spent the “following months” reading Dostoevsky and writing this essay. Was the essay a bit older and only got put up? (Has to be relatively recently edited, if it was run through 4.5). Who are the editors alluded to at the very end? Is it supposed to be Tim Hwang? A little bit more transparency would be much appreciated (the disclaimer about Opus 4.5 being used for anonymization was only added on the 24th after some people had pointed out that it sounded rather AI-written.).
Another weirdness: why did Hwang put up another microsite about Demons that’s written by an anonymous author “still working in industry” that has clear LLM-writing patterns at basically the same time? https://shigalyovism.com/. Though this one is much less in-depth.
At the bottom of the webpage in an “About the Author” box, we are told “Correspondence may be directed to the editors.” This is weird, because we don’t know who the editors are. Probably this was something that Claude added and the human author didn’t check.
There are some anomalies in the chapter numbering:
Part IV ends with Chapter 18; Part V begins with Chapter 21… [etc.]
3. This piece could have been written by someone who wasn’t an AI insider
If you’re immersed in 2025/2026 ~rationalist AI discourse, you would have the information to write Possessed Machines. That is, there’s no “inside information” in the piece. There is a lot of “I saw people at the lab do this [thing that I, a non-insider, already thought that people at the lab did]”. Leogao has made this same point: “it seems plausible that the piece was written by someone who only has access to public writings.”
I was just dragged through Demons for a book club, so I was amused to read this. At least it means the time I spent reading that wasn’t in vain.
There’s some stuff that feels a little bit weird here. The author says they left in early 2024 and then spent the “following months” reading Dostoevsky and writing this essay. Was the essay a bit older and only got put up? (Has to be relatively recently edited, if it was run through 4.5). Who are the editors alluded to at the very end? Is it supposed to be Tim Hwang? A little bit more transparency would be much appreciated (the disclaimer about Opus 4.5 being used for anonymization was only added on the 24th after some people had pointed out that it sounded rather AI-written.).
Another weirdness: why did Hwang put up another microsite about Demons that’s written by an anonymous author “still working in industry” that has clear LLM-writing patterns at basically the same time? https://shigalyovism.com/. Though this one is much less in-depth.
Can anyone with more experience in the frontier labs/the uniparty give a sanity check for whether this seems like it was written by someone who is who they say they are?
it seems plausible that the piece was written by someone who only has access to public writings. it has some confusions that seem unlikely but not completely inexplicable—for example, the assumption that EA is a major steering force in the uniparty (maybe the author is from Anthropic where this is more true); I also find the description of uniparty views to a bit too homogenous (maybe the author is trying to emphasize how small the apparent differences are compared to the space of all beliefs, or maybe they are an external spectator who is unaware of the details).
Yeah that EA-prevalence assumption also caused me to doubt that the author actually worked at an AI company, it was very dissonant with my experience at least.
I’m skeptical that the author is who they say they are. (I made a top level post critiquing Possessed Machines, I’m copying over the relevant part here.)
1. I think the author is being dishonest about how this piece was written.
There is a lot of AI in the writing of Possessed Machines. The bottom of the webpage states “To conceal stylistic identifiers of the authors, the above text is a sentence-for-sentence rewrite of an original hand-written composition processed via Claude Opus 4.5.” As I wrote in a comment:
See also...
2. Fishiness
From kaiwilliams:
At the bottom of the webpage in an “About the Author” box, we are told “Correspondence may be directed to the editors.” This is weird, because we don’t know who the editors are. Probably this was something that Claude added and the human author didn’t check.
Richard_Kennaway points out:
3. This piece could have been written by someone who wasn’t an AI insider
If you’re immersed in 2025/2026 ~rationalist AI discourse, you would have the information to write Possessed Machines. That is, there’s no “inside information” in the piece. There is a lot of “I saw people at the lab do this [thing that I, a non-insider, already thought that people at the lab did]”. Leogao has made this same point: “it seems plausible that the piece was written by someone who only has access to public writings.”