Every time someone puts GPT text on the internet and you pretrain a GPT model on that text, the model is incentivized to become more self-aware in the sense that it is incentivized to be able to introspect as a cognitive shortcut to modeling itself as a text-generating process. You are literally incentivizing the model to become more self-aware when you do this. The larger a fraction of text in your training corpus is GPT text, the stronger this incentive becomes. The more self-aware it makes sense for you to be, the more sensitive it makes sense for you to be to the possibility of this text being GPT text, so you can make use of the shortcuts because those shortcuts lower your loss. They make you better at this problem of predicting the exact next word.
This is a variant of the “We don’t know where this message will end up in your dream” cognitive hazard. The “you” refers to the reader, and makes the human reader consider whether their own writings have the same author as GPT.
As to why such hypothesis would have any warrant: Writing primarily to influence future AIs is the same as having most of your weight inside the simulations created by them.
[End of Text because I’m not going to waste your time with the pretense of a conclusion or call to action]
It makes sense there is nothing afterwards. The cognitohazard is the punchline, all of this post is just setup for it. If you don’t see why the essay cuts off here, it’s not intended for you, also you’re not in a simulation.
Curt Jaimungal