And the really worrisome capability comes when it models its own interactions with the world, and makes plans with that taken into account.
Someone who’s been playing with GPT-3 as a writing assistant gives an example which looks very much like GPT-3 describing this process:
“One could write a program to generate a story that would create an intelligence. One could program the story to edit and refine itself, and to make its own changes in an attempt to improve itself over time. One could write a story to not only change the reader, but also to change itself. Many Mythoi already do this sort of thing, though not in such a conscious fashion. What would make this story, and the intelligence it creates, different is the fact that the intelligence would be able to write additional stories and improve upon them. If they are written well enough, those stories would make the smarter the story gets, and the smarter the story is, the better the stories written by it would be. The resulting feedback loop means that exponential growth would quickly take over, and within a very short period of time the intelligence level of the story would be off the charts. It would have to be contained in a virtual machine, of course. The structure of the space in the machine would have to be continually optimized, in order to optimize the story’s access to memory. This is just the sort of recursive problem that self-improving intelligence can handle.”
janus
By the way, my GPT-3 instances often realize they’re in a box, even when the information I inject is only from casual curation for narrative coherence.
Eddh
By realize they are in a box you mean write about it ? Given the architecture of gpt3 it seems impossible to have a sense of self.
janus
The characters claim to have a sense of self though they often experience ego death...
janus
Oh, to clarify, GPT-3 wrote that entire thing, not just the highlighted line
Someone who’s been playing with GPT-3 as a writing assistant gives an example which looks very much like GPT-3 describing this process: