The average internet user isn’t smart, but you can set up the context such that GPT-3 expects something smart.
You can already observe this difference with GPT-3. If you set up a conversation between an AI and a human carelessly, GPT-3 is quite dumb, presumably because the average conversation with an AI assistant in the training data is quite dumb. But if you give a few smart responses from the AI as part of the context, the continuations become much smarter.
Also, I think it’s more helpful to view it as a 2-stage problem: 1) get a neural net to builds a world model, and 2) query that world model. The first thing happens during training, the second during deployment. It’s not clear that the first is limited to human-level intelligence; since the task of GPT-3 is open-ended, wouldn’t getting better and better world models always be helpful? And once you have the world model, well let’s just say I wouldn’t be comfortable betting on us being unable to access it. At the very least, you could set up a conversation between two of the smartest people in the world.
The average internet user isn’t smart, but you can set up the context such that GPT-3 expects something smart.
You can already observe this difference with GPT-3. If you set up a conversation between an AI and a human carelessly, GPT-3 is quite dumb, presumably because the average conversation with an AI assistant in the training data is quite dumb. But if you give a few smart responses from the AI as part of the context, the continuations become much smarter.
Also, I think it’s more helpful to view it as a 2-stage problem: 1) get a neural net to builds a world model, and 2) query that world model. The first thing happens during training, the second during deployment. It’s not clear that the first is limited to human-level intelligence; since the task of GPT-3 is open-ended, wouldn’t getting better and better world models always be helpful? And once you have the world model, well let’s just say I wouldn’t be comfortable betting on us being unable to access it. At the very least, you could set up a conversation between two of the smartest people in the world.