More and more, I’m updating towards that a non-trivial (perhaps almost all) of GPT-4′s capability is downstream of being able to store the ~entire internet in it’s mind, which trivializes the problem.
That doesn’t mean it’s a stochastic parrot, or not intelligent, but it does have implications for a potential future, if AI pretraining was only scaled:
More and more, I’m updating towards that a non-trivial (perhaps almost all) of GPT-4′s capability is downstream of being able to store the ~entire internet in it’s mind, which trivializes the problem.
That doesn’t mean it’s a stochastic parrot, or not intelligent, but it does have implications for a potential future, if AI pretraining was only scaled:
https://x.com/DimitrisPapail/status/1888325914603516214
https://www.lesswrong.com/posts/i7JSL5awGFcSRhyGF/shortform-2#s6xSyKkDLgpcD9wPw