It may be the case that such models are the only generally intelligent systems, but systems trained in such a way do not exhibit strong orthogonality.
And it does not follow that we can in full generality, point such systems at arbitrary other targets.
I disagree with the first claim, primarily due to the only part. I do believe that language modeling might be enough, though I also think certain other paths are enough, like RL. It’s just that SSL took off first here.
I disagree with the first claim, primarily due to the only part. I do believe that language modeling might be enough, though I also think certain other paths are enough, like RL. It’s just that SSL took off first here.
Well, the argument holds if there’s a meaningful time period in which all general AI systems were trained via self supervised learning.