I’ve also been tracking the development of these kinds of techniques for a year, and they have consistently been showing surprising improvements. The test time training people have been publishing for a while, and the ceiling seems nowhere in sight.
To be honest a complete recounting of why I believe this is somewhat beyond a comment’s length. I also feel very strongly that progress is accelerating, and that if I don’t say something now there will not be much time to react. Hence the post.
I think that the most legible arguments are already in the post. The only thing I would make clearer is that the human brain is an existence proof that such highly efficient continuous learning algorithms exist, and therefore I see the development of these models as not particularly surprising. Stephen Brynes has some similar intuitions in his Foom and Doom series.
In which case, the best thing is probably for us to wait and see if the predictions come to pass, if that’s okay with you. I might also be afk for a bit so might not be able to immediately reply to any further comments.
I’ve also been tracking the development of these kinds of techniques for a year, and they have consistently been showing surprising improvements. The test time training people have been publishing for a while, and the ceiling seems nowhere in sight.
To be honest a complete recounting of why I believe this is somewhat beyond a comment’s length. I also feel very strongly that progress is accelerating, and that if I don’t say something now there will not be much time to react. Hence the post.
Perhaps it’s beyond the length of a comment, but why not recount it in the post?
I think that the most legible arguments are already in the post. The only thing I would make clearer is that the human brain is an existence proof that such highly efficient continuous learning algorithms exist, and therefore I see the development of these models as not particularly surprising. Stephen Brynes has some similar intuitions in his Foom and Doom series.
I read (some of) that series and disagreed with his assessment on the same grounds.
In which case, the best thing is probably for us to wait and see if the predictions come to pass, if that’s okay with you. I might also be afk for a bit so might not be able to immediately reply to any further comments.