Yes I see that you agree that RSI doesn’t lead to a sudden takeoff. I also see that you can make lots of relevant data post training with RLVR. That would sure help, but I am not sure about helping the AI make the kind of blue-sky conceptual breakthroughs that geniuses make occasionally. For some things there simply isn’t enough data even in principle that could be created. For example, “create a new branch of mathematics”—I don’t think LLM’s could ever do.
You are right that inference speed is >> than model release, and that ties to something related and perhaps unavoidably linked to data efficiency. Biology also learns with far less FLOPs than ANN as well as needing far less data. I am assuming that any data efficient architecture will also require far less training FLOPS, also solving the slow model release problem.
Yes I see that you agree that RSI doesn’t lead to a sudden takeoff. I also see that you can make lots of relevant data post training with RLVR. That would sure help, but I am not sure about helping the AI make the kind of blue-sky conceptual breakthroughs that geniuses make occasionally. For some things there simply isn’t enough data even in principle that could be created. For example, “create a new branch of mathematics”—I don’t think LLM’s could ever do.
You are right that inference speed is >> than model release, and that ties to something related and perhaps unavoidably linked to data efficiency. Biology also learns with far less FLOPs than ANN as well as needing far less data. I am assuming that any data efficient architecture will also require far less training FLOPS, also solving the slow model release problem.