I kinda think that continual learning is not that hard as a science problem, and its adoption is being slowed more by engineering constraints than anything else. (At least for the more simple forms of continual learning.) Basically, I would not be surprised at all if aicos are already working internally with non-toy models with some kind of continual learning mechanism. But updating such a model on a given set of information is likely to be much more expensive per token than context-reading or generation.
I kinda think that continual learning is not that hard as a science problem, and its adoption is being slowed more by engineering constraints than anything else. (At least for the more simple forms of continual learning.) Basically, I would not be surprised at all if aicos are already working internally with non-toy models with some kind of continual learning mechanism. But updating such a model on a given set of information is likely to be much more expensive per token than context-reading or generation.