This is a really good point. Even if you could train a neuralese model, it would rapidly accumulate errors during inference and go out of distribution.
This is already a problem with tokenized models, where one incorrect token forces the model to condition on that token, but for continuous models we’d expect basically every output to have some error.
This is a really good point. Even if you could train a neuralese model, it would rapidly accumulate errors during inference and go out of distribution.
This is already a problem with tokenized models, where one incorrect token forces the model to condition on that token, but for continuous models we’d expect basically every output to have some error.