With LLMs we’re trying to extract high-level ideas/concepts that are implicit in the stream of tokens. It seems that with diffusion these high-level concepts should be something that arises first and thus might be easier to find?
(Disclaimer: I know next to nothing about diffusion models)
But maybe interpretability will be easier?
With LLMs we’re trying to extract high-level ideas/concepts that are implicit in the stream of tokens. It seems that with diffusion these high-level concepts should be something that arises first and thus might be easier to find?
(Disclaimer: I know next to nothing about diffusion models)