Diffusion LLMs and autoregressive LLMs seem like basically the same technology to me.
Agreed. I highly recommend this blog post (https://sander.ai/2024/09/02/spectral-autoregression.html) for concretely understanding why autoregressive and diffusion models are so similar, despite seeming so different.
Diffusion LLMs and autoregressive LLMs seem like basically the same technology to me.
Agreed. I highly recommend this blog post (https://sander.ai/2024/09/02/spectral-autoregression.html) for concretely understanding why autoregressive and diffusion models are so similar, despite seeming so different.