The Unsloth guys claim Torch.compile over-powers all manual kernel coding at this part of their recent presentation: https://youtu.be/uIiA6DquRiE?t=6277 (1:44:30+) . I cannot verify their claim, but their releases( including kernels[0] for speed-up) are quite popular, I am sure.
There is some newer, recent evidence in Kimi K3′s system card: https://www.kimi.com/blog/kimi-k3. It seems to outperform kernels and compilers. Probably something like “hardware-software co-design” is still a human thing, but all else probably not.
The Unsloth guys claim Torch.compile over-powers all manual kernel coding at this part of their recent presentation: https://youtu.be/uIiA6DquRiE?t=6277 (1:44:30+) . I cannot verify their claim, but their releases( including kernels[0] for speed-up) are quite popular, I am sure.
0: https://github.com/unslothai/unsloth/tree/main/unsloth/kernels
There is some newer, recent evidence in Kimi K3′s system card: https://www.kimi.com/blog/kimi-k3. It seems to outperform kernels and compilers. Probably something like “hardware-software co-design” is still a human thing, but all else probably not.