gwern comments on Memory bandwidth constraints imply economies of scale in AI inference

[gwern]

They appear to have launched ‘Sohu’, for LLaMA-3-70b: https://​​www.etched.com/​​announcing-etched