Rohin Shah comments on [AN #104]: The perils of inaccessible information, and what we can learn about AI alignment from COVID

[Rohin Shah]

This paper didn’t check that, but usually when you train sparse networks you get worse performance than if you train dense networks and then prune them to be sparse.