So ZIP achieves high compression for “kinda boring reasons”, in the sense that we already knew all about that compressibillity but just don’t leverage it in day-to-day operations because our float arithmetic hardware uses IEEE.
Could this be verified? Like, estimate the compression ratio under the assumption that it’s all about compressing IEEE floats, then run the ZIP and compare the actual result to the expectation?
Easiest test would be to zip some trained net params, and also zip some randomly initialized standard normals of the same shape as the net params (including e.g. parameter names if those are in the net params file), and see if they get about the same compression.
Could this be verified? Like, estimate the compression ratio under the assumption that it’s all about compressing IEEE floats, then run the ZIP and compare the actual result to the expectation?
Easiest test would be to zip some trained net params, and also zip some randomly initialized standard normals of the same shape as the net params (including e.g. parameter names if those are in the net params file), and see if they get about the same compression.