Hm, does the number of non-trivial dimensions follow a smooth curve? From the histogram figure it clearly grows as net size increases, but I can’t see how fast it’s decelerating.
Does the same pattern obtain on more complicated data? What if you then add regularization?
I think it was smooth-ish, although we didn’t plot it. It was decelerating really slowly, but we also didn’t quantify that. We could send you the notebook (bit of a mess)?
I really wanted to extend the experiment to more complicated data, and bigger networks with more layers, but we didn’t have time. I plan to do this after the MATS program ends in late september if someone else hasn’t done it first.
Hm, does the number of non-trivial dimensions follow a smooth curve? From the histogram figure it clearly grows as net size increases, but I can’t see how fast it’s decelerating.
Does the same pattern obtain on more complicated data? What if you then add regularization?
I think it was smooth-ish, although we didn’t plot it. It was decelerating really slowly, but we also didn’t quantify that. We could send you the notebook (bit of a mess)?
I really wanted to extend the experiment to more complicated data, and bigger networks with more layers, but we didn’t have time. I plan to do this after the MATS program ends in late september if someone else hasn’t done it first.