I might plot the CDF instead. That way you don’t need to smooth.
The surprising thing shown is the significant positive correlation between log loss of a network, and log loss of the reverse. This means if you take a pruned network that scores well, and reverse it, turning on all the nodes that were off, and turning off all the nodes that were on, the result is usually still a well scoring network.
I suspect that this is only because you have a single hidden layer.
I might plot the CDF instead. That way you don’t need to smooth.
Only by applying a very smoothing transformation, namely integration. I think its harder to see what is going on in CDF plots, because its easy to see a line falling by 5%, but hard to notice a line getting 5% less steep.
For example, which of these plots is easier to read
Or
Plotting the CDF has turned a very obvious massive spike into a slightly flatter section. One of these curves is from normally distributed data. You can tell at a glance which it is from the top plot. The bottom plot makes it less obvious.
Yep. Testing this on bigger networks is on my todo list.
I might plot the CDF instead. That way you don’t need to smooth.
I suspect that this is only because you have a single hidden layer.
Only by applying a very smoothing transformation, namely integration. I think its harder to see what is going on in CDF plots, because its easy to see a line falling by 5%, but hard to notice a line getting 5% less steep.
For example, which of these plots is easier to read
Or
Plotting the CDF has turned a very obvious massive spike into a slightly flatter section. One of these curves is from normally distributed data. You can tell at a glance which it is from the top plot. The bottom plot makes it less obvious.
Yep. Testing this on bigger networks is on my todo list.
Perhaps it’s just me—I find the latter substantially more informative than the former. (For instance, the tail behaviour is rather more visible.)
(Also, your scale is off on the latter chart. It should be between 0-1, by definition.)