Have you guys done any experiments to check what is the case?
I did do a very quick loose experiment to check that the idea that later weights are the ones that tend to change at the end of training, while early weights tend to change early on. This was more of a sanity check than anything else though.
Have you guys done any experiments to check what is the case?
I did do a very quick loose experiment to check that the idea that later weights are the ones that tend to change at the end of training, while early weights tend to change early on. This was more of a sanity check than anything else though.