Fwiw, dropout hasn’t fallen out of favor very much.
I think dropout makes nets less interpretable (wrt. naive interp strats). This is based on my recollection, I forget what exact experiments we have and haven’t run.
OK, good to know. I had a cached belief that it had declined in popularity which probably exaggerated the extent.
Fwiw, dropout hasn’t fallen out of favor very much.
I think dropout makes nets less interpretable (wrt. naive interp strats). This is based on my recollection, I forget what exact experiments we have and haven’t run.
OK, good to know. I had a cached belief that it had declined in popularity which probably exaggerated the extent.