Here is my best attempt at working out my thoughts on this, but I noticed I reached some confusion at various points. I figured I’d post it anyway in case it either actually makes sense or people have thoughts they feel like sharing that might help my confusion.
Edit: The article is now deprecated. Thanks for everyone commenting here for helping me understand the different definitions of optimizer. I do suspect my misunderstanding of Nate’s point might mirror why there is relatively common pushback against his claim? But maybe I’m typical minding.
Here is my best attempt at working out my thoughts on this, but I noticed I reached some confusion at various points. I figured I’d post it anyway in case it either actually makes sense or people have thoughts they feel like sharing that might help my confusion.
Edit: The article is now deprecated. Thanks for everyone commenting here for helping me understand the different definitions of optimizer. I do suspect my misunderstanding of Nate’s point might mirror why there is relatively common pushback against his claim? But maybe I’m typical minding.