Vanilla SGD is perfectly equivalent to a genetic search with an infinitesimally small mutation size and infinite samples per generation (I could make a proof here but won’t unless someone is interested in it)
Oh, I actually did that a year or so ago
Oh, I actually did that a year or so ago