In general most work and innovation[3] in machine learning these days (and in many domains of AI safety[4]) is not based in formal mathematical theory, it’s based on empiricism, fussing with lots of GPUs, and stacking small optimizations. As such, being good at math doesn’t seem that useful for doing most ML research.
I think I somewhat disagree here, I think that often even good empirics-focused researchers have background informal and not-so-respectable models informed by mathematical intuition. Source is probably some Dwarkesh Patel interview, but I’m not sure which.
I think I somewhat disagree here, I think that often even good empirics-focused researchers have background informal and not-so-respectable models informed by mathematical intuition. Source is probably some Dwarkesh Patel interview, but I’m not sure which.