On resources: I agree that Karpathy’s writeups are very helpful. Other useful resources:
Chris Olah’s blog and his new interactive journal Distil are good explanations of why deep-learning tricks work.
Michael Nielsen’s book on neural networks and deep learning is also good for conceptual understanding.
The Keras documentation is good. Armed with somebody else’s Github code as a template, the Keras documentation, and a paper suggesting a variation on that project, you can try reimplementing your own version of the code (and training it on your own data) and that’s probably the learning experience you want.
You’re going to read a lot of papers, and then look up terms you don’t understand and find papers or blog posts that explain them. It helps to have a system for saving and taking notes on useful papers and links. The field just isn’t that consolidated yet, so a lot of good info is spread out this way and not captured in courses or textbooks.
I also recommend becoming familiar enough with Python and the basic libraries (panda, numpy, sklearn, matplotlib) so that data cleaning and basic exploratory data analysis is easy to do. Find a development environment that works for you. (I am happiest with Anaconda and doing everything in Jupyter notebooks.) If you’re just now transitioning to coding regularly, you should be aware that initial setup/installation/environment woes happen to everyone, they are not a sign that you are unusually “bad with computers”, and they will go away once you have found an effective work setup. If you haven’t already done so, it’s good to get an up-to-date laptop, and maybe set up access to higher-performance computers like an AWS account. (I spent a year blaming myself for terrible code performance before a friend diagnosed the problem: my ancient laptop.)
I’ve been doing machine learning for about 2.5 years now and using python for longer than that and I’m also a big jupyter notebook fan. I still have a bit of trouble reading other people’s code almost always, what I’m really hoping is that I’ll be able to dive into the keras documentation more as this undertaking moves along.
I’ll check out the blogs also, thanks for the references!
Hi! Math PhD-turned-data-scientist here.
On resources: I agree that Karpathy’s writeups are very helpful. Other useful resources:
Chris Olah’s blog and his new interactive journal Distil are good explanations of why deep-learning tricks work.
Michael Nielsen’s book on neural networks and deep learning is also good for conceptual understanding.
The Keras documentation is good. Armed with somebody else’s Github code as a template, the Keras documentation, and a paper suggesting a variation on that project, you can try reimplementing your own version of the code (and training it on your own data) and that’s probably the learning experience you want.
You’re going to read a lot of papers, and then look up terms you don’t understand and find papers or blog posts that explain them. It helps to have a system for saving and taking notes on useful papers and links. The field just isn’t that consolidated yet, so a lot of good info is spread out this way and not captured in courses or textbooks.
I also recommend becoming familiar enough with Python and the basic libraries (panda, numpy, sklearn, matplotlib) so that data cleaning and basic exploratory data analysis is easy to do. Find a development environment that works for you. (I am happiest with Anaconda and doing everything in Jupyter notebooks.) If you’re just now transitioning to coding regularly, you should be aware that initial setup/installation/environment woes happen to everyone, they are not a sign that you are unusually “bad with computers”, and they will go away once you have found an effective work setup. If you haven’t already done so, it’s good to get an up-to-date laptop, and maybe set up access to higher-performance computers like an AWS account. (I spent a year blaming myself for terrible code performance before a friend diagnosed the problem: my ancient laptop.)
I’ve been doing machine learning for about 2.5 years now and using python for longer than that and I’m also a big jupyter notebook fan. I still have a bit of trouble reading other people’s code almost always, what I’m really hoping is that I’ll be able to dive into the keras documentation more as this undertaking moves along.
I’ll check out the blogs also, thanks for the references!