And machine learning is a thing which we can’t seem to get very far with.
Standard vanilla supervised machine learning (e.g. backprop neural networks and SVMs) is not going anywhere fast, but deep learning is really a new thing under the sun.
but deep learning is really a new thing under the sun.
On the contrary, the idea of making deeper nets is nearly as old as ordinary 2-layer neural nets, successful implementations dates back to the late 90′s in the form of convolutional neural nets, and they had another burst of popularity in 2006.
Advances in hardware, data availability, heuristics about architecture and training, and large-scale corporate attention have allowed the current burst of rapid progress.
This is both heartening, because the foundations of its success are deep, and tempering, because the limitations that have held it back before could resurface to some degree.
Standard vanilla supervised machine learning (e.g. backprop neural networks and SVMs) is not going anywhere fast, but deep learning is really a new thing under the sun.
On the contrary, the idea of making deeper nets is nearly as old as ordinary 2-layer neural nets, successful implementations dates back to the late 90′s in the form of convolutional neural nets, and they had another burst of popularity in 2006.
Advances in hardware, data availability, heuristics about architecture and training, and large-scale corporate attention have allowed the current burst of rapid progress.
This is both heartening, because the foundations of its success are deep, and tempering, because the limitations that have held it back before could resurface to some degree.