Are you sure that Google’s 2016 net uses graphic cards? I would think that they use their Tensor Flow ASICs. The switch from graphic cards to ASICs is part of what allowed them huge performance improvements in a short time frame. I don’t think that they will continue to improve much better than Moore’s law.
Are you sure that Google’s 2016 net uses graphic cards? I would think that they use their Tensor Flow ASICs. The switch from graphic cards to ASICs is part of what allowed them huge performance improvements in a short time frame. I don’t think that they will continue to improve much better than Moore’s law.
“We trained our models using TensorFlow (Abadi et al., 2016) on clusters containing 16-32 Tesla K40 GPUs” https://arxiv.org/pdf/1701.06538.pdf
So they did it before they implement Tensorflow hardware or didn’t use it.
Current price of such Tesla cluster is around 50-100 K USD