Due to the need to iterate the vs until convergence, the predictive coding network had roughly a 100x greater computational cost than the backprop network.
The paper claims that predictive coding takes more compute. I agree that predictive coding ought to be more parallelizable. If you are using a GPU then backpropagation is already sufficiently parallelizable. However, it may be that neuromorphic hardware could parallelize better than a GPU, thus producing an increase in compute power that outstrips the 100x greater computational cost of the algorithm itself.
Kind of. Neuromorphics don’t buy you too much benefit for generic feedforward networks, but they dramatically reduce the expenses of convergence. Since the 100x in this paper derives from iterating until the network converges, a neuromorphics implementation (say on Loihi) would directly eliminate that cost.
The paper claims that predictive coding takes more compute. I agree that predictive coding ought to be more parallelizable. If you are using a GPU then backpropagation is already sufficiently parallelizable. However, it may be that neuromorphic hardware could parallelize better than a GPU, thus producing an increase in compute power that outstrips the 100x greater computational cost of the algorithm itself.
Kind of. Neuromorphics don’t buy you too much benefit for generic feedforward networks, but they dramatically reduce the expenses of convergence. Since the 100x in this paper derives from iterating until the network converges, a neuromorphics implementation (say on Loihi) would directly eliminate that cost.