It’s possible that you (jacobjacob) and jbash are actually in agreement that (part of) the brain does something that is not literally backprop but “relevantly similar” to backprop—but you’re emphasizing the “relevantly similar” part and jbash is emphasizing the “not literally” part?
I think that’s likely correct. What I mean is that it’s not running all the way to the end of a network, computing a loss function at the end of a well defined inference cycle, computing a bunch of derivatives, etc… and also not doing anything like any of that mid-cycle. If you’re willing to accept a large class of feedback systems as “essentially back propagation”, then it depends on what’s in your class. And I surely don’t know what it’s actually doing.
I have an important appointment this weekend that will take up most of my time, but hope to come back to this after that, but wanted to quickly note:
Why?
Last time I looked into this 6 years ago seemed like an open question and it could plausibly be backprop or at least close enough: https://www.lesswrong.com/posts/QWyYcjrXASQuRHqC5/brains-and-backprop-a-key-timeline-crux
3yrs ago Daniel Kokotajlo shared some further updates in that direction: https://www.lesswrong.com/posts/QWyYcjrXASQuRHqC5/brains-and-backprop-a-key-timeline-crux?commentId=RvZAPmy6KStmzidPF
It’s possible that you (jacobjacob) and jbash are actually in agreement that (part of) the brain does something that is not literally backprop but “relevantly similar” to backprop—but you’re emphasizing the “relevantly similar” part and jbash is emphasizing the “not literally” part?
I think that’s likely correct. What I mean is that it’s not running all the way to the end of a network, computing a loss function at the end of a well defined inference cycle, computing a bunch of derivatives, etc… and also not doing anything like any of that mid-cycle. If you’re willing to accept a large class of feedback systems as “essentially back propagation”, then it depends on what’s in your class. And I surely don’t know what it’s actually doing.