the surprising thing from my current perspective is: backprop is basically for supervised learning, while Hebbian learning is basically for unsupervised learning
That’s either poetic analogy or factual error.. For example autoencoders belongs to unsupervised learning and are trained through backprop.
I don’t think my reasoning was particularly strong there, but the point is less “how can you use gradient descent, a supervised-learning tool, to get unsupervised stuff????” and more “how can you use Hebbian learning, an unsupervised-learning tool, to get supervised stuff????”
Autoencoders transform unsupervised learning into supervised learning in a specific way (by framing “understand the structure of the data” as “be able to reconstruct the data from a smaller representation”).
But the reverse is much less common. EG, it would be a little weird to apply clustering (an unsupervised learning technique) to a supervised task. It would be surprising to find out that doing so was actually equivalent to some pre-existing supervised learning tool. (But perhaps not as surprising as I was making it out to be, here.)
“how can you use Hebbian learning, an unsupervised-learning tool, to get supervised stuff????”
Thanks for the precision. I guess the key insight for this is: they’re both Turing complete.
“be able to reconstruct the data from a smaller representation”
Doesn’t this sound like the thalamus includes a smaller representation than the cortices?
it would be a little weird to apply clustering (an unsupervised learning technique) to a supervised task.
Actually this is one form a feature engineering., I ’m confident you can find many examples on kaggle! Yes, you’re most probably right this is telling something important, like it’s telling something important that in some sense all NP-complete problems are arguably the same problem.
That’s either poetic analogy or factual error.. For example autoencoders belongs to unsupervised learning and are trained through backprop.
I don’t think my reasoning was particularly strong there, but the point is less “how can you use gradient descent, a supervised-learning tool, to get unsupervised stuff????” and more “how can you use Hebbian learning, an unsupervised-learning tool, to get supervised stuff????”
Autoencoders transform unsupervised learning into supervised learning in a specific way (by framing “understand the structure of the data” as “be able to reconstruct the data from a smaller representation”).
But the reverse is much less common. EG, it would be a little weird to apply clustering (an unsupervised learning technique) to a supervised task. It would be surprising to find out that doing so was actually equivalent to some pre-existing supervised learning tool. (But perhaps not as surprising as I was making it out to be, here.)
Thanks for the precision. I guess the key insight for this is: they’re both Turing complete.
Doesn’t this sound like the thalamus includes a smaller representation than the cortices?
Actually this is one form a feature engineering., I ’m confident you can find many examples on kaggle! Yes, you’re most probably right this is telling something important, like it’s telling something important that in some sense all NP-complete problems are arguably the same problem.