I don’t think my reasoning was particularly strong there, but the point is less “how can you use gradient descent, a supervised-learning tool, to get unsupervised stuff????” and more “how can you use Hebbian learning, an unsupervised-learning tool, to get supervised stuff????”
Autoencoders transform unsupervised learning into supervised learning in a specific way (by framing “understand the structure of the data” as “be able to reconstruct the data from a smaller representation”).
But the reverse is much less common. EG, it would be a little weird to apply clustering (an unsupervised learning technique) to a supervised task. It would be surprising to find out that doing so was actually equivalent to some pre-existing supervised learning tool. (But perhaps not as surprising as I was making it out to be, here.)
“how can you use Hebbian learning, an unsupervised-learning tool, to get supervised stuff????”
Thanks for the precision. I guess the key insight for this is: they’re both Turing complete.
“be able to reconstruct the data from a smaller representation”
Doesn’t this sound like the thalamus includes a smaller representation than the cortices?
it would be a little weird to apply clustering (an unsupervised learning technique) to a supervised task.
Actually this is one form a feature engineering., I ’m confident you can find many examples on kaggle! Yes, you’re most probably right this is telling something important, like it’s telling something important that in some sense all NP-complete problems are arguably the same problem.
I don’t think my reasoning was particularly strong there, but the point is less “how can you use gradient descent, a supervised-learning tool, to get unsupervised stuff????” and more “how can you use Hebbian learning, an unsupervised-learning tool, to get supervised stuff????”
Autoencoders transform unsupervised learning into supervised learning in a specific way (by framing “understand the structure of the data” as “be able to reconstruct the data from a smaller representation”).
But the reverse is much less common. EG, it would be a little weird to apply clustering (an unsupervised learning technique) to a supervised task. It would be surprising to find out that doing so was actually equivalent to some pre-existing supervised learning tool. (But perhaps not as surprising as I was making it out to be, here.)
Thanks for the precision. I guess the key insight for this is: they’re both Turing complete.
Doesn’t this sound like the thalamus includes a smaller representation than the cortices?
Actually this is one form a feature engineering., I ’m confident you can find many examples on kaggle! Yes, you’re most probably right this is telling something important, like it’s telling something important that in some sense all NP-complete problems are arguably the same problem.