I ran into the same problem recently. A slashdot discussion referred me to Geoffrey Hinton’sRestricted Boltzmann Machines, a method of unsupervised learning that he claims to have successfully applied to character recognition.
Despite reading several papers, a ppt presentation billed as a “gentle introduction” to the topic, and the video presentation, I could never discern what the actual algorithm is that Hinton is using. Fortunately, he posted the code and so I could look at the core algorithm (rbm.m).
But even then I couldn’t figure out why exactly it’s able to find structure in the data by working through toy problems (which it seemed to choke on). Eventually I gave up and started reading about Kohonen maps, which appear to do the same thing, but are much easier to understand.
OTOH, I could just be really stupid and unqualified to read this stuff to begin with.
Don’t worry, you’re not the only one who has difficulty reading Hinton’s papers. They are quite confusing. The problem is that while there are only two or three big important ideas, they’re spread out over the course of something like 20 papers.
This post makes me think we should have a LW math/ai/machine learning reading group. Choose a paper, read it, someone summarizes it in a main post, and then every one else chimes in with their 2c.
This post makes me think we should have a LW math/ai/machine learning reading group. Choose a paper, read it, someone summarizes it in a main post, and then every one else chimes in with their 2c.
Great idea! Also, make a list the links to these articles accessible from the Wiki. It would also be a great opportunity to apply the LW filters (as applicable to the situation) like:
-What regularity of the search space does the algorithm exploit?
-Why does the algorithm appear to make use of randomness, and how can it be derandomized?
-What phenomena does this theory rule out?
I’m sure there are some ways I could contribute, esp. with the Perceptual Control Theory stuff going around. I’m almost done with the four BYTE magazine articles on it that pjeby keeps referring to.
That sounds good, but in my experience with paper reading groups, most people don’t find the time to actually read the material under discussion. It ends up being a venue for signalling and socializing. Maybe that wouldn’t apply here.
I ran into the same problem recently. A slashdot discussion referred me to Geoffrey Hinton’s Restricted Boltzmann Machines, a method of unsupervised learning that he claims to have successfully applied to character recognition.
Despite reading several papers, a ppt presentation billed as a “gentle introduction” to the topic, and the video presentation, I could never discern what the actual algorithm is that Hinton is using. Fortunately, he posted the code and so I could look at the core algorithm (rbm.m).
But even then I couldn’t figure out why exactly it’s able to find structure in the data by working through toy problems (which it seemed to choke on). Eventually I gave up and started reading about Kohonen maps, which appear to do the same thing, but are much easier to understand.
OTOH, I could just be really stupid and unqualified to read this stuff to begin with.
Don’t worry, you’re not the only one who has difficulty reading Hinton’s papers. They are quite confusing. The problem is that while there are only two or three big important ideas, they’re spread out over the course of something like 20 papers.
This post makes me think we should have a LW math/ai/machine learning reading group. Choose a paper, read it, someone summarizes it in a main post, and then every one else chimes in with their 2c.
Great idea! Also, make a list the links to these articles accessible from the Wiki. It would also be a great opportunity to apply the LW filters (as applicable to the situation) like:
-What regularity of the search space does the algorithm exploit? -Why does the algorithm appear to make use of randomness, and how can it be derandomized? -What phenomena does this theory rule out?
I’m sure there are some ways I could contribute, esp. with the Perceptual Control Theory stuff going around. I’m almost done with the four BYTE magazine articles on it that pjeby keeps referring to.
That sounds good, but in my experience with paper reading groups, most people don’t find the time to actually read the material under discussion. It ends up being a venue for signalling and socializing. Maybe that wouldn’t apply here.