Certainly at least a decent step in natural language processing, right?
It’s substantially better than other question answering systems, so in that sense yes. On the other hand, it’s probably still movement towards a local maximum, rather than a robust, general strategy.
Robust, general natural language processing requires a proper grammar and parser, to get back an actionable semantic representation. That’s the kind of language processing module you’ll be able to reuse in lots of different applications.
Watson probably isn’t doing that. The system probably cherry-picks keywords, and uses a statistical classifier to predict the category of answer expected. Maybe a syntactic parse is used to help find clues, but I doubt it’s the main method.
I did my PhD on statistical parsing and am continuing to work on it as a post-doc. We’re getting better, but it’s still usually less practical than an ad hoc strategy for any given task. That doesn’t mean Watson isn’t impressive, of course. Watson shows us what can be done right now, and apparently what can be done is pretty damn sweet.
Though it’s doing more than just individual keyword stuff. I think one major point is that it’s looking at context (ie, I think it’s supposed to have at least a basic ability to deal with puns and such.)
Also, I think it is set up to learn the theme of a category if it’s not initially sure (via associated questions and answers), and using that info to get an idea of what types of answers are being sought in a particular category.
If it’s not parsing, if it’s just keyword analysis rather than any analysis of grammar, it’s going way beyond just judging the keywords individually. (Not to mention, it’s parsing enough to at least figure out which words are the ones to use for its keyword search, I think.)
Do you think that Watson is anywhere near the local maximum associated with the strategies you think is being used by that system, incidentally?
Having looked through their overview paper, I’m no longer sure. They do have modules that do parsing and semantic role labelling and such. But their model is a mixture of dozens of individual models. So it’s tough to say much about how things are fitting together. They use more sophisticated techniques than I thought, although I don’t know how much contribution those techniques actually make in the final decision.
It’s substantially better than other question answering systems, so in that sense yes. On the other hand, it’s probably still movement towards a local maximum, rather than a robust, general strategy.
Robust, general natural language processing requires a proper grammar and parser, to get back an actionable semantic representation. That’s the kind of language processing module you’ll be able to reuse in lots of different applications.
Watson probably isn’t doing that. The system probably cherry-picks keywords, and uses a statistical classifier to predict the category of answer expected. Maybe a syntactic parse is used to help find clues, but I doubt it’s the main method.
I did my PhD on statistical parsing and am continuing to work on it as a post-doc. We’re getting better, but it’s still usually less practical than an ad hoc strategy for any given task. That doesn’t mean Watson isn’t impressive, of course. Watson shows us what can be done right now, and apparently what can be done is pretty damn sweet.
Huh, thanks.
Though it’s doing more than just individual keyword stuff. I think one major point is that it’s looking at context (ie, I think it’s supposed to have at least a basic ability to deal with puns and such.)
Also, I think it is set up to learn the theme of a category if it’s not initially sure (via associated questions and answers), and using that info to get an idea of what types of answers are being sought in a particular category.
If it’s not parsing, if it’s just keyword analysis rather than any analysis of grammar, it’s going way beyond just judging the keywords individually. (Not to mention, it’s parsing enough to at least figure out which words are the ones to use for its keyword search, I think.)
Do you think that Watson is anywhere near the local maximum associated with the strategies you think is being used by that system, incidentally?
Having looked through their overview paper, I’m no longer sure. They do have modules that do parsing and semantic role labelling and such. But their model is a mixture of dozens of individual models. So it’s tough to say much about how things are fitting together. They use more sophisticated techniques than I thought, although I don’t know how much contribution those techniques actually make in the final decision.
Thanks for looking at the paper and passing on the info about what’s actually going in inside it, btw.