Thursday, February 17, 2011

Thoughts on Watson

Ok, let me start by admitting that I don't watch Jeopardy. I don't even own a TV. I have, however, been following the epic man-vs-machine battle that played out this week, and I must say, I'm honestly surprised. Not that the computer won. That was inevitable. If not this year, then someday soon. No, I was surprised that most of the people who commented on Watson's victory completely missed the point.

If you look online (go ahead, you know you want to), you'll find a lot of people talking about how, of course, the computer won. They say its reaction time gave it an unfair advantage. That it could push its button faster, cutting out the human opponents. Or they talk about its database. Of course, if you load all that data into a machine, you'll be able to answer any trivia question with ease.

In both cases, the commenters are fixating on the minor details and missing the main point. Watson was able to parse natural language questions and come up with reasonable answers. That was the hard part. That was the key accomplishment. Natural language processing is unbelievably difficult (trust me, I worked in the NLP lab as an undergraduate). Button pushing and database access are trivial.

The interesting thin is, this shows our natural bias. As humans, parsing the question is easy, so easy we don't even think about it. Instead, we focus on the things that give us trouble. Do we know the answer? Can we beat our opponent to the buzzer? Those are the areas that concern us, so any perceived advantage in those areas seems grossly unfair. But, in doing this, we forget the first step. You must understand the question before you can answer it.

This just highlights the differences between humans and computers. Our brains and Watson's brain have vastly different areas of competence. And lets face it, our brains work very hard to make many tasks seem trivial (object recognition, natural language processing, etc.). Even the dumbest Wheel of Fortune contestant has more processing power inside their skull than Watson could ever dream of. But here's the thing. Computers can always add more processing power. Human brains--not so much. Once computers get as complicated as a human brain, then things really get interesting.

I will say, I am a bit more moved by Noam Chomsky's criticism of Watson, dismissing it as "a bigger steamroller". Chomsky claims that Watson doesn't really understand the questions. But, I'm not so sure. How do we measure understanding? It seems like this line of argument steps into a murky, metaphysical swamp, from which we can never escape.

Instead, I tend to agree with Kurzweil's comments (from the same article):

Kurzweil says that Chomsky’s “answers are so brief that it is difficult to understand what he is trying to say. I would say that Watson is clearly not yet ‘strong AI’, but it is an important step in that direction. It is the clearest demonstration I’ve seen of computers handling the subtleties of language including metaphors, puns and jokes, something people had said would not be possible. I don’t agree with Chomsky that Watson is not impressive in that regard. As long as AI has any flaws or limitations, people will jump on these. By the time that the set of these limitations is nil, AI will have long since surpassed unaided human intelligence.”