Tag Archives: Artificial Intelligence

If you liked my blog, you’d like this post. Trust me!

One of the sites that most impressed me when I first started browsing the web was called MovieCritic.com. You would rate a few movies you saw, then it would predict whether you’d like a new movie. It would even let you find one that matches both your taste and your girlfriend’s. Pure magic, for that time. For me that was the first demonstration of what we can achieve with the web as a medium.

MovieCritic is dead for a few years now, but recommender systems are now everywhere. NetFlix runs one of the most successful commercial implementations (Amazon another classic example, “People who bought this book…”), and two years ago they challenged researches to come up with a system that would perform 10% better than their own, in predicting users’ ratings. The best achieving team so far almost got there, and today I attended a talk in the Technion by Yehuda Koren, one of the team members and a researcher at Yahoo! Research Haifa lab.

Most methods follow the neighborhood-based model – find an item’s neighbours (in some representation), and predict based on their rating. This may be done in a user-user matching (find users like this user, then check their rating) or item-item (find items like the rated item, then predict based on how the user rated those items). One of the interesting approaches proposed by Koren’s team represented both users and movies in the same space, then looked for similarity in this unified space.

The most striking finding for me, however, was that winning strategies did not use anything from the movie’s “content” features. Genre, director, actors, length, etc. – all these did not produce any additional value beyond the plain statistical analysis and correlation of ratings and users, and are therefore not used at all. In fact, Koren claims that knowing that a certain user is a Tom Hanks fan makes no difference, we will infer this from the recommendations anyway (assuming there are enough of them of course).

I find that almost sad… Not being able to intelligently reason over the underlying logic exposed by an AI software is a tremendous drawback in my eyes, even if the overall prediction score is better. Telling the user “you may want to watch this movie because A and B and C” can benefit in more satisfaction by the user, understanding even the incorrect predictions, and possibly leading to a feedback cycle. Doing away with it is like showing web search results without keyword highlighting, no visible cue for the user why this result was returned (“…trust me, I know what’s the right answer for you!“).

Solving Checkers

I attended a talk today at the Technion by Jonathan Schaeffer from the University of Alberta in Canada, the person behind Chinook. Chinook is practically the world champion in the game of checkers, if only the world checkers federations would allow computer programs to compete.

Solving checkers, according to Schaeffer, turns out to be much more a laborious process of solving lots of hardware problems, rather than artificial intelligence algorithms. It took 19 years to build a database of board positions sufficiently large to solve from any given position (and there are 5 billion billion such positions, mind you). Since relying on a wrong piece of data could be catastrophic to the complex calculation, there was a lot of dealing with disk failures, network failures, grid calculation. Sounds a lot like running a large-scale search engine, only that search engines can afford to say “Oops…”.

In one case, Schaeffer, who started out in 1989, reached the 32-bit limit in his files. He started refactoring his code and databases to accommodate 64-bit pointer arithmetic, but ended up deciding it’s better to wait 3 more years for 64-bit to become mainstream. So in fact sitting idle for 3 years can do wonders to your projects. In another case, a bug discovered in 2005 was traced back to data created in 1992, which forced a re-calculation of all the data that relied on it. Bummer.

But what I truly found most interesting was the human story behind it all. Chinook’s major opponent was the late Marion Tinsley, world champion and undisputably the best human checkers player ever, who in his career of thousands of games lost only five. Schaeffer tells of hate letters he received after Chinook beat Tinsley, who died shortly after, and of his decision then to prove that a human simply can not beat a machine in the game.

The bottom line? Schaeffer has now indeed proven that the best an opponent of Chinook can achieve is a draw. This positions humans as those who could err and lose. A beautiful definition…