Can The Jeopardy Challenge Help IBM Compete With Apple And Google? - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
01:39 PM
Doug Henschen
Doug Henschen
Connect Directly

Can The Jeopardy Challenge Help IBM Compete With Apple And Google?

Noted author Stephen Baker says this week's man vs. machine TV contest will help Big Blue attract the best and brightest. Early uses of the technology will show up on cell phones, he predicts.

Stephen Baker already knows the outcome of this week's Jeopardy quiz-show challenge between IBM's Watson super computer and grand champions Ken Jennings and Brad Rutter. But he won't say who (or what) comes out on top.

Like everyone present during the taping of the shows (to be broadcast on February 14, 15 and 16), Baker signed what he describes as a "Draconian" non-disclosure agreement. What sets Baker apart from everyone else in the audience that day is the fact that he will reveal the in-depth, behind-the-scenes story of Watson in great detail.

Author of the notable 2008 big-data-computing expose "The Numerati," Baker has spent the last year researching and writing Final Jeopardy: Man Vs. Machine and the Quest to Know Everything, set for release on February 17, the day after the matches conclude. In fact, the first 11 chapters are already available as an e-book, with the 12th and final chapter to be downloadable the day the print edition is released.

Baker couldn't discuss the outcome of the matches. But in a pre-show interview with Information Week, he shared details on the wake-up call IBM had in the early days of the project and the disagreements the company had with Jeopardy's producers. Baker also shares his take on why the "contrivance" of a TV quiz-show challenge is a serious matter for IBM and what's likely to come of the technology breakthroughs.

A Darwinian Experiment

You might think IBM could assemble all the technology needed to run Watson from its vast portfolio of commercial computing technologies, but Baker says that's not the case. Sure, it runs on off-the-shelf IBM Power 750 Servers, but much of the analytic software -- the heart of Watson's brain -- was developed from scratch.

Back in 2007, when the project was in its opening stages at IBM Research, David Ferrucci, the computer scientist leading the challenge, feared that the project would either fail miserably or, worse, be discovered and outdone by some basement hacker taking a novel approach.

To set up internal competition that would subject his researchers to "Darwinian pressures," as Baker writes in his book, Ferrucci assigned James Fan, a recent doctoral grad and new researcher on his team, to develop a system that would take on IBM's best incumbent question and answer technology, a platform called Piquant.

Inside Watson, IBM's Jeopardy Computer
(click image for larger view)
Slideshow: Inside Watson, IBM's Jeopardy Computer

The two rival teams, Fan and a group of veteran researchers experienced with Piquant, were given four weeks to develop and train a system on the same set of 500 sample Jeopardy clues. The Piquant developers could draw from an existing platform based on years of research, but Fan had to cobble a system together completely from scratch. To do so, he combined entity extraction technologies (software that can spot people, places, things, dates and concepts) with lots of algorithms -- some sophisticated, some incredibly crude.

In one example, he developed a program that would submit the Jeopardy clue as a Google search and then take the title of the first Wikipedia page appearing in the result set as the correct answer.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 3
Comment  | 
Print  | 
More Insights
11 Ways DevOps Is Evolving
Lisa Morgan, Freelance Writer,  2/18/2021
Graph-Based AI Enters the Enterprise Mainstream
James Kobielus, Tech Analyst, Consultant and Author,  2/16/2021
What Comes Next for AWS with Jassy to Become Amazon CEO
Joao-Pierre S. Ruth, Senior Writer,  2/4/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll