SAN FRANCISCO – IBM Watson blew minds when it successfully defeated human champions on Jeopardy, but compared to Project Debater, which was demonstrated at IBM Think this week, that AI looks like a one trick pony.
Project Debater is the first artificial intelligence (AI) system designed to debate humans on a variety of topics. It makes use of work from IBM Research on listening comprehension, data-driven speechwriting, and modeling human dilemmas.
A few hundred media and conference attendees gathered in the Yerba Buena Center to hear Debater go head-to-head with debating champion Harish Natarajan on whether preschools should be subsidized. The topic was chosen from a curated list and revealed to both participants a few minutes before the debate began. Debater took the pro stance, with Natarajan arguing against subsidies.
The debate was hosted by Emmy Award winner John Donvan, moderator of Intelligence Squared debates since 2008.
The format was simple: each participant had four minutes to state an argument, four minutes to put forth a rebuttal to the opponent, and then each offered a final two-minute summation. There was a short break between segments to allow the debaters to gather their thoughts. The audience voted online both before and after the debate to gauge what, if any, change in its opinion came about as a result.
Unlike Watson, who has been personified as a male, Debater, whose physical manifestation was a tall black tower featuring undulating blue lights, spoke in a mellow female voice. She more than exceeded expectations, according to Project Debater principal investigator Noam Slomin, who noted that Debater showed empathy for human needs, while Natarajan’s arguments were more logic-based. The team was surprised (but delighted) that she responded correctly to implied arguments, something they hadn’t seen her do before.
She got her information from a database of 10 million sentences about various topics, weeding relevant information from it in almost real time. The system relies on an offline technique known in natural language processing as Wikification, which identifies Wikipedia concepts regardless of the words used. For example, autonomous vehicles could also be known as autonomous cars or self-driving cars. This information is stored and helps the system retrieve appropriate sentences during the debate.
Debater is a multi AI system, noted Ranit Aharonov, worldwide manager, Project Debater. Most AIs can only handle a single task, such as recognizing an image; Debater includes some deep learning and some more explainable AI to comprehend what her opponent says and formulate arguments. She even makes jokes.
While Debater is something of a novelty, she is part of a grander vision to allow AIs and humans to work together. In past challenges like Jeopardy, there was one right answer; in a debate there isn’t. Debater could have argued either side. Part of the magic was her ability to recognize the information she had that supported her side, and to argue consistently. Project Debater researchers reported that she achieved close to 100 percent accuracy in this task.
Who won? In this case, the human swayed the audience’s opinion, taking 17 percentage points away from the initial pro votes. But, said IBM Labs director Dr. Dario Gil, “It’s not about winning or losing, it’s about the ability to build complementary technology.” He sees AI as something to augment people.
Slomin added that he’d like to bring Debater into the world of education and use her to teach critical thinking. “We have a lot in the pipeline,” he said. “This is just the beginning.”
