IBM’s AI debate computer loses to human champion

Harish Natarajan, Project Debater's opponent at Think 2019, is a 2016 World Debating Championships Grand Finalist and 2012 European Debate Champion. Harish holds the world record for most debate competition victories.

People are great at arguing. But a project from IBM shows that computers are getting quite good at it, too.

On Monday, Harish Natarajan, a grand finalist in 2016’s World Debating Championships, faced off against IBM’s Project Debater — a computer touted by the company as the first artificial-intelligence system built to meaningfully debate humans. Natarajan won, but the computer demonstrated the increasingly complex arguments that AI is starting to make.

Project Debater, which has been in the works since 2012, is designed to come up with coherent, convincing speeches of its own, while taking in the arguments of a human opponent and creating its own rebuttal. It even formulates its own closing argument. To generate its arguments and rebuttals, Project Debater uses newspaper and magazine articles from its own database, and also takes in the nuances of the human opponent’s arguments. It is not connected to the internet and cannot crib arguments from sites like Wikipedia.

Monday’s debate, which was organized by nonprofit debate-hosting company Intelligence Squared US, was held in front of an audience in San Francisco’s Yerba Buena Center for the Arts. The topic of the debate — whether or not preschool should be subsidized — wasn’t revealed to the AI system or Natarajan until 15 minutes before they took to the stage. Project Debater argued in favor of subsidized preschool.

It followed traditional debate style. Each side gave a 4-minute opening speech, then they each came up with a 4-minute rebuttal to the other party. At the end, they gave a 2-minute closing argument. The audience was asked to vote for one side or the other at the start of the debate, and again at the end.

“Greetings, Harish,” Project Debater began, speaking in a mainly monotonous, female voice. It argued, among other things, that subsidized preschool can help break the poverty cycle. It spoke in complete sentences, and drew from a range of studies (including by the US Centers for Disease Control).

Natarajan followed, arguing against the resolution, saying subsidies would consume resources that middle-class families could use for other things. He also argued that subsidizing preschool doesn’t mean that all children will be able to attend.

“There will still be individuals who will be priced out because of the realities of the market,” he said.

The rebuttal segment of the debate was where some of the big differences between human and computer (beyond looks and vocal capabilities) were laid bare. Natarajan addressed specific parts of Project Debater’s arguments and rebuffed them — such as saying it’s unrealistic to expect a government has an unrestricted budget to put toward helpful programs.

Project Debater’s rebuttal, while eloquently phrased, seemed more like a continuation of its initial argument than a true rebuttal of Natarajan’s points. It saved its best counter arguments for its closing statement. While out of order, the elements of a proper debate all seemed present.

While waiting for the final vote, Natarajan said it was interesting that Project Debater could contextualize information and pull details from research. Combining its skills with those of a human, he said, “could be incredibly powerful.”

Before the debate, 79 percent of the audience agreed that preschool should be subsidized and 13 percent disagreed. By the end, 62 percent of the crowd agreed and 30 percent disagreed. Because this style of debate is scored by which side gains the most percentage points, Natarajan took the win.

Project Debater shows how AI systems have become increasingly flexible in recent years. The AI we’re used to seeing — like digital assistants built into smart speakers — can only be used in very narrow ways, such as answering specific questions. But IBM’s system shows how the technology may also be used to explore problems that don’t necessarily have a single answer. This might help people find new ways to work with computers, and to use AI to help us come up with more solutions to problems.

“It’s really pushing the boundaries [of the] kinds of AI systems that are more interactive with us and can understand us better,” IBM Research director Dario Gil said on CNN’s First Move Monday.

By Rachel Metz, CNN Business

Notice: you are using an outdated browser. Microsoft does not recommend using IE as your default browser. Some features on this website, like video and images, might not work properly. For the best experience, please upgrade your browser.