ACADEMIA

Georgia Tech researcher develops test to assess a computer's capacity for human-level intelligence through its ability to create rather than to converse or deceive

A Georgia Tech professor is offering an alternative to the celebrated "Turing Test" to determine whether a machine or computer program exhibits human-level intelligence. The Turing Test - originally called the Imitation Game - was proposed by computing pioneer Alan Turing in 1950. In practice, some applications of the test require a machine to engage in dialogue and convince a human judge that it is an actual person.

Creating certain types of art also requires intelligence observed Mark Riedl, an associate professor in the School of Interactive Computing at Georgia Tech, prompting him to consider if that might lead to a better gauge of whether a machine can replicate human thought.

"It's important to note that Turing never meant for his test to be the official benchmark as to whether a machine or computer program can actually think like a human," Riedl said. "And yet it has, and it has proven to be a weak measure because it relies on deception. This proposal suggests that a better measure would be a test that asks an artificial agent to create an artifact requiring a wide range of human-level intelligent capabilities."

To that end, Riedl has created the Lovelace 2.0 Test of Artificial Creativity and Intelligence.

For the test, the artificial agent passes if it develops a creative artifact from a subset of artistic genres deemed to require human-level intelligence and the artifact meets certain creative constraints given by a human evaluator. Further, the human evaluator must determine that the object is a valid representative of the creative subset and that it meets the criteria. The created artifact needs only meet these criteria but does not need to have any aesthetic value. Finally, a human referee must determine that the combination of the subset and criteria is not an impossible standard.

The Lovelace 2.0 Test stems from the original Lovelace Test as proposed by Bringsjord, Bello and Ferrucci in 2001. The original test required that an artificial agent produce a creative item in such a way that the agent's designer cannot explain how it developed the creative item. The item, thus, must be created in such a way that is valuable, novel and surprising.

Riedl contends that the original Lovelace test does not establish clear or measurable parameters. Lovelace 2.0, however, enables the evaluator to work with defined constraints without making value judgments such as whether the artistic object created surprise.

Riedl's paper, available here, will be presented at Beyond the Turing Test, an Association for the Advancement of Artificial Intelligence (AAAI) workshop to be held January 25 - 29, 2015, in Austin, Texas.