Turing test almost passed

This past weekend saw the annual attempt by computer programmers to try and fool a panel of judges that they were talking to humans rather than machines. The Turing Test is run every year, and is used as the benchmark of whether Artificially Intelligent machines can be considered as thinking or not. The test states that, if a panel of judges cannot determine whether they are talking to a machine or a human being, then whoever is on the other end of the conversation can be said to be thinking, the holy grail of all Artificial Intelligence experts.

The test takes the form of a 5 minute, unrestricted conversation, during which time the judges can ask any question, or discuss any topics that they choose. The conversation takes place vi an Instant Message type application, and so the judges have no sight of who or what they are talking with. If 30% of the judges are convinced that they are conversing with a human, and it turns out to be a machine, then that application is declared the winner and one of the longest running tests is computing will be over.

One system this year convinced 25% of the judges, which is the closest any system has ever come. One of my favourite anecdotes around this test was of one system which failed on its first question a number of years ago. When asked, “What is your favourite Woody Allen film?”, the response came back, “I quite like the smell of pine needles, although I wouldn’t say I have a favourite wood”.