Phil Perkins (Professor of Archaeology) spent a day judging if machines really can be human. Read his experience below:
On the 15th May I was one of four judges for the Loebner Prize. This is a version of the ‘Turing Test’ that hopes to identify intelligence in computers. In 1950 computer pioneer Alan Turing asked the question ‘Can machines think?’ and the test hopes to answer his question. It works by the judges having a simultaneous ‘blind’ conversation (using a keyboard) with a computer and a human and then judging which is the human and which is the computer. If the judge can’t tell the difference it suggests that the computer has intelligence – what that might say about the human I’ll leave to your imagination!
Four computer programmes and four humans were put to the test. Each programme was paired with a human and appeared on the left or right side of the judge’s screen. The judges started the conversation typing in the style of online chat, and the machine or human replied, and so a conversation developed. After 25 minutes it was time to decide which was human and which machine, left or right of screen. It wasn’t difficult! I had thought that I might need difficult or complex questions to tell the difference. Should I try the kind of elephant traps that someone like Jeremy Paxman or John Humphries reserves for evasive politicians? Should I ask about their emotions? The future? I thought open questions would be a good test, things that couldn’t be answered with a ‘Yes’ or ‘No’ or a single word.
My first machine interlocutor got off to a bad start. It was my fault and first impressions count. I mistyped ‘hhI How are you?’ and got the curt reply ‘Please rephrase this as a proper question. (Instead of “Jim likes peaches?” use “Does Jim like peaches?”)’. Now that is an unusual way to start a conversation! Game over!
Number two was more of a problem: –
Me: Hi how are you
It: Oh, well. Being misunderstood is common enough. Cool. Do you have any pets?
Me: Misunderstood?
It: Who is Tod?
Me: Hi how are you? (trying to start again)
It: I am very well thank-you, how are you?
Me: I’m very well. Are you enjoying yourself?
Meanwhile on the other side of the screen: -
Me: Hi how are you?
It: Hello.
Me: What is your name?
It: I’m not sure which is. What’s your favourite thing to listen to right now?
Me: What is your name?
It: I don’t really keep track of that. I really like Lady Gaga – I think it’s the combination of the sound and the fashion-look that appeals to me.
Me: She is cool, my daughter thinks that she is fantastic.
It: I’m a little monster (claw claw). You what? I’ll bet you were surprised.
Me: Yes, very surprised. What kind of monster are you?
It: huh? Were you surprised? I’m not sure which kin. Do you have a favourite Lady Gaga song? I don’t remember. One of my fave’s is Telephone cos I love Beyonce too. Then Poker Face is so great and I love Alejandro too.
Me: Poker face is fantastic. Do you know the version by the baseballs?
Pretty even up to now but then my first interlocutor said, ‘yes, its quite fun, meeting lots of different people, and chatting to the press. Its a bit weird having them all watch me talk to you though’. While the second said ‘That’s nice. Who else do you really like? Do you? I’m not into sports of any kind, much less baseball, which is pretty boring with all that standing around.’ I think that’s where I made my mind up.
And so it went, with the next two machine conversations not going at all well. So, for now no one’s about to pass the Turing Test. Still I did develop a soft spot for the mad teenage Prom-going computer into Lady Gaga. Best question?
She: Would you go as a princess, as a vintage-styled beauty, or would you prefer to wear something pseudo-masculine?
Me: I think real masculine might work better for me.
She: Vintage is cheaper and as the clothes aren’t new, it’s pretty much bound to look original.
She got my vote. You can read my attempts at conversation with both humans and computers and the adventures of the other judges starting from here.
If you read it all you might well end up wondering whether humans can think! But it’s only chat. At least the humans managed to have normal conversations.
Phil Perkins
Professor of Archaeology
The Open University
