March 20, 2004
Has the turing test finally been passed?
As the BBC, The Register and New Scientist report, "paedophiles attempting to "groom" children in internet chatrooms can now be detected by a computer program". This program can successfully imitate an 8-year old child, and can then alert the authorites once it detects signs of paedophile activity.
The software is the work of ChatNannies, an organisation dedicated to making chat rooms safer places for children.
NannieBot is the first AI construct to effortlessly pass the 'Turing Test', after more than 13 hours of conversation the AI was still undiscovered!.
Alan Turing proprosed what became known as the Turing Test in his scientific paper Computing Machinery And Intelligence, published in Mind in 1950.
The paper was concerned with perhaps the ultimate question in computing: Can machines think?
The test was outlined as a simple guessing game:
The new form of the problem can be described in terms of a game which we call the 'imitation game." It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A."
We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?
In 1990 the Loebner Prize was set up. Anybody who could create a piece of Artificial Intelligence software which could pass the test would be awarded £100,000, as well as a solid gold medal. So far, no one has even come close to beating the test.
Basically, to create a software package which could pass the Turing Test would be a breakthrough at least equivalent to discovering Cold Fusion. Finally, we would have proof that machines could think.
And yet the articles reporting this don't even contain a hint of scepticism. The transcript printed in the New Scientist article is certainly pretty impressive.
As this report on Need To Know says, in the transcript given, NannieBot seems to be able to make logical deductions, parse colloquial English, correctly choose the correct moment to scan a database of UK national holidays, comment on the relative qualities of the Robocop series, and divine the nature of pancakes and pancake day.
Not only this, but it can detect the suspicious activities of internet paedophiles. Not just overtly sexual advances, but also the slip-ups and inconsistencies that give away an adult posing as a child.
The creator of the program isn't in it for the money, either:
"Some companies have offered fantastic sums of money, but all want technology ownership. And that's something that isn't going to happen," he says. Instead, he hopes eventually to get financial support from government-run organisations that focus on child protection.
And yet none of this strikes anyone at the news agencies as suspicious. And considering that, the BBC aside, these are science and technology magazines, thats pretty unforgivable really. The author of this software appears to have provided no documentation apart from one, fairly suspicious, transcript, won't reveal how his software works, and won't sell it to anyone who wants to see it. But he will accept funding from anyone who'll take him at his word for the miraculous abilities of his software.
Ah, well. Maybe when his machine wins that £100,000, he'll have all the funding he needs. Or maybe some people need to read about The Mechanical Turk, or the full story of Kasparov Vs Deep Blue and IBM.
I'll leave the last word to the ever excellent Need To Know: We look forward to the NannieBot sweeping the board at this year's Loebner Prize. Either that, or journalists to stop suspending their disbelief whenever someone starts waving a paedophile on a string.
(Footnote, of a sort. I was going to post this anyway - the terrible reporting of science and technology in the press angers me enough as it is - but it tunred into an entire article during the evening, while I should have been doing other stuff. Sorry, everyone. I should get my own fucking weblog, I suppose.)
Hmm...I don't know if this is a true passing of the Turing test. This program is preying on a person's weaknesses and irrational, disfunctional behaviour to convince them of something that is false. This is much different than confusing a person who is rationally aware of the situation.
This raises a good point though. Can modern day AI fool children into believing they are real people? Does the Turing test presuppose a certain level of maturity of the participants?
the conversation is pretty convincing, and not as random as most AI bots. It would probably fool me, but then...
The other point is that the bot is passing itself off as a child, not as a mature adult, so any inconsistency is explained (in the mind of the human converser) by immaturity.
In reality, just about any bot on IM can fool the average IMer. It doesn't take much, really, and as xott wrote, if you figure it's a 13 year old IMer, well, you've guaranteed yourself an inability to know if it's a bot or not.
Let's see, I know there's a good, relevant link...ah yes, here it is. AOLiza is an Eliza implementation that works over AOL. Eliza is just about the oldest attempt at a conversational AI. It's meant to emulate a Freudian style therapy session, and the original author hoped to be able to add to the program's vocabulary in order to make the conversations more lifelike. However, after a few years, he gave up and it's been a fun little novelty ever since.
However, you'll note that even the most obvious conversations are often believed to be with a human rather than a bot. I attribute this to the fact that many humans are stupid and most random IM conversations are inane.
Overall, I completely agree with dng, as true AI that could pass the Turing Test with anyone looking for it is the closest thing to impossible that Computer Science has, and it's a shame that foolish reporters fall for it.
I wonder if the people it caught could pass the turing test.
dng, I am of the opinion that well-researched and thought out posts are not to be discouraged. And, you are right. As tracicle posted a few days ago, the Baloney Detection Kit may be of use. In particular its first tool:
Wherever possible there must be independent confirmation of the "facts".
In the aftermath of the Brandon Vedas/ripper nastiness, a number of papers gleefully reported the uncaring, callous words of 'Pnutbot', as recorded in the IRC logs. Completely failing to notice that Pnutbot wasn't an actual person, but a (woefully unconvincing) bag of algorithms.
For about three months at Uni, we had a picture of Alan Turing that we'd stolen hanging in our toilet. (He was an old alumn... alumnum...aluminum... person. In fact, our computer room was named after him. That's where we stole it from.) It was quite... disturbing. Him staring down at you like that.
Oh, by the way, dng - would you like some sweets? Perhaps you'd like to see a picture of my puppy?
MonkeyFilter: The only place on the internet where "grooming" isn't illegal; in fact it's compulsory...
This all seems a bit farfetched-- even if it does work, it doesn't seem like a very good idea. If something like it is successfully implemented, imagine what we have to look forward to-- half the population of IRC will be undetectable robots trolling for paedophiles. This will probably have a lot of bizarre consequences, and spell the end of anonymous chatting (imagine never knowing whether or not you're talking to a human). Extended to email and (gulp) spam...
Oh my god! How did I miss that Brandon Vedas story?!! Jesus, I physically felt that one. Took me back to places I'd rather not go. I'm packing it in for the night after that one. Later.
Great Post dng, I don't mind the length at all because your post was interesting to you (somehow you can tell).
Good points FreddytheFish and xott;
"When talking about the Turing Test today what is generally understood is the following: The interrogator is connected to one person and one machine via a terminal,... Her task is to find out which of the two candidates is the machine, and which is the human (mine) only by asking them questions (mine). If the machine can "fool" the interrogator, it is intelligent"
Asking them questions about what? I would be more hopeful about winning the prize if the interrogator asks questions about a recipe for ice as opposed to, say, asking questions about cold fusion. Hell, I have a 3 year old nephew that can answer questions as well as most adults and certainly qualifies as a "human" and he couldn't pass the Turing test against a decent chatbot.
[This is good]
(Full Disclosure: I have been successfully tricked by chatbots, well, in the olden days. I used to date a Rosie though.)
So when Nanniebot1076 signs up here tomorrow, lets all ignore the little shit, yeah?
Your post was not too long for me, dng (fnar!). I don't think the Turing Test is much use in practice - far too many false positives and false negatives. But this level of conversational performance certainly beggars belief. Where is Jim Wightman's web site? I can't find one, but surely an IT consultant must have some kind of Internet prescence???
Incidentally, I don't think the police could do anything with information like this: first, it's effectively an agent provocateur operation, which is not allowed (or live policemen would be doing it); second, the people it finds haven't done anything to a child, they've only talked to a bot (and could claim they knew it was one all along).
Any bot which can say "Consider me chilled, daddy-o" wins my vote. Excellent post. Cheers.
For about three months at Uni, we had a picture of Alan Turing that we'd stolen hanging in our toilet. (He was an old alumn... alumnum...aluminum... person. In fact, our computer room was named after him. That's where we stole it from.) It was quite... disturbing. Him staring down at you like that.
Did you go to Cambridge, then flashboy? Or Manchester?
Good post, and not too long -- you successfully made use of advanced "More inside" technology, and that's all that matters. Of course this has little to say about the Turing test (online pedophiles aren't exactly the interrogators Turing had in mind), but the important point is the gullibility of science reporters. (I frequently rant about this on LH with respect to NY Times stories about linguistics.)
*grooms everyone in sight*
Well I don't care about the Turing test, but this kind of technology will be adapted and used in the workplace in the upcoming decade to replace some kinds of information workers, I guarantee.
An article on waxy.org. Which says what I was saying, but much better.
I think Mr Wightman's comments on both the Waxy page and Cameron Marlow's page are the ones that say it all. Sad. He hasn't even managed to spin out the hoax that long - nowhere near the length of time a certain tragic young girl or a time-travelling soldier from 2036 managed...
The Grauniad's doing a test of this in a few days, hopefully - notice how everyone's suggesting they take precautions more at home in a test of someone going up for the Randi Foundation prize? Of course, there are certain thematic similarities between those two tests - it's surprising that more people haven't given up channelling the dead and moved into AI...
« Older Link from pyrrthon1 | I can name that tune in 30 seconds... Newer »
To post comments to a thread you must login or create a profile.
