March 20, 2004

Has the turing test finally been passed? As the BBC, The Register and New Scientist report, "paedophiles attempting to "groom" children in internet chatrooms can now be detected by a computer program". This program can successfully imitate an 8-year old child, and can then alert the authorites once it detects signs of paedophile activity. The software is the work of ChatNannies, an organisation dedicated to making chat rooms safer places for children. NannieBot is the first AI construct to effortlessly pass the 'Turing Test', after more than 13 hours of conversation the AI was still undiscovered!.

Alan Turing proprosed what became known as the Turing Test in his scientific paper Computing Machinery And Intelligence, published in Mind in 1950. The paper was concerned with perhaps the ultimate question in computing: Can machines think? The test was outlined as a simple guessing game: The new form of the problem can be described in terms of a game which we call the 'imitation game." It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A." We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? In 1990 the Loebner Prize was set up. Anybody who could create a piece of Artificial Intelligence software which could pass the test would be awarded

posted by dng 20 years ago
  • Hmm...I don't know if this is a true passing of the Turing test. This program is preying on a person's weaknesses and irrational, disfunctional behaviour to convince them of something that is false. This is much different than confusing a person who is rationally aware of the situation. This raises a good point though. Can modern day AI fool children into believing they are real people? Does the Turing test presuppose a certain level of maturity of the participants?
  • the conversation is pretty convincing, and not as random as most AI bots. It would probably fool me, but then... The other point is that the bot is passing itself off as a child, not as a mature adult, so any inconsistency is explained (in the mind of the human converser) by immaturity.
  • In reality, just about any bot on IM can fool the average IMer. It doesn't take much, really, and as xott wrote, if you figure it's a 13 year old IMer, well, you've guaranteed yourself an inability to know if it's a bot or not. Let's see, I know there's a good, relevant link...ah yes, here it is. AOLiza is an Eliza implementation that works over AOL. Eliza is just about the oldest attempt at a conversational AI. It's meant to emulate a Freudian style therapy session, and the original author hoped to be able to add to the program's vocabulary in order to make the conversations more lifelike. However, after a few years, he gave up and it's been a fun little novelty ever since. However, you'll note that even the most obvious conversations are often believed to be with a human rather than a bot. I attribute this to the fact that many humans are stupid and most random IM conversations are inane. Overall, I completely agree with dng, as true AI that could pass the Turing Test with anyone looking for it is the closest thing to impossible that Computer Science has, and it's a shame that foolish reporters fall for it.
  • I wonder if the people it caught could pass the turing test.
  • dng, I am of the opinion that well-researched and thought out posts are not to be discouraged. And, you are right. As tracicle posted a few days ago, the Baloney Detection Kit may be of use. In particular its first tool: Wherever possible there must be independent confirmation of the "facts".
  • In the aftermath of the Brandon Vedas/ripper nastiness, a number of papers gleefully reported the uncaring, callous words of 'Pnutbot', as recorded in the IRC logs. Completely failing to notice that Pnutbot wasn't an actual person, but a (woefully unconvincing) bag of algorithms. For about three months at Uni, we had a picture of Alan Turing that we'd stolen hanging in our toilet. (He was an old alumn... alumnum...aluminum... person. In fact, our computer room was named after him. That's where we stole it from.) It was quite... disturbing. Him staring down at you like that. Oh, by the way, dng - would you like some sweets? Perhaps you'd like to see a picture of my puppy? MonkeyFilter: The only place on the internet where "grooming" isn't illegal; in fact it's compulsory...
  • This all seems a bit farfetched-- even if it does work, it doesn't seem like a very good idea. If something like it is successfully implemented, imagine what we have to look forward to-- half the population of IRC will be undetectable robots trolling for paedophiles. This will probably have a lot of bizarre consequences, and spell the end of anonymous chatting (imagine never knowing whether or not you're talking to a human). Extended to email and (gulp) spam...
  • Oh my god! How did I miss that Brandon Vedas story?!! Jesus, I physically felt that one. Took me back to places I'd rather not go. I'm packing it in for the night after that one. Later.
  • Great Post dng, I don't mind the length at all because your post was interesting to you (somehow you can tell). Good points FreddytheFish and xott; "When talking about the Turing Test today what is generally understood is the following: The interrogator is connected to one person and one machine via a terminal,... Her task is to find out which of the two candidates is the machine, and which is the human (mine) only by asking them questions (mine). If the machine can "fool" the interrogator, it is intelligent" Asking them questions about what? I would be more hopeful about winning the prize if the interrogator asks questions about a recipe for ice as opposed to, say, asking questions about cold fusion. Hell, I have a 3 year old nephew that can answer questions as well as most adults and certainly qualifies as a "human" and he couldn't pass the Turing test against a decent chatbot. [This is good] (Full Disclosure: I have been successfully tricked by chatbots, well, in the olden days. I used to date a Rosie though.)
  • So when Nanniebot1076 signs up here tomorrow, lets all ignore the little shit, yeah?
  • Your post was not too long for me, dng (fnar!). I don't think the Turing Test is much use in practice - far too many false positives and false negatives. But this level of conversational performance certainly beggars belief. Where is Jim Wightman's web site? I can't find one, but surely an IT consultant must have some kind of Internet prescence??? Incidentally, I don't think the police could do anything with information like this: first, it's effectively an agent provocateur operation, which is not allowed (or live policemen would be doing it); second, the people it finds haven't done anything to a child, they've only talked to a bot (and could claim they knew it was one all along).
  • Any bot which can say "Consider me chilled, daddy-o" wins my vote. Excellent post. Cheers.
  • For about three months at Uni, we had a picture of Alan Turing that we'd stolen hanging in our toilet. (He was an old alumn... alumnum...aluminum... person. In fact, our computer room was named after him. That's where we stole it from.) It was quite... disturbing. Him staring down at you like that. Did you go to Cambridge, then flashboy? Or Manchester?
  • Good post, and not too long -- you successfully made use of advanced "More inside" technology, and that's all that matters. Of course this has little to say about the Turing test (online pedophiles aren't exactly the interrogators Turing had in mind), but the important point is the gullibility of science reporters. (I frequently rant about this on LH with respect to NY Times stories about linguistics.) *grooms everyone in sight*
  • Well I don't care about the Turing test, but this kind of technology will be adapted and used in the workplace in the upcoming decade to replace some kinds of information workers, I guarantee.
  • An article on waxy.org. Which says what I was saying, but much better.
  • I think Mr Wightman's comments on both the Waxy page and Cameron Marlow's page are the ones that say it all. Sad. He hasn't even managed to spin out the hoax that long - nowhere near the length of time a certain tragic young girl or a time-travelling soldier from 2036 managed... The Grauniad's doing a test of this in a few days, hopefully - notice how everyone's suggesting they take precautions more at home in a test of someone going up for the Randi Foundation prize? Of course, there are certain thematic similarities between those two tests - it's surprising that more people haven't given up channelling the dead and moved into AI...