 |
I propose to consider the question, "Can Machines Think?" -Alan Mathison Turing |
 |
Alan Turing proposed an operational test of intelligence as a replacement for the philosophical question, "Can machines think?" Variations of this test have been used to assess performance levels of many AI programs. A restricted adaptation of the Turing Test popularized by the Loebner Competition has been fun, although currently many AI scientists criticize the restricted version of the test because it does not provide insights into the mechanisms of thought. Turing's contributions to mathematics, computer science and early AI extend well beyond his seminal paper proposing the test (below).
Computing Machinery and Intelligence. By Alan M. Turing. "Originally published by Oxford University Press on behalf of MIND (the Journal of the Mind Association), vol. LIX, no. 236, pp. 433-60, 1950. Published on the abelard site by permission of Oxford University Press." An all-time classic paper that discusses the prospects of AI and dismisses some still-current arguments against AI.
The Turing Test. By Graham Oppy & David Dowe. Entry in the Stanford Encyclopedia of Philosophy. "The phrase 'The Turing Test' is most properly used to refer to a proposal made by Turing (1950) as a way of dealing with the question whether machines can think. ... The phrase 'The Turing Test' is sometimes used more generally to refer to some kinds of behavioural tests for the presence of mind, or thought, or intelligence in putatively minded entities. So, for example, it is sometimes suggested that The Turing Test is prefigured in Descartes' Discourse on the Method. (Copeland (2000:527) finds an anticipation of the test in the 1668 writings of the Cartesian de Cordemoy. Gunderson (1964) provides an early instance of those who find that Turing's work is foreshadowed in the work of Descartes.) ... There are many different objections to The Turing Test which have surfaced in the literature during the past fifty years, but which we have not yet discussed. We cannot hope to canvass all of these objections here. However, there is one argument -- Searle's 'Chinese Room' argument -- that is mentioned so often in connection with the Turing Test that we feel obliged to end with some discussion of it."
Definition of Turing Test. From the University of Alberta Cognitive Science Dictionary, maintained by Dr. Michael Dawson and Dr. David Medler.
The Turing Test. By Lynellen D.S. Perry. ACM Crossroads Student Magazine. A critical look at the test with many helpful links.
It's the thought that counts. By Dylan Evans. Guardian (October 6, 2001). "Will machines ever be able to think for themselves? And will we be able to tell if and when they do? Pondering these questions in 1950, the British mathematician Alan Turing came up with a simple way of settling the matter. Put a machine in one room, he suggested, and a human being in another. ... Turing hoped that his test would cut through a lot of fruitless semantic debate. It was an engineer's solution, rather than a philosopher's. " The article ends by challenging you to decide which of the excerpted conversations were conducted with humans and which with non-humans.
Human possibilities. By Jim McClellan. The Guardian (October 23, 2003). "'Tell me a joke.' A small audience sits in front of a big screen waiting for a response to pop up. A short pause - then some type flickers up onscreen. 'Why did the chicken cross the road?' A slight groan from the audience. A reply is dutifully typed up. 'I don't know - why did the chicken cross the road?' Another pause. Up on screen, more type appears. 'Because it was stapled to the elephant.' Welcome to the Loebner prize contest, an annual attempt to find the world's most 'human-seeming' chatbot. A chatbot is a program designed imitate human conversation in text form. This year's event took place at the University of Surrey. ... At the end of the afternoon, as expected, the two humans came out top, though rather perplexingly, one judge decided that both only rated one on a scale of five when it came to seeming human. (The same judge gave all the bots one, as well.) The chatbot that came next (and hence won) was Jabberwock, created by Juergen Pirner, a German publisher of fantasy and science fiction. ... Organiser Lynn Hamill, of Surrey University's Digital World Research Centre, says she saw the contest as an amusing way of advancing the interests of the Centre, which was set up to look at the way people and technologies interact. 'The Loebner prize is a useful way of getting people to think about these things,' she says, adding that it may help AI research in general."
Look What's Talking: Software Robots With Chatterbots on the Web, Conversation Can Be Surprising, or Surprisingly Limited. By David Pescovitz. The New York Times on the Web; March 18, 1999, available from Web Lab. "In the near future, chatterbots are expected to act as the voices of other Web-based intelligent agents, generally called bots, which gather data or perform other tasks automatically for users. ... The yardstick for judging machine intelligence is whether it can play what the British mathematician Alan M. Turing called an "imitation game," now known universally as the Turing test."
Alan Turing: Father of the computer. BBC News (April 28, 1999). "Turing believed that machines could be created that would mimic the processes of the human brain. He acknowledged the difficulty people would have accepting a machine to rival their own intelligence, a problem that still plagues artificial intelligence today."
Donald Michie: The very early days. Interviewed by Michael Bain for the seminar, Artificial Intelligence - Recollections of the Pioneers (October 2002). "Q: What was your earliest contact with the idea of intelligent machinery? A: Arriving at Bletchley Park in 1942 I formed a friendship with Alan Turing, and in April 1943 with Jack Good. The three of us formed a sort of discussion club focused around Turing's astonishing 'child machine' concept. His proposal was to use our knowledge of how the brain acquires its intelligence as a model for designing a teachable intelligent machine." You can also watch the interview: Quicktime or Realmedia.
Alan
Turing and Artificial Intelligence. A Special
Issue of the Journal of Logic Language and Information. Volume 9, Issue
4 (October 2000). Varol Akman and Patrick Blackburn, guest editors.
Simulating
Conversations: The Communion Game. By Stephen J. Cowley and Karl F.
MacDorman. Appeared in AI & Society (1995) 9:116-137.
Being Real. By Judith S. Donath, MIT Media Lab. [To appear in Goldberg, K. (ed.) The Robot in the Garden: Telerobotics and Telepistemology in the Age of the Internet, MIT Press.] "This essay approaches these issues by focusing on a question with special resonance for both technologists and philosophers: can one tell if the person at the other end of an online discussion is indeed a person?"
Some challenges and grand challenges for computational intelligence. By Edward A. Feigenbaum. Journal of the ACM (JACM), Volume 50 , Issue 1, Pages: 32 - 40 (January 2003). Available from ACM and KurzweilAI.net. "The Turing Test is a very ambitious Grand Challenge. The 'Feigenbaum Test' is more manageable: focus on natural science, engineering, or medicine with conversation in the jargonized and stylized language of these disciplines."
Accelerating Problem Solving. Listen to this presentation by David Fogel, CEO of Natural Selection, delivered at the Accelerating Change 2005 conference, and made available by IT Conversations: "The Turing Test is often seen as an accurate indication of machine intelligence. David Fogel refutes this by quoting Alan Turing himself and by showing how current internet use undermines the traditional application of the test's results. In place of this flawed definition, Fogel suggests one of his own; namely, that intelligence may be viewed as the ability to adapt behavior to meet goals in a range of environments."
The Turing Test. From Chapter One (available online) of George F. Luger's textbook, Artificial Intelligence: Structures and Strategies for Complex Problem Solving, 5th Edition (Addison-Wesley; 2005). "The Turing test measures the performance of an allegedly intelligent machine against that of a human being, arguably the best and only standard for intelligent behavior. ... The Turing test, in spite of its intuitive appeal, is vulnerable to a number of justifiable criticisms. One of the most important of these is aimed at its bias toward purely symbolic problem-solving tasks. It does not test abilities requiring perceptual skill or manual dexterity, even though these are important components of human intelligence. Conversely, it is sometimes suggested that the Turing test needlessly constrains machine intelligence to fit a human mold. Perhaps machine intelligence is simply different from human intelligence and trying to evaluate it in human terms is a fundamental mistake. Do we really wish a machine would do mathematics as slowly and inaccurately as a human?"
Can Computers Think? Mapping Great Debates. Visit MacroVU's site and if you are patient, you can preview their "7 poster-sized argumentation maps that chart the entire history of the debate. The maps outline arguments put forth since 1950 by more than 380 cognitive scientists, philosophers, artificial intelligence researchers, mathematicians, and others." Map # 2 is titled: Can the Turing test determine whether computers can think?
Chatterbots, Tinymuds, And The Turing Test: Entering The Loebner Prize Competition. By Michael L. Mauldin. Presented at AAAI-94.
What's It Mean to Be Human, Anyway? By Charles Platt. Wired Magazine 3 (4): Features. "Our purpose is to find out whether 10 judges can tell the difference between humans and artificial-intelligence programs, when they are online at the same time. ... The inspiration for this event dates back to the earliest days of computing. In 1950, pioneer Alan Turing proposed that if a computer could successfully impersonate a human being during a free-form exchange of text messages, then for all practical purposes, the computer should be considered intelligent."
The First Hacker and his Imaginary Machine. Chapter 3 of the 1985 edition of Howard Rheingold's Tools for Thought (The MIT Press).
Human or Computer? Take This Test. By Sara Robinson. The New York Times (December 10, 2002; no-fee reg. req'd). "As chief scientist of the Internet portal Yahoo, Dr. Udi Manber had a profound problem: how to differentiate human intelligence from that of a machine. His concern was more than academic. Rogue computer programs masquerading as teenagers were infiltrating Yahoo chat rooms, collecting personal information or posting links to Web sites promoting company products. ... The roots of Dr. Manber's philosophical conundrum lay in a paper written 50 years earlier by the mathematician Dr. Alan Turing, who imagined a game in which a human interrogator was connected electronically to a human and a computer in the next room. The interrogator's task was to pose a series of questions that determined which of the other participants was the human. ... Dr. Manuel Blum, a professor of computer science at Carnegie Mellon who took part in the Yahoo conference, realized that the failures of artificial intelligence might provide exactly the solution Yahoo needed. Why not devise a new sort of Turing test, he suggested, that would be simple for humans but would baffle sophisticated computer programs. Dr. Manber liked the idea, so with his Ph.D. student Luis von Ahn and others Dr. Blum devised a collection of cognitive puzzles based on the challenging problems of artificial intelligence. The puzzles have the property that computers can generate and grade the tests even though they cannot pass them. The researchers decided to call their puzzles Captchas, an acronym for Completely Automated Public Turing Test to Tell Computers and Humans Apart (on the Web at www.captcha.net)."
Chinese Room Argument. Entry by John R. Searle in the MIT Encyclopedia of Cognitive Science. "The Chinese room argument is a refutation of strong artificial intelligence. 'Strong AI' is defined as the view that an appropriately programmed digital computer with the right inputs and outputs, one that satisfies the Turing test, would necessarily have a mind."
- Also see:
- Chinese room - An argument forwarded by John Searle intended to show that the mind is not a computer and how the Turing Test is inadequate. By Chris Eliasmith. Dictionary of Philosophy of Mind.
- 2 interviews with John Searle.
Lessons from a Restricted Turing Test. Stuart M. Shieber. Communications of the Association for Computing Machinery (1994), Volume 37, Number 6, pages 70-78. "We report on the recent Loebner prize competition inspired by Turing's test of intelligent behavior. The presentation covers the structure of the competition and the outcome of its first instantiation in an actual event, and an analysis of the purpose, design, and appropriateness of such a competition." Also available from the author's collection of publications is his rejoinder to Loebner's response.
Alan Mathison Turing. From the MacTutor History of Mathematics Archive, School of Mathematical and Computational Sciences, University of St. Andrews, Fife, Scotland. Includes a biographical sketch, a list of references, and more!
Alan Turing Home Page. Maintained by Andrew Hodges, author, Alan Turing: The Enigma. Includes biographical information on Turing as well as description of the Turing Test.
The Turing Archive for the History of Computing. Maintained by Jack Copeland and Gordon Aston. In addition to what you'd expect to discover in an archive, you'll also find an wide ranging collection of reference articles, such as What is a Turing Machine?, What is Artificial Intelligence?, and Turing's Last Programs.
Turing Digital Archive. "This digital archive contains mainly unpublished personal papers and photographs of Alan Turing from 1923-1972. The originals are in the Turing archive in King's College Cambridge. This is a pilot project which will hopefully be extended to the whole collection in the future."
The Age of Intelligent Machines: "A (Kind of) Turing Test". By Ray Kurzweil. "The essence of the Turing Test is that the computer attempts to act like a human within the context of an interview over terminal lines. A narrower concept of a Turing test is for a computer to successfully imitate a human within a particular domain of human intelligence. We might call these domain-specific Turing tests. One such domain-specific Turing test, based on a computer's ability to write poetry [including haiku], is presented here."
The Loebner Prize: "The First Turing Test." "Dr. Loebner pledged a Grand Prize of $100,000 and a Gold Medal for the first computer whose responses were indistinguishable from a human's. Each year an annual prize of $2000 [NOTE: the prize will be $3000 for the 2005 contest] and a bronze medal is awarded to the most human computer. The winner of the annual contest is the best entry relative to other entries that year, irrespective of how good it is in an absolute sense. Further information on the development of the Loebner Prize and the reasons for its existence is available in Loebner's article In Response to the article Lessons from a Restricted Turing Test by Stuart Shieber." Contest information, past winners and transcripts can be accessed via links that can be found by scrolling down the Loebner Prize home page.
The Blurring Test. "Turning the Turing Test upside down, MRMIND challenges you to prove to him that you are human. Can you claim that your 'human' attributes will forever be exclusively human? The Blurring Test is about human progress: Someday it might be important to convince our computers (and each other) that we are human." Visit the site and converse with the chatterbot, MR MIND.
The Turing Test. Part of Contemporary Philosophy of Mind: An Annotated Bibliography. Compiled by David J. Chalmers, Department of Philosophy, Washington University, St. Louis MO.
The Turing Test Page. Maintained by Ayse Pinar Saygin.
Turing Tournament @ Caltech. "We would like to announce the beginning of the 'Turing Tournament', and to invite you to participate in it. The Turing Tournament is a two sided tournament designed to find, on the one hand, the best computer programs to mimic human behavior, and on the other hand, the best computer programs to detect the difference between machine and human behavior. Two types of submissions will be accepted: an emulator, which generates a dataset that mimics human behavior, or a detector, which detects the difference between datasets generated by human and machine behavior."
Turing's World. Logic Software from CSLI. By Jon Barwise and John Etchemendy. Be sure to also check out their page about Turing Machines.
- General Index to AI in the news: Turing Test
- Namesakes: Alan Turing
- Natural Language Processing (where you'll find more Chatterbots)
- NLP Understanding & Generation
- Nature of Intelligence
- Philosophy
Alper, G. 1990. A Psychoanalyst Takes the Turing Test. Psychoanalytic Review 77 (1): 59-68.
Bleich, H. L. 1995. Alan Turing: The Machine, the Enigma, and the Test. MD Computing 12 (5): 330.
Cohen, Paul R. 1995. Empirical Methods for Artificial Intelligence. Cambridge, MA: MIT Press. An in-depth treatment of statistical measures of performance for AI programs.
Colby, K., S. Weber, and F. Hilf 1971. Artificial Paranoia. Artificial Intelligence 2: 1-25.
Crockett, Larry J. 1994. The Turing Test and the Frame Problem. Norwood, NJ: Ablex Publishing Co.
Dennett, Daniel. 1998. Brainchildren. Contains an excellent discussion of the Turing Test. Cambridge, MA: MIT Press.
Epstein, Robert. 1992. The Quest for the Thinking Computer. AI Magazine 13 (2): 81-95. "Can machines think? Alan Turing's decades-old question still influences artificial intelligence because of the simple test he proposed. In this article, AI Magazine collects presentations from the first round of the competition, held November 8, 1991 at The Computer Museum, Boston. Robert Epstein, Director Emeritus, Cambridge Center for Behavioral Studies, summarizes some of the difficult issues during the planning of this first real-time competition, and describes the event. Article is accompanied by the transcript of the session that won the Loebner Prize Competition--Joseph Weintraub's computer program PC Therapist."
Ford, Kenneth, and Patrick Hayes. 1998. On Computational Wings: Rethinking the Goals of Artificial Intelligence. Scientific American Presents 9 (4): 78-83. Drawing the analogy that airplanes fly without mimicking birds, the authors conclude that AI can and already has contributed via machine intelligence, and that requiring it to mimic human intelligence is unecessary and even harmful to scientific research.
Hayes, Patrick, and Kenneth Ford. 1995. Turing Test Considered Harmful. Proceedings of the IJCAI-95, August 20-25, 1995, in Montreal, Quebec. Volume 1, page 972. The authors, who consider Turing to be one of the greatest scientists of the 20th century, examine problems with the Turing Test and conclude that a goal of passing the test is harmful to the field of AI research.
Heiser, J. F., K. M. Colby, W. S. Faught, et al. 1979. Can Psychiatrists Distinguish a Computer Simulation From the Real Thing? The Limitations of Turing-like Tests as Measure of the Adequacy of Simulations. Journal of Psychiatric Research 15 (3): 149-162.
Hodges, Andrew. 1983. Alan Turing: The Enigma of Intelligence. New York: Simon & Schuster. Johnson, George. 1997. The Artist's Angst is All in Your Head. New York Times, (Late NY Edition/November 16, 1997): Section 4, p.16.
Millican, Peter and Andy Clark, editors. 1999. Machines and Thought: The Legacy of Alan Turing, Volume I. Oxford University Press - USA. The Table of Contents is available online.
Putnam, Hilary. 1988. Much Ado About Not Very Much. Daedalus 117 (1): 269-281. Reprinted in The Artificial Intelligence Debate: False Starts, Real Foundations, edited by Stephen Graubard, 1988.
Quittner, Joshua. 1997. What's Hot in Bots. Time Magazine 150: 31.
Shortliffe, Edward H. 1984. The Problem of Evaluation. In Rule Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project, ed. Buchanan, Bruce G. and Edward H. Shortliffe, 571-589 (Chapter 30). Reading, MA: Addison-Wesley Publishing Co., Inc.
Waltz, David L. 1988. The Prospects for Building Truly Intelligent Machines. In The Artificial Intelligence Debate, ed. Graubard, Stephen R., Cambridge, MA: The MIT Press.
Yu, Victor L., Lawrence M. Fagan, Sharon Wraith Bennett, et al. 1979. An Evaluation of MYCIN's Advice. In Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project, ed. Buchanan, Bruce G. and Edward H. Shortliffe, 589-596 (Chapter 31). Reading, MA: Addison-Wesley Publishing Co., Inc., 1984. (Originally published in 1979 in the Journal of the American Medical Association 242: 1279-1282).