Good Places to Start

Software Tutors Offer Help and Customized Hints. By Katie Hafner. The New York Times (September 16, 2004; subscription req'd.). "As she sat at a computer screen, she kept typing 2.8, an incorrect answer. Eventually a hint popped up: 'Think about the sign of your answer.' When Rochelle finally typed the correct sum, -1.8, the computer showed its appreciation by allowing her to move on to a new problem. She smiled at her small triumph. Since January, Middle School 301 in the Bronx, where Rochelle is an eighth grader, has been using a software program called Cognitive Tutor to help students learn math. The software, from Carnegie Learning, a six-year-old company that got its start at Carnegie Mellon University, is designed to give students individualized instruction when personal attention is scarce. Although such intelligent tutoring systems have their share of skeptics, students at schools that use them have not only improved their performance in math but now profess to enjoy a subject they once loathed. ... Broadly defined, an intelligent tutoring system is educational software containing an artificial intelligence component. The software tracks students' work, tailoring feedback and hints along the way. By collecting information on a particular student's performance, the software can make inferences about strengths and weaknesses, and can suggest additional work. When Rochelle, for instance, displayed a weakness when working with negative numbers, the program repeatedly asked her to solve similar problems. ... The artificial intelligence built into the Carnegie Learning program helps set it apart. Not only does the program present drills according to a student's weaknesses, but it watches the work step by step, detecting where the student stumbles, and chimes in when necessary."

Transforming America's Schools. By Jonathan Potts. Carnegie Mellon Today (November 2005). "The technology that drives intelligent tutoring systems is grounded in research into artificial intelligence and cognitive psychology, which seeks to understand the mechanisms that underlie human thought, including language processing, mathematical reasoning, learning and memory. As students perform problems using these tutoring systems, the program analyzes their strengths and weaknesses and on that basis provides individualized instruction. Intelligent tutoring systems do not replace teachers. Rather, they allow teachers to devote more one-on-one time to each student and to work with students of varying abilities simultaneously."

Intelligent Tutoring, Intelligent Instruction and Artificial Intelligence background information and a related collection of Intelligent Tutoring and Artificial Intelligence Resources from the Human Performance Center's HPC SPIDER, "the Navy's premier online resource for human performance and training technology for lifelong learning."

High-tech tools help with FCAT - Students can access online tutors and test aids that monitor individual progress. By Beth Kormanik. The Times-Union & Jacksonville.com (September 20, 2004)." Effective personal tutors can raise student scores by two grade levels, [Ken Koedinger, professor of human-computer interaction and psychology at Carnegie Mellon University] said, but the average human tutor helps raise grade level only by one-half. His computer-based system falls in between, raising students' scores by one grade level."

Artificial Intelligence. By Kristen Kennedy. Technology & Learning (November 2002). "They don't do windows -- but the next generation of AI applications can teach, tutor, and even grade essays." This is just one of the articles that is part of this issue's cover story: Top 10 Smart Technologies for Schools.

Experts Use AI to Help GIs Learn Arabic. By Eric Mankin. USC News (June 21, 2004). " To teach soldiers basic Arabic quickly, USC computer scientists are developing a system that merges artificial intelligence with computer game techniques. The Rapid Tactical Language Training System, created by the USC Viterbi School of Engineering's Center for Research in Technology for Education (CARTE) and partners, tests soldier students with videogame missions in animated virtual environments where, to pass, the students must successfully phrase questions and understand answers in Arabic. ... 'Most adults find it extremely difficult to acquire even a rudimentary knowledge of a language, particularly in a short time,' said CARTE director W. Lewis Johnson. 'We’re trying to build an improved model of instruction, one that can be closely tailored to both the needs and the abilities of each individual student,' Johnson said." Read the story and then watch the video!

Artificial intelligence alive and well. The University of Auckland News (January 19, 2005). "While statistics students at The University of Auckland are taking a break from studies for summer, their new 'teacher' can’t wait for the new semester to begin. Maria, an assistant teacher in Statistical Interference, is an unusual individual. She looks to be in her mid-twenties but her age, she says, cannot be computed in human years. With a vocabulary of 203,000 words, a repertoire of 106,000 grammatical rules and 118,000 rules of logical inference, Maria is capable of conversation at quite a complex level. Maria is a robot, or artificial intelligence entity, created over two years of intense work and study by Shahin Maghsoudi, a PhD student and member of the Artificial Intelligence Group in the Faculty of Science. As part of his Masters degree in Computer Science, Shahin embarked on a project to create virtual robots which could be used as teaching assistants, helpdesk operators and web-based marketing assistants."

Intelligent Tutoring Systems. A brief introduction by Eric Thomas. Part of San Diego State University's Encyclopedia of Educational Technology.

Applications of AI in Education. By Joseph Beck, Mia Stern, and Erik Haugsjaa. ACM Crossroads (student magazine of the Association for Computing Machinery), 1996. "In this paper, we start by providing an overview of the main components of intelligent tutoring systems. We then provide a brief summary of different types of ITSs. Next, we present a detailed discussion of two components, the student model and the pedagogical module. We close by discussing some of the open questions in ITS as well as future directions of the field."

The Roles of Artificial Intelligence in Education: Current Progress and Future Prospects. David McArthur, Matthew Lewis, and Miriam Bishay. (1993) RAND DRU-472-NSF. A very good overview with lots of basic information about intelligent tutoring systems.

Intelligent Tutoring Systems: The What and the How. By Jim Ong and Sowmya Ramachandran. From the February 2000 edition of Learning Circuits, a Webzine About E-Learning from the American Society for Training & Development (ASTD). "Imagine that each learner in a classroom or WBT setting has a personal training assistant who pays attention to the participant's learning needs, assesses and diagnoses problems, and provides assistance as needed. ... Providing a personal training assistant for each learner is beyond the training budgets of most organizations. However, a virtual training assistant that captures the subject matter and teaching expertise of experienced trainers provides a captivating new option. The concept, known as intelligent tutoring systems (ITS) or intelligent computer-aided instruction (ICAI), has been pursued for more than three decades by researchers in education, psychology, and artificial intelligence."

A teacher who gets by on artificial intelligence" (International Herald Tribune and Israeli Haaretz Daily, 12/20/98) and "Intelligent agents help humans learn from computers" (CNN Interactive, 8/25/97) are just two of the exciting articles about Pedagogical Agents and Guidebots that you'll find at CARTE's very informative site. [CARTE = The Center for Advanced Research in Technology for Education which is part of the Information Sciences Institute at the University of Southern California.] Be sure that you don't miss the demos and videos, or their many pedagogical agents and guidebots (see: research and projects).

Readings Online

Automated Essay Evaluation: The Criterion Online Writing Service. By Jill Burstein, Martin Chodorow, and Claudia Leacock. AI Magazine 25(3): Fall 2004, 27-36. "In this article, we describe a deployed educational technology application: the Criterion Online Essay Evaluation Service, a web-based system that provides automated scoring and evaluation of student essays. Criterion has two complementary applications: (1) CritiqueWriting Analysis Tools, a suite of programs that detect errors in grammar, usage, and mechanics, that identify discourse elements in the essay, and that recognize potentially undesirable elements of style, and (2) e-rater version 2.0, an automated essay scoring system. Critique and e-rater provide students with feedback that is specific to their writing in order to help them improve their writing skills and is intended to be used under the instruction of a classroom teacher. Both applications employ natural language processing and machine learning techniques. All of these capabilities outperform baseline algorithms, and some of the tools agree with human judges in their evaluations as often as two judges agree with each other."

Artificial Intelligence as Tutor. By Josh Chamot. National Science Foundation Office of Legislative and Public Affairs News Tip (May 6, 2002). "Inspired by the methods of his rural Kentucky high-school chemistry teacher, an NSF-supported researcher has developed an artificial intelligence tutoring software that helps students confront complex science questions. In 1998, chemist Benny Johnson founded Quantum Simulations, Inc. with high school mentor Dale Holder and colleague Rebecca Renshaw to create highly interactive tutoring software for the sciences. Many pre-existing tutoring programs store information in a database and do not allow for student input beyond multiple choice answers or simple responses. In contrast, the Quantum Tutors 'converse' with students, said Johnson, 'providing real-time feedback . . . Tutors respond to student questions, give hints, show correct next steps and even explain why a step is correct or incorrect using scientific principles.'"

• Visit the Quantum Simulations web site.

Intelligent Tutoring Systems: a video of Cristina Conati's talk at CSE Colloquia - 2005, The University of Washington Computer Science & Engineering Colloquium Series, available from the ResearchChannel ("a non-profit organization founded in 1996 by a consortium of leading research universities, institutions and corporate research centers dedicated to creating a widely accessible voice for research through video and Internet channels").

The Love Machine - Building computers that care. By David Diamond. Wired Magazine (December 2003). "Intelligent tutoring systems are not new, but they are limited; unlike flesh-and-blood tutors, they can't tell if you're bored, frustrated, engrossed, or angry and then adjust the teaching accordingly. That's why MIT has been working to add such capability to two systems. One, an automated reading tutor, was developed by Jack Mostow, a Carnegie Mellon computer science professor. The system, which is helping hundreds of students learn to read, was used in a recent study proving the positive effects of praise and encouragement. The other, AutoTutor, was built by University of Memphis professor Arthur Graesser and his Tutoring Research Group and is used by U of M students. Designed to observe and respond to a student's cognitive state, AutoTutor relies on Latent Semantic Analysis, a natural language parser that analyzes the sentences you type in and figures out how much you know by contrasting your semantics against an internal model of an ideal student. A clever animated avatar spits back information to fill in the gaps in your understanding."

Talking Up a Good Game - Computer Simulation to Stimulate Soldiers to Speak in Tongues. By Paul Eng. ABCNEWS.com (March 9, 2004). "The first part of the game, says [Lewis] Johnson, acts as basically an 'intelligent tutoring' program.' ... But what makes the program really 'intelligent' are the computer-generated and -controlled characters, such as a virtual village leader and a virtual 'team member' that acts as an in-game guide. These game characters are programmed to react in ways that are unique to each individual user."

• visit the Tactical Language Project at CARTE

Encouraging Student Reflection and Articulation using a Learning Companion. By Bradley Goodman, Amy Soller, Frank Linton, and Robert Gaimari (1998). International Journal of Artificial Intelligence in Education, 9(3-4). "The goal of the research presented in this paper is to promote more effective instructional exchanges between a student and an intelligent tutoring system The approach taken to meet this goal involves providing a simulated peer as a partner for the student in learning and problem solving. The learning companion described in this paper enhances learning by initiating a dialogue with a student forcing reflection and articulation on the student's learning."

Intelligent Tutoring Systems with Conversational Dialogue. By Arthur C. Graesser, Kurt VanLehn, Carolyn P. Rose, Pamela W. Jordan, and Derek Harter (2001). AI Magazine 22(4): 39-52. "We have been working on a new generation of intelligent tutoring systems that hold mixed-initiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AutoTutor is a conversational agent, with a talking head, that helps college students learn about computer literacy. andes, atlas, and why2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations."

• visit the AutoTutor web site

Using technology for learning & teaching science. IST Results (November 3, 2004). "Researchers are demonstrating how technologies when applied to science learning can help motivate and engage pupils and promote better take-up of scientific disciplines at school and university. The following eight IST research projects are focusing on technology-enhanced learning methods, in subjects as varied as astronomy, space research, physics, mathematics and the earth sciences. ... A learner-centred approach is the tack taken by the LeActiveMath project. It aims to design a third generation intelligent learning environment to support Web-based active learning in maths, adapted to the needs and context of the learner by offering interactivity and personalisation. This 36-month project that started in January 2004 builds on its successful forerunner, ActiveMath. Learner feedback from this earlier project revealed that 'students like a lot of interactivity in exercises and benefit from it,' says Erica Melis, the coordinator of LeActiveMath at the German Research Center for Artificial Intelligence. While ActiveMath had some of these features, LeActiveMath will offer much more. ... It will provide intelligent feedback and involve the student in tutorial dialogues that stimulate the student to think rather than learn by heart. 'Dialogues are a natural way to communicate and human-centred dialogues are known to improve learning,' says Melis."

• Visit the LeActiveMath web site.

Pitch-perfect PC - Software that turns a computer into a smart, sensitive practice partner for music students. By Alex Markels. U.S. News & World Report (March 17, 2003).

The F-16 Maintenance Skills Tutor. By Christopher Marsh. The Edge - The MITRE Advanced Technology Newsletter (March 1999). "How do you keep technicians trained to repair systems that are highly reliable? ... With the downsizing of the Air Force, there are fewer technicians per aircraft and many of the experienced technicians are retiring leaving fewer people to train novices. In response to this need, research was performed in two areas: cognitive task analysis techniques to capture troubleshooting strategies used by experts and novices, and intelligent tutoring systems that take the results of the cognitive task analysis to provide a practice environment for working authentic troubleshooting problems while coaching the student with hints and feedback. The result of this research is the F-16 Maintenance Skills Tutor. Using this type of tutor for 20 hours is equivalent to 3.5 to 4 years of experience on the flight line."

City pushes computer tutor for struggling algebra students. By Maggi Newhouse. Tribune-Review (March 8, 2004)/ available from PittsburghLIVE.com. "About 40 percent of the city's ninth graders fail first-year algebra every year, and Pittsburgh Public Schools officials say it's time to expand an innovative math program used by some schools to the rest of the district. ... The centerpiece of the Carnegie Learning method, developed by Carnegie Mellon University researchers, is a computer program that combines traditional algebra problems with technology that can assess a student's progress and skill level. The Cognitive Tutor program can then use the student information to offer individualized instruction and provide instant feedback for a student and teacher. 'What you're seeing here is artificial intelligence,' said Jackie Smith, an instructional support director for mathematics. 'The computer is learning and building a profile of every single student as it diagnoses their strengths and weaknesses.'"

• visit the Cognitive Tutor site

How to Fix America's Schools. By William C. Symonds. BusinessWeek Magazine. (March 19, 2001) "A new generation of software is proving far more effective than traditional programs, which are often little more than rote learning dressed up for the Digital Age. Take the Cognitive Tutor, developed by Carnegie Learning in Pittsburgh to teach algebra and geometry. The program uses artificial intelligence to determine what students understand and what they need to tackle next. Rather than drill kids on equations, it requires them to use algebra to solve real problems. Kids using Cognitive Tutor score higher on math tests than students in traditional algebra classes and are more than twice as likely to complete geometry and higher algebra."

Related Web Sites

"Adaptive Training Systems (ATSs), SHAI's proprietary versions of intelligent tutoring systems (ITSs), can dramatically reduce the cost of education while offering many of the benefits of one-to-one instruction. ATS's dynamically and adaptively monitor individual students in learning specific principles as they perform exercises. ATSs use their monitoring capability to ensure each student is always presented with material that is not so easy that he is bored, or material that is above his ability to learn and frustrates him. This monitoring also enables our systems to detect misconceptions and provide each student with individualized instruction."

"ARIES, the Laboratory for Advanced Research in Intelligent Educational Systems [at the University of Saskatchewan], is a focal point for research projects in the areas of intelligent tutoring systems and adaptive learning environments. The mission of the ARIES Laboratory is to advance the development of Learning Technologies through the integration of Artificial Intelligence techniques and to advance Artificial Intelligence research through attempts to solve real-world education and training problems." Be sure to check out their many projects, both present and past.

"AutoTutor is an intelligent tutoring system developed by an interdisciplinary research team.This team is currently being funded by the Office of Naval Research and the National Science Foundation and is is comprised of approximately 35 researchers from psychology, computer science, linguistics, physics, engineering, and education. ... AutoTutor works by having a conversation with the learner. AutoTutor appears as an animated agent that acts as a dialog partner with the learner. The animated agent delivers AutoTutor's dialog moves with synthesized speech, intonation, facial expressions, and gestures. Students are encouraged to articulate lengthy answers that exhibit deep reasoning, rather than to recite small bits of shallow knowledge."

CIRCLE: Center for Interdisciplinary Research on Constructive Learning Environments. "CIRCLE is an NSF-funded research center located at the University of Pittsburgh and Carnegie Mellon University, with multiple partnerships among schools, industries and other research institutions. CIRCLE's mission is to determine why highly effective forms of instruction, such as human one-on-one tutoring, work so well, and to develop computer-based constructive learning environments that foster equally impressive learning." Be sure to follow the links to "Projects" for that's where you'll find systems such as:

• Andes: An Intelligent Tutoring System for Classical Physics
• ITSPOKE: Spoken Dialogue for Intelligent Tutoring Systems

The EPSILON [Encouraging Positive Social Interaction while Learning ON-Line] Project at the Learning Research and Development Center, University of Pittsburgh "is an interdisciplinary effort to provide dynamic, adaptive support for on-line learning communities. The support, in the form of an intelligent software agent, will focus on helping students improve their social and communication management skills. ... The EPSILON software will be driven by a computational model of effective learning interaction. The project will explore methods for dynamically analyzing on-line interaction during structured learning activities. Artificial Intelligence techniques will be employed for analyzing, studying, and characterizing on-line interaction."

"The ICICLE system (Interactive Computer Identification and Correction of Language Errors) is an intelligent tutoring system under development in the NLP/AI Group of the CIS Department of the University of Delaware. The primary goal of ICICLE is to employ natural language processing and generation to tutor deaf students on their written English."

Journals and Conferences related to ITS ... and workshops too! Compiled by the Tutor Research Group at Worcester Polytechnic Institute.

Journals on Intelligent Tutoring Systems & Textbooks on Intelligent Tutoring Systems. Compiled by Noboru Matsuda, Postdoctoral Research Fellow at the Human Computer Interaction Institute, School of Computer Science, Carnegie Mellon University.

METUTOR: A means-end tutoring system. From Prof. Neil C. Rowe, Department of Computer Science, U.S. Naval Postgraduate School. "METUTOR is a tutoring environment for teaching of procedural skills. It uses planning methods from artificial intelligence to infer what a student is doing, and tutors intelligently when the students diverges from the correct plan. ... The teacher's job is further simplified through with the use of the MEBUILD expert-system shell under development, which uses an object-oriented world description to infer most of the necessary action specification for METUTOR. MEBUILD also allows a teacher to reuse parts of one lesson in another."

Pedagogical Agents and Learning Systems (PALS) Research Group members at the Center for Research of Innovative Technologies for Learning (RITL), Florida State University, "investigate the affordances and constraints of animated pedagogical agents within eLearning environments. What is a 'pedagogical agent?' (As stated by W. Lewis Johnson): Pedagogical agents are autonomous agents that support human learning, by interacting with students in the context of interactive learning environments. They extend and improve upon previous work on intelligent tutoring systems in a number of ways. ..."

Project Listen: A Reading Tutor That Listens. By Jack Mostov, project director. "The Reading Tutor adapts Carnegie Mellon's state-of-the art Sphinx-II speech recognizer to analyze the student's oral reading. The Reading Tutor intervenes when the reader asks for help, makes mistakes, gets stuck, or is likely to encounter difficulty. The Reading Tutor responds with assistance modelled in part after expert reading teachers, but adapted to the capabilities and limitations of the technology. A successful computer reading tutor that uses speech recognition to listen to children read. The computer tutor intervenes when students ask for help or make mistakes. Links to a bibliography that includes abstracts of articles and some full text."

• Project LISTEN was selected as one of the National Science Foundation's nifty50: "The nifty50 are NSF-funded inventions, innovations and discoveries that have become commonplace in our lives. This interactive section of the Web site allows visitors to click on each innovation and explore it in greater depth." After clicking here, click on nifty50 and then go to number 40.

Virtual Environments for Training (VET). Information Sciences Institute, University of Southern California. Description of a project in intelligent tutoring, with access to many online papers.

Related Pages

• Assistive Technologies
• Education
• General Index to AI in the news - Education
• Interfaces

More Readings

Friedland, Noah S. and Paul G. Allen, Gavin Matthews, Michael Witbrock, David Baxter, Jon Curtis, Blake Shepard, Pierluigi Miraglia, Jürgen Angele, Steffen Staab, Eddie Moench, Henrik Oppermann, Dirk Wenke, David Israel, Vinay Chaudhri, Bruce Porter, Ken Barker, James Fan, Shaw Yi Chaw, Peter Yeh, Dan Tecuci, Peter Clark. 2004. Project Halo: Towards a Digital Aristotle. AI Magazine 25(4): 29-47. Abstract: "Project Halo is a multistaged effort, sponsored by Vulcan Inc, aimed at creating Digital Aristotle, an application that will encompass much of the world’s scientific knowledge and be capable of applying sophisticated problem solving to answer novel questions. Vulcan envisions two primary roles for Digital Aristotle: as a tutor to instruct students in the sciences and as an interdisciplinary research assistant to help scientists in their work. As a first step towards this goal, we have just completed a six-month pilot phase designed to assess the state of the art in applied knowledge representation and reasoning (KR&/R). Vulcan selected three teams, each of which was to formally represent 70 pages from the advanced placement (AP) chemistry syllabus and deliver knowledge-based systems capable of answering questions on that syllabus. The evaluation quantified each system’s coverage of the syllabus in terms of its ability to answer novel, previously unseen questions and to provide human- readable answer justifications. These justifications will play a critical role in building user trust in the question-answering capabilities of Digital Aristotle. Prior to the final evaluation, a 'failure taxonomy' was collaboratively developed in an attempt to standardize failure analysis and to facilitate cross-platform comparisons. Despite differences in approach, all three systems did very well on the challenge, achieving performance comparable to the human median. The analysis also provided key insights into how the approaches might be scaled, while at the same time suggesting how the cost of producing such systems might be reduced. This outcome leaves us highly optimistic that the technical challenges facing this effort in the years to come can be identified and overcome. This article presents the motivation and longterm goals of Project Halo, describes in detail the six-month first phase of the project -- the Halo Pilot -- its KR&R; challenge, empirical evaluation, results, and failure analysis. The pilot’s outcome is used to define challenges for the next phase of the project and beyond."

Graesser, Arthur C. and Kurt VanLehn, Carolyn P. Rose, Pamela W. Jordan, and Derek Harter. 2001. Intelligent Tutoring Systems with Conversational Dialogue. AI Magazine 22(4): 39-52. "Many of the intelligent tutoring systems that have been developed during the last 20 years have proven to be quite successful, particularly in the domains of mathematics, science, and technology. They produce significant learning gains beyond classroom environments. They are capable of engaging most students' attention and interest for hours. We have been working on a new generation of intelligent tutoring systems that hold mixed-initiative conversational dialogues with the learner. The tutoring systems present challenging problems and questions to the learner, the learner types in answers in English, and there is a lengthy multiturn dialogue as complete solutions or answers evolve. This article presents the tutoring systems that we have been developing. AutoTutor is a conversational agent, with a talking head, that helps college students learn about computer literacy. andes, atlas, and why2 help adults learn about physics. Instead of being mere information-delivery systems, our systems help students actively construct knowledge through conversations."

Murray, Tom. 1999. Authoring Intelligent Tutoring Systems: An analysis of the state of the art. International Journal of Artificial Intelligence in Education (1999), 10: 98-129. "This paper consists of an in-depth summary and analysis of the research and development state of the art for intelligent tutoring system (ITS) authoring systems. A seven-part categorization of two dozen authoring systems is given, followed by a characterization of the authoring tools and the types of ITSs that are built for each category. An overview of the knowledge acquisition and authoring techniques used in these systems is given. A characterization of the design tradeoffs involved in building an ITS authoring system is given. Next the pragmatic questions of real use, productivity findings, and evaluation are discussed. Finally, I summarize the major unknowns and bottlenecks to having widespread use of ITS authoring tools."