There are fundamental questions to be answered about the architecture of a visual system. For nearly two decades, the field has assumed that the visual system can be decomposed into independent modules, each performing a well-defined function, like estimating color, and that their outputs are integrated at a later stage. Is this a valid hypothesis? -Harry G. Barrow & J. M. Tenenbaum, Retrospective on "Interpreting Line Drawings as 3-Dimensional Surfaces"

Vision involves both the acquisition and processing of visual information. AI powered technologies have made possible such astounding achievements as vehicles that are able to safely steer themselves along our superhighways, and computers that can recognize and interpret facial expressions. Just consider the complexity of the analysis that your brain must engage in before determining that something as apparently simple as the fact that the black squares on a chess board are not holes, but rather part of the surface, and you'll have an idea of how sophisticated vision systems must be in order to reliably perform their objectives.

And there is so much more to vision than meets the eye, such as when fog or snow obscures a portion of the road ahead of you (or imagine that you are scuba diving in murky waters...). Just as you are able to fill in the missing pieces of vaguely defined areas based upon experience and your general knowledge of the environment, AI programs make possible the enhancement, interpretation, recognition, identification and other processing of partial images.

AI vision technology has made possible such applications as: image stabilization, 3D modeling, image synthesis, surgical navigation, handwritten document recognition, and vision based computer interfaces. Follow the links below to see what vision projects AI scientists are currently working on.

Good Places to Start

Machine learns games 'like a human.' By Will Knight. New Scientist News (January 24, 2005). "A computer that learns to play a 'scissors, paper, stone' by observing and mimicking human players could lead to machines that automatically learn how to spot an intruder or perform vital maintenance work, say UK researchers. CogVis, developed by scientists at the University of Leeds in Yorkshire, UK, teaches itself how to play the children's game by searching for patterns in video and audio of human players and then building its own 'hypotheses' about the game's rules. In contrast to older artificial intelligence (AI) programs that mimic human behaviour using hard-coded rules, CogVis takes a more human approach, learning through observation and mimicry, the researchers say. ... 'A system that can observe events in an unknown scenario, learn and participate just as a child would is almost the Holy Grail of AI,' says Derek Magee from the University of Leeds." Be sure to see the sidebar with related articles & web sites.

Computer system said to help stop drowning. By Ed Frauenheim. CNET News.com (January 31, 2005). "A man swimming in a pool near Paris almost drowned last week but was rescued with the help of a computer vision surveillance system, the maker of the system said. The Poseidon drowning-detection system also helped lifeguards save the life of a teenager in France who nearly drowned in 2000, and last year it helped lifeguards in Germany rescue an elderly man who nearly drowned after a heart attack, said Poseidon's maker, Vision IQ. ... Poseidon is a computer vision surveillance system designed to recognize texture, volume and movement within a pool."

Recent advances in computer vision. By Massimo Picardi and Tony Jan. The Industrial Physicist (February/March 2003; Volume 9, Number 1). "Computer vision is the branch of artificial intelligence that focuses on providing computers with the functions typical of human vision. To date, computer vision has produced important applications in fields such as industrial automation, robotics, biomedicine, and satellite observation of Earth. ... The availability of affordable hardware and software has opened the way for new, pervasive applications of computer vision."

Computer Vision. By Chris. Bowerman of the University of Sunderland. "This document aims to guide you through some of the jargon used in this new technology and help you gain insight into how vision systems work. We will review some of the principals, techniques and technologies available and try to discover if the technology exists to build a machine with vision capabilities anywhere near as advanced as the eyesight that we as humans posses. We will illustrate our investigation more clearly by way of a couple of examples. Firstly that of a production line quality control system : how the image itself is captured and stored, and how the image is interpreted. Then as a more complex example how a vision system might be achieved for an automated system such as 'robot' : what would be required to make it truly aware of its complex and changing environment."

Computer Vision Homepage. Computer Science Department, Carnegie Mellon University. This site is a clearinghouse for information on everything about computer vision research. Links to groups, papers, software and more . . . and be sure to see their collection of Computer Vision Online Demos.

Computer Vision (CV) Online. By Robert Fisher, Division of Informatics, University of Edinburgh. A collection of hypertext summaries on central topics in the field of computer vision. Beginners may be especially interested in a terrific 'Applications' page as well as one covering 'Famous Vision Systems'.

Machine Vision Glossaries of Terms:

• This glossary from Image Labs International covers everything from Aberration to Zoom Lens.
• and this one from the Automated Imaging Association spans Acquisition to Verification.

IBM gets smart about Artificial Intelligence. By Pamela Kramer. IBM Think Research (June 2001). "Computer vision is important to speech recognition, too. Visual cues help computers decipher speech sounds that are obscured by environmental noise. Chalapathy Neti, manager of IBM's audiovisual speech technologies (AVST) group at Watson, often cites HAL's lip-reading ability in 2001 in promoting the group's work."

• IBM Exploratory Computer Vision Group
• Also see our Speech and Natural Language pages

Applications of Machine Vision. A very interesting collection from The British Machine Vision Association and Society for Pattern Recognition.

Readings Online

Machine Vision Online. From AIA (Automated Imaging Association). News, feature articles, archives, a glossary, and a very comprehensive collection of machine vision market data.

Vision. An AI Qual Summary by Patrick Doyle. A very comprehensive (and sometimes quite technical) overview.

Computers Seeing People. By Ifran A. Essa. AI Magazine 20(2): Summer 1999, 69-82. "AI researchers are interested in building intelligent machines that can interact with them as they interact with each other. Science fiction writers have given us these goals in the form of HAL in 2001: A Space Odyssey and Commander Data in Star Trek: The Next Generation. However, at present, our computers are deaf, dumb, and blind, almost unaware of the environment they are in and of the user who interacts with them. In this article, I present the current state of the art in machines that can see people, recognize them, determine their gaze, understand their facial expressions and hand gestures, and interpret their activities. I believe that by building machines with such abilities for perceiving, people will take us one step closer to building HAL and Commander Data."

Giving robots the gift of sight. By Ed Frauenheim. CNET (December 30, 2002). "Hans Moravec has completed work on a three-dimensional robotic vision system he says will allow machines to make their way through offices and homes. ... Moravec's system consists of stereoscopic digital cameras and a 3D grid set up in the robot's computer brain."

Getting computer vision systems to recognise reality. IST Results (September 21, 2004). "Enabling cognitive computer vision systems to emulate human capabilities is the driving force behind the VAMPIRE project, as will be demonstrated at the IST2004 event with its object localisation via hybrid tracking methods, context-aware scene augmentation and interactive object learning. To date, computer vision systems have been unable to emulate the full capabilities of the human visual system. The human eye-brain combination has proved able to categorise previously unseen objects with ease, using background knowledge and context. We recognise a pig as a pig because of the shape of its body and because we see it in a farmyard or field. ... [T]he VAMPIRE project seeks to enable cognitive computer vision systems to develop similar capabilities."

• Visit Vampire "a research project on cognitive computer vision which is funded by the European Union. On the one hand, it investigates Visual Active Memory Processes; its other important goal is to develop advanced techniques for Interactiv REtrieval
• Project Objectives: "The aim of the project VAMPIRE is embedded in the framework that has been recently proposed in the last meeting on Cognitive Vision Systems: "Integration of intelligence into vision systems will transform them from processing engines that deliver information, to perception engines that acquire, maintain and deliver selective knowledge, i.e., to cognitive vision systems. A cognitive vision system may therefore be said to combine the acquisition and processing of visual data relating to objects, processes or events, and the mobilisation of specific knowledge for reasoning, establishing conclusions and making decisions. Put another way, it is a vision system with the capability of perceiving and interacting with its environment. ... The methods and technologies that facilitate reasoning in response to visual perception come mostly from the domain of Artificial Intelligence: *Intelligent agents.... *Self-adaptive software.... *Qualitative reasoning.... *Methods for memory and knowledge organisation and the intelligent deployment of that knowledge...."

Seeing is Believing: Computer Vision and Artificial Intelligence. By Christopher O. Jaynes. ACM Crossroads (the student magazine of the Association for Computing Machinery), 1996. "The importance of computer vision to the field of AI is fairly obvious: intelligent agents need to acquire knowledge of the world through a set of sensors. What is not so obvious is the importance that AI has to the field of computer vision. Indeed, I believe that the study of perception and intelligence are necessarily intertwined. This article will look at the role that knowledge plays in computer vision and how the use of reasoning, context, and knowledge in visual tasks reduces the complexity of the general problem."

Vision in Bad Weather. By S. K. Nayar, S. G. Narasimhan, Proceedings of IEEE International Conference on Computer Vision, Corfu, Greece, September 1999.

Bugs Taking Over Robot Guidance. By Lakshmi Sandhana. Wired News (January 14, 2004). "Large UAVs that fly at high altitudes employ sensing mechanisms based on GPS or radar technologies, but those methods fail when it comes to scaled-down vehicles with smaller wingspans.... To create intelligent artificial-vision packages that weigh only a few grams and contain all the necessary optics, hardware and software, researchers have turned to creatures that manage it all with brains that weigh less than a milligram. 'Insects are a natural source of inspiration for aerospace, primarily since they were the first creatures to fly, 300 million or so years ago,' says Javaan Chahl of the Australian Defence Science and Technology Organisation's weapons systems division. Chahl is working along with professor M.V. Srinivasan, director of the Centre for Visual Sciences at the Australian National University, to design vision and navigation systems based on the honeybee. ... The key lies in understanding how insects perceive their world -- a concept called optic flow. ... Besides military applications, the technology has enormous commercial spinoffs, including intelligent vehicle systems, autonomous robots, intelligent toys, panoramic imaging systems and sensors that aid the blind, just to name a few."

Cars that Think. PBS television broadcast of Scientific American Frontiers show (January 26, 2005). "The fully automatic car may be down the road a ways, but cars that do your thinking for you are just around the corner -- they watch out for hazards, they listen to you, they read your lips, they even know when you're distracted."

Certainly it's not human - The machine vision process consists of four byte-intensive steps. By Nicholas Sheble. ISA - The Instrumentation, Systems, and Automation Society. (February 1, 2005). "Image interpretation consists of matching the processed image against a set of stored images to make identification. Template matching involves superposing the processed image over the stored image and measuring, for example, the percentage of pixels that do not correspond. Feature matching, a more sophisticated approach, involves calculating a weighted function of a number of features of the processed image and comparing it with the same function calculated for the stored image."

Vision as a Computational Process. Chapter 9 of Aaron Sloman's 1978 text, The Computer Revolution in Philosophy, has been recently annotated and is now available online. As stated in the Introduction: "In this chapter I wish to elaborate on a theme which Immanuel Kant found obvious: there is no perception without prior knowledge and abilities."

A Summary of the 2001 North American Machine Vision Market Study. By Nello Zuech. AIA's Machine Vision Online, March 2002. "Examining the sales of 320 companies selling machine vision products into the North American market, the North American machine vision market only declined 6.8% in terms of revenues and 17.5% in terms of units in 2001 in spite of the terrible economy."

Related Web Sites

AI on the Web: Perception and Robotics. A resource companion to Stuart Russell and Peter Norvig's "Artificial Intelligence: A Modern Approach" with links to reference material, people, research groups, books, companies and much more.

Artificial Intelligence Center Perception Program at SRI International - "3D modeling and interpretation, analysis of range images, image matching and autonomous navigation, integration of multiple information sources, linear feature detection and analysis, model-based scene analysis, natural object recognition, optimization-based recognition, partitioning and perceptual organization, representation of natural scenes, stereo analysis."

Artificial Intelligence, Vision & Robotics research at Yale University's Department of Computer Science. "The name 'artificial intelligence' covers a lot of disparate problem areas, united mainly by the fact that they involve complex inputs and outputs that are difficult to compute (or even check for correctness when supplied). One of the most interesting such areas is sensor-controlled behavior, in which a machine acts in the real world using information gathered from sensors such as sonars and cameras. This is a major focus of A.I. research at Yale. The difference between sensor-controlled behavior and what computers usually do is that the input from a sensor is ambiguous."

Computer Vision Course Home Pages. Maintained by Qiang Ji, Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute.

The Computer Vision Industry. By David Lowe, "This web page provides information on some commercially successful applications of computer vision with pointers to company home pages."

Introduction to Active Contours and Visual Dynamics. A very interesting collection of video clips from Andrew Blake, Visual Dynamics Group, Department of Engineering Science, University of Oxford that answers the question: " Computers have been getting better and better at seeing movement on video. How is it that they read lips, follow a dancing girl or copy an actor making faces?" Subjects covered include Marker-free biometrics, Surveillance, Traffic monitoring, and Tracking agile motion ("A key factor in recent successes in making computers 'see' moving objects has been in getting the computer to anticipate.")

Omnidirectional Vision Home Page. "Hosted by the GRASP Laboratory, maintained by Kostas Daniilidis, part of the IEEE Robotics and Automation Society, [Technical Committee] on Computer and Robot Vision activities." A collection of links to reseach projects and companies from around the world.

On-Line Video Demonstrations from CAVE, the Columbia Automated Vision Environment at Columbia University. An interesting collection of video and interactive demos.

Some Vision Groups:

• Computer Vision Group. UC Berkeley Computer Vision
• Computer Vision Group. The University of Wisconsin-Madison Computer Vision Group
• Computer Vision Lab: University of Tokyo ( IKEUCHI Laboratory, Institute of Industrial Science)
• Computer Vision and Pattern Recognition Group. Computer Vision and Pattern Recognition Group, University of Bonn
• Computer Vision, Pattern Recognition and Artificial Intelligence Research Group (CVPRAI) at the Computing Research Centre, Sheffield Hallam University, Sheffield, United Kingdom
• Machine Learning And Perception Group at Microsoft Research, Cambridge, UK
• Machine Vision Group. Imaging technologies Laboratory, Department of Electrical Engineering, University of Ljubljana
• Medical Vision Group. MIT AI Lab

Related Pages in AI Topics

• General Index to AI in the news: Vision
• Image Understanding

More Readings

AI Magazine, Volume Twenty-Five, Number Two (Summer 2004): Cognitive Vision, an Editorial Introduction by Henrik I. Christensen. Abstract: "The integration of AI and vision has been a longterm goal of both disciplines for more than three decades. This special issue illustrates some recent work on bridging the gap." The three related articles in this special issue are: Dynamic Vision-Based Intelligence by Ernst D. Dickmanns; Steps toward a Cognitive Vision System by Hans-Hellmut Nagel; and Unrestricted Recognition of 3D Objects for Robotics Using Multilevel Triplet Invariants by Gösta H. Granlund and Anders Moe.

Beautiful Vision - Artificial vision is bringing robots out of the dark ages. By Jarret Adams. Red Herring Magazine (Subscription req'd.; Issue 81, August 2000)

Eyes for Computers: How HAL Could "See." By Azriel Rosenfeld. Chapter 10 of HAL's Legacy: 2001's Computer as Dream and Reality, edited by David G. Stork (MIT Press, 1996). The abstract is available online: "At the time 2001 was filmed, work was well underway on giving computers the ability to 'see' the world by analyzing images. ... The field of computer vision deals with methods a computer can use to obtain information about objects and events in a scene by analysing images of the scene. These methods need not resemble those used by humans (or animals) to see the world as long as they yield correct results. ...."