Three-dimensional visualization environment for multisensor data analysis, interpretation, and model-based object recognition

Model-based object recognition must solve three-dimensional geometric problems involving the registration of multiple sensors and the spatial relationship of a three-dimensional model to the sensors. Observation and verification of the registration and recognition processes requires display of these geometric relationships. We have developed a prototype software system which allows a user to interact with the sensor data and model matching system in a three-dimensional environment. This visualization environment combines range imagery, color imagery, thermal (infrared) imagery, and CAD models of objects to be recognized. We are currently using imagery of vehicles travelling off-road (a challenging environment for the object recognizer). Range imagery is used to create a partial three-dimensional representation of a scene. Optical imagery is mapped onto this partial 3D representation. Visualization allows monitoring of the recognizer as it solves for the type and position of the object...