Robotics Research at the GRASP Laboratory

The General Robotics and Active Sensory Perception (GRASP) Laboratory of the University of Pennsylvania does research in various areas of robotics including coordinated control of multiple robot manipulators, strategics for robotic sensing, multi-sensor integration, distributed real-time operating systems, telerobotics with communication delays, image understanding, and range image analysis

Methods for multi-spectral image fusion: identifying stable and repeatable information across the visible and infrared spectra

Fusion of images captured from different viewpoints is a well-known challenge in computer vision with many established approaches and applications; however, if the observations are captured by sensors also separated by wavelength, this challenge is compounded significantly. This dissertation presents an investigation into the fusion of visible and thermal image information from two front-facing sensors mounted side-by-side. The primary focus of this work is the development of methods that enable us to map and overlay multi-spectral information; the goal is to establish a combined image in which each pixel contains both colour and thermal information. Pixel-level fusion of these distinct modalities is approached using computational stereo methods; the focus is on the viewpoint alignment and correspondence search/matching stages of processing. Frequency domain analysis is performed using a method called phase congruency. An extensive investigation of this method is carried out with two major objectives: to identify predictable relationships between the elements extracted from each modality, and to establish a stable representation of the common information captured by both sensors. Phase congruency is shown to be a stable edge detector and repeatable spatial similarity measure for multi-spectral information; this result forms the basis for the methods developed in the subsequent chapters of this work. The feasibility of automatic alignment with sparse feature-correspondence methods is investigated. It is found that conventional methods fail to match inter-spectrum correspondences, motivating the development of an edge orientation histogram (EOH) descriptor which incorporates elements of the phase congruency process. A cost function, which incorporates the outputs of the phase congruency process and the mutual information similarity measure, is developed for computational stereo correspondence matching. An evaluation of the proposed cost function shows it to be an effective similarity measure for multi-spectral information

Extending an industrial root controller : implementation and applications of a fast open sensor interface

An overview is given of the design and implementation of a platform for fast external sensor integration in an industrial robot system called ABB S4CPlus. As an application and motivating example, the implementation of force-controlled grinding and deburring within the AUTOFETT-project is discussed. Experiences from industrial usage of the fully developed prototype confirms the appropriateness of the design choices, thus also confirming the fact that control and software need to be tightly integrated. The new sensor can be used for the prototyping and development of a wide variety of new application

Sensor Fusion for Video Surveillance

In this paper, a multisensor data fusion system for object tracking is presented. It is able to track in real-time multiple targets in outdoor environments. The system can take advantage of the redundant information coming from different sensors monitoring the same scene. The measurements (positions of the targets) obtained from the available sources are fused together to obtain a more accurate estimate. Data fusion is performed considering sensor reliability at every time instant. A confidence measure has been employed to weight sensor data in the fusion process. Compared to single camera systems, the adopted approach has produced more accurate and continuous trajectories, reducing calibration and segmentation errors

Timed Atomic Commitment

In a large class of hard-real-time control applications, components execute concurrently on distributed nodes and must coordinate, under timing constraints, to perform the control task. As such, they perform a type of atomic commitment. Traditional atomic commitment differs, however, because there are no timing constraints; agreement is eventual. We therefore define timed atomic commitment (TAC) which requires the processes to be functionally consistent, but allows the outcome to include an exceptional state, indicating that timing constraints have been violated. We then present centralized and decentralized protocols to implement TAC and a high-level language construct that facilitates its use in distributed real-time programming

New techniques for the automatic registration of microwave and optical remotely sensed images

Remote sensing is a remarkable tool for monitoring and mapping the land and ocean surfaces of the Earth. Recently, with the launch of many new Earth observation satellites, there has been an increase in the amount of data that is being acquired, and the potential for mapping is greater than ever before. Furthermore, sensors which are currently operational are acquiring data in many different parts of the electromagnetic spectrum. It has long been known that by combining images that have been acquired at different wavelengths, or at different times, the ability to detect and recognise features on the ground is greatly increased. This thesis investigates the possibilities for automatically combining radar and optical remotely sensed images. The process of combining images, known as data integration, is a two step procedure: geometric integration (image registration) and radiometric integration (data fusion). Data fusion is essentially an automatic procedure, but the problems associated with automatic registration of multisource images have not, in general, been resolved. This thesis proposes a method of automatic image registration based on the extraction and matching of common features which are visible in both images. The first stage of the registration procedure uses patches as the matching primitives in order to determine the approximate alignment of the images. The second stage refines the registration results by matching edge features. Throughout the development of the proposed registration algorithm, reliability, robustness and automation were always considered priorities. Tests with both small images (512x512 pixels) and full scene images showed that the algorithm could successfully register images to an acceptable level of accuracy

Robust Multi-Object Tracking: A Labeled Random Finite Set Approach

The labeled random finite set based generalized multi-Bernoulli filter is a tractable analytic solution for the multi-object tracking problem. The robustness of this filter is dependent on certain knowledge regarding the multi-object system being available to the filter. This dissertation presents techniques for robust tracking, constructed upon the labeled random finite set framework, where complete information regarding the system is unavailable