An evaluation of the pedestrian classification in a multi-domain multi-modality setup

The objective of this article is to study the problem of pedestrian classification across different light spectrum domains (visible and far-infrared (FIR)) and modalities (intensity, depth and motion). In recent years, there has been a number of approaches for classifying and detecting pedestrians in both FIR and visible images, but the methods are difficult to compare, because either the datasets are not publicly available or they do not offer a comparison between the two domains. Our two primary contributions are the following: (1) we propose a public dataset, named RIFIR , containing both FIR and visible images collected in an urban environment from a moving vehicle during daytime; and (2) we compare the state-of-the-art features in a multi-modality setup: intensity, depth and flow, in far-infrared over visible domains. The experiments show that features families, intensity self-similarity (ISS), local binary patterns (LBP), local gradient patterns (LGP) and histogram of oriented gradients (HOG), computed from FIR and visible domains are highly complementary, but their relative performance varies across different modalities. In our experiments, the FIR domain has proven superior to the visible one for the task of pedestrian classification, but the overall best results are obtained by a multi-domain multi-modality multi-feature fusion

Registration of Multimodal Imagery with Occluding Objects using Mutual Information

Abstract. In this paper we introduce and analyze a method for registering multimodal images with occluding objects in the scene. An analysis of multimodal image registration gives insight into the limitations of assumptions made in current approaches and motivates the methodology of the developed algorithm. Using calibrated stereo imagery, we use maximization of mutual information in sliding correspondence windows that inform a disparity voting algorithm to demonstrate successful registration of objects in color and thermal imagery where there is significant occlusion. Extensive testing of scenes with multiple objects at different depths and levels of occlusion shows high rates of successful registration. Ground truth experiments demonstrate the utility of disparity voting techniques for multimodal registration by yielding qualitative and quantitative results that outperform approaches that do not consider occlusions

Occupant Posture Analysis Using Reflectance and Stereo Images for "Smart" Airbag Deployment

Robust detection of vehicle occupant posture is necessary for intelligent airbag deployment. This paper presents a vision-based method of estimating the size, posture and pose of the occupant. Utilizing raw reflectance and stereo disparity images, this algorithm presents a mixed-mode approach to finding occupant features. Extensive experiments show the robustness of this mixed-mode method in identifying the occupant's head location and suggest feasibility in extending the analysis system to include robust detection of features such as occupant arms and foreign objects

Real-Time Stereo-Based Head Detection using Size, Shape and Disparity Constraints

Abstract — A real-time stereo-based head detection algorithm is presented and evaluated as an occupant posture analysis system for ”smart ” airbag deployment. The algorithm uses several constraints to limit the number of head ellipse ”candidates” found in the image. These constraints are based on shape, size and depth of the occupant’s head. Results of ground truth experiments show that the head detection can accurately estimate the three-dimensional location of the occupant head. Extended experiments illustrate the robustness of the algorithm to poor lighting conditions, to occlusion, and to the presence of other competing head-like objects in the scene. I