Object detection and activity recognition in digital image and video libraries

This thesis is a comprehensive study of object-based image and video retrieval, specifically for car and human detection and activity recognition purposes. The thesis focuses on the problem of connecting low level features to high level semantics by developing relational object and activity presentations. With the rapid growth of multimedia information in forms of digital image and video libraries, there is an increasing need for intelligent database management tools. The traditional text based query systems based on manual annotation process are impractical for today\u27s large libraries requiring an efficient information retrieval system. For this purpose, a hierarchical information retrieval system is proposed where shape, color and motion characteristics of objects of interest are captured in compressed and uncompressed domains. The proposed retrieval method provides object detection and activity recognition at different resolution levels from low complexity to low false rates. The thesis first examines extraction of low level features from images and videos using intensity, color and motion of pixels and blocks. Local consistency based on these features and geometrical characteristics of the regions is used to group object parts. The problem of managing the segmentation process is solved by a new approach that uses object based knowledge in order to group the regions according to a global consistency. A new model-based segmentation algorithm is introduced that uses a feedback from relational representation of the object. The selected unary and binary attributes are further extended for application specific algorithms. Object detection is achieved by matching the relational graphs of objects with the reference model. The major advantages of the algorithm can be summarized as improving the object extraction by reducing the dependence on the low level segmentation process and combining the boundary and region properties. The thesis then addresses the problem of object detection and activity recognition in compressed domain in order to reduce computational complexity. New algorithms for object detection and activity recognition in JPEG images and MPEG videos are developed. It is shown that significant information can be obtained from the compressed domain in order to connect to high level semantics. Since our aim is to retrieve information from images and videos compressed using standard algorithms such as JPEG and MPEG, our approach differentiates from previous compressed domain object detection techniques where the compression algorithms are governed by characteristics of object of interest to be retrieved. An algorithm is developed using the principal component analysis of MPEG motion vectors to detect the human activities; namely, walking, running, and kicking. Object detection in JPEG compressed still images and MPEG I frames is achieved by using DC-DCT coefficients of the luminance and chrominance values in the graph based object detection algorithm. The thesis finally addresses the problem of object detection in lower resolution and monochrome images. Specifically, it is demonstrated that the structural information of human silhouettes can be captured from AC-DCT coefficients

An Appearance-Based Framework for 3D Hand Shape Classification and Camera Viewpoint Estimation

An appearance-based framework for 3D hand shape classification and simultaneous camera viewpoint estimation is presented. Given an input image of a segmented hand, the most similar matches from a large database of synthetic hand images are retrieved. The ground truth labels of those matches, containing hand shape and camera viewpoint information, are returned by the system as estimates for the input image. Database retrieval is done hierarchically, by first quickly rejecting the vast majority of all database views, and then ranking the remaining candidates in order of similarity to the input. Four different similarity measures are employed, based on edge location, edge orientation, finger location and geometric moments.National Science Foundation (IIS-9912573, EIA-9809340

An efficient image retrieval scheme for colour enhancement of embedded and distributed surveillance images

From the past few years, the size of the data grows exponentially with respect to volume, velocity, and dimensionality due to wide spread use of embedded and distributed surveillance cameras for security reasons. In this paper, we have proposed an integrated approach for biometric-based image retrieval and processing which addresses the two issues. The first issue is related to the poor visibility of the images produced by the embedded and distributed surveillance cameras, and the second issue is concerned with the effective image retrieval based on the user query. This paper addresses the first issue by proposing an integrated image enhancement approach based on contrast enhancement and colour balancing methods. The contrast enhancement method is used to improve the contrast, while the colour balancing method helps to achieve a balanced colour. Importantly, in the colour balancing method, a new process for colour cast adjustment is introduced which relies on statistical calculation. It adjusts the colour cast and maintains the luminance of the image. The integrated image enhancement approach is applied to the enhancement of low quality images produced by surveillance cameras. The paper addresses the second issue relating to image retrieval by proposing a content-based image retrieval approach. The approach is based on the three features extraction methods namely colour, texture and shape. Colour histogram is used to extract the colour features of an image. Gabor filter is used to extract the texture features and the moment invariant is used to extract the shape features of an image. The use of these three algorithms ensures that the proposed image retrieval approach produces results which are highly relevant to the content of an image query, by taking into account the three distinct features of the image and the similarity metrics based on Euclidean measure. In order to retrieve the most relevant images, the proposed approach also employs a set of fuzzy heuristics to improve the quality of the results further. The result

Vision technology/algorithms for space robotics applications

The thrust of automation and robotics for space applications has been proposed for increased productivity, improved reliability, increased flexibility, higher safety, and for the performance of automating time-consuming tasks, increasing productivity/performance of crew-accomplished tasks, and performing tasks beyond the capability of the crew. This paper provides a review of efforts currently in progress in the area of robotic vision. Both systems and algorithms are discussed. The evolution of future vision/sensing is projected to include the fusion of multisensors ranging from microwave to optical with multimode capability to include position, attitude, recognition, and motion parameters. The key feature of the overall system design will be small size and weight, fast signal processing, robust algorithms, and accurate parameter determination. These aspects of vision/sensing are also discussed

Digital Image Access & Retrieval

The 33th Annual Clinic on Library Applications of Data Processing, held at the University of Illinois at Urbana-Champaign in March of 1996, addressed the theme of "Digital Image Access & Retrieval." The papers from this conference cover a wide range of topics concerning digital imaging technology for visual resource collections. Papers covered three general areas: (1) systems, planning, and implementation; (2) automatic and semi-automatic indexing; and (3) preservation with the bulk of the conference focusing on indexing and retrieval.published or submitted for publicatio

Approaches for Automated Object Recognition and Extraction from Images — a Study

Digital Image Interpretation is one of the most challenging and important tasks in many scientific and engineering applications. The two vital subtasks in image interpretation are recognition and extraction of object(s) of interest (OOI) from an image. When such tasks are manually performed, it calls for human experts, making them more time consuming, less cost effective and highly constrained. These negative factors led to the development of a computer system which performed an automatic analysis of visual information in order to bring in consistency, efficiency and accuracy in image analysis. This paper focuses on the survey of various existing automated approaches for recognition and extraction of OOI from an image in various scientific and engineering applications. In this work a categorization of these approaches is made based on the four principle factors (Input, Object, Feature, Attention) with which each approach is driven. Most of the approaches discussed in this paper are proved to work efficiently in real environments