Low Complexity Image Recognition Algorithms for Handheld devices

Content Based Image Retrieval (CBIR) has gained a lot of interest over the last two decades. The need to search and retrieve images from databases, based on information (“features”) extracted from the image itself, is becoming increasingly important. CBIR can be useful for handheld image recognition devices in which the image to be recognized is acquired with a camera, and thus there is no additional metadata associated to it. However, most CBIR systems require large computations, preventing their use in handheld devices. In this PhD work, we have developed low-complexity algorithms for content based image retrieval in handheld devices for camera acquired images. Two novel algorithms, ‘Color Density Circular Crop’ (CDCC) and ‘DCT-Phase Match’ (DCTPM), to perform image retrieval along with a two-stage image retrieval algorithm that combines CDCC and DCTPM, to achieve the low complexity required in handheld devices are presented. The image recognition algorithms run on a handheld device over a large database with fast retrieval time besides having high accuracy, precision and robustness to environment variations. Three algorithms for Rotation, Scale, and Translation (RST) compensation for images were also developed in this PhD work to be used in conjunction with the two-stage image retrieval algorithm. The developed algorithms are implemented, using a commercial fixed-point Digital Signal Processor (DSP), into a device, called ‘PictoBar’, in the domain of Alternative and Augmentative Communication (AAC). The PictoBar is intended to be used in the field of electronic aid for disabled people, in areas like speech rehabilitation therapy, education etc. The PictoBar is able to recognize pictograms and pictures contained in a database. Once an image is found in the database, a corresponding associated speech message is played. A methodology for optimal implementation and systematic testing of the developed image retrieval algorithms on a fixed point DSP is also established as part of this PhD work

Shape-based invariant features extraction for object recognition

International audienceThe emergence of new technologies enables generating large quantity of digital information including images; this leads to an increasing number of generated digital images. Therefore it appears a necessity for automatic systems for image retrieval. These systems consist of techniques used for query specification and re-trieval of images from an image collection. The most frequent and the most com-mon means for image retrieval is the indexing using textual keywords. But for some special application domains and face to the huge quantity of images, key-words are no more sufficient or unpractical. Moreover, images are rich in content; so in order to overcome these mentioned difficulties, some approaches are pro-posed based on visual features derived directly from the content of the image: these are the content-based image retrieval (CBIR) approaches. They allow users to search the desired image by specifying image queries: a query can be an exam-ple, a sketch or visual features (e.g., colour, texture and shape). Once the features have been defined and extracted, the retrieval becomes a task of measuring simi-larity between image features. An important property of these features is to be in-variant under various deformations that the observed image could undergo. In this chapter, we will present a number of existing methods for CBIR applica-tions. We will also describe some measures that are usually used for similarity measurement. At the end, and as an application example, we present a specific ap-proach, that we are developing, to illustrate the topic by providing experimental results

EXTENDING CONVOLUTION THROUGH SPATIALLY ADAPTIVE ALIGNMENT

Convolution underlies a variety of applications in computer vision and graphics, including efficient filtering, analysis, simulation, and neural networks. However, convolution has an inherent limitation: when convolving a signal with a filter, the filter orientation remains fixed as it travels over the domain, and convolution loses effectiveness in the presence of deformations that change alignment of the signal relative to the local frame. This problem metastasizes when attempting to generalize convolution to domains without a canonical orientation, such as the surfaces of 3D shapes, making it impossible to locally align signals and filters in a consistent manner. This thesis presents a unified framework for transformation-equivariant convolutions on arbitrary homogeneous spaces and 2D Riemannian manifolds called extended convolution. This approach is based on the the following observation: to achieve equivariance to an arbitrary class of transformations, we only need to consider how the positions of points as seen in the frames of their neighbors deform. By defining an equivariant frame operator at each point with which we align the filter, we correct for the change in the relative positions induced by the transformations. This construction places no constraints on the filters, making extended convolution highly descriptive. Furthermore, the framework can handle arbitrary transformation groups, including higher-dimensional non-compact groups that act non-linearly on the domain. Critically, extended convolution naturally generalizes to arbitrary 2D Riemannian manifolds as it does not need a canonical coordinate system to be applied. The power and utility of extended convolution is demonstrated in several applications. A unified framework for image and surface feature descriptors called Extended Convolution Histogram of Orientations (ECHO) is proposed, based on the optimal filters maximizing the response of the extended convolution at a given point. Using the generalization of extended convolution to surface vector fields, state-of-the-art surface convolutional neural networks (CNNs) are constructed. Last, we move beyond rotations and isometries and use extended convolution to design spherical CNNs equivariant to Mobius transformations, representing a first step toward conformally-equivariant surface networks

Geometric and photometric affine invariant image registration

This thesis aims to present a solution to the correspondence problem for the registration of wide-baseline images taken from uncalibrated cameras. We propose an affine invariant descriptor that combines the geometry and photometry of the scene to find correspondences between both views. The geometric affine invariant component of the descriptor is based on the affine arc-length metric, whereas the photometry is analysed by invariant colour moments. A graph structure represents the spatial distribution of the primitive features; i.e. nodes correspond to detected high-curvature points, whereas arcs represent connectivities by extracted contours. After matching, we refine the search for correspondences by using a maximum likelihood robust algorithm. We have evaluated the system over synthetic and real data. The method is endemic to propagation of errors introduced by approximations in the system.BAE SystemsSelex Sensors and Airborne System

On-board three-dimensional object tracking: Software and hardware solutions

We describe a real time system for recognition and tracking 3D objects such as UAVs, airplanes, fighters with the optical sensor. Given a 2D image, the system has to perform background subtraction, recognize relative rotation, scale and translation of the object to sustain a prescribed topology of the fleet. In the thesis a comparative study of different algorithms and performance evaluation is carried out based on time and accuracy constraints. For background subtraction task we evaluate frame differencing, approximate median filter, mixture of Gaussians and propose classification based on neural network methods. For object detection we analyze the performance of invariant moments, scale invariant feature transform and affine scale invariant feature transform methods. Various tracking algorithms such as mean shift with variable and a fixed sized windows, scale invariant feature transform, Harris and fast full search based on fast fourier transform algorithms are evaluated. We develop an algorithm for the relative rotations and the scale change calculation based on Zernike moments. Based on the design criteria the selection is made for on-board implementation. The candidate techniques have been implemented on the Texas Instrument TMS320DM642 EVM board. It is shown in the thesis that 14 frames per second can be processed; that supports the real time implementation of the tracking system under reasonable accuracy limits