The Local Structure of Space-Variant Images

Local image structure is widely used in theories of both machine and biological vision. The form of the differential operators describing this structure for space-invariant images has been well documented (e.g. Koenderink, 1984). Although space-variant coordinates are universally used in mammalian visual systems, the form of the operators in the space-variant domain has received little attention. In this report we derive the form of the most common differential operators and surface characteristics in the space-variant domain and show examples of their use. The operators include the Laplacian, the gradient and the divergence, as well as the fundamental forms of the image treated as a surface. We illustrate the use of these results by deriving the space-variant form of corner detection and image enhancement algorithms. The latter is shown to have interesting properties in the complex log domain, implicitly encoding a variable grid-size integration of the underlying PDE, allowing rapid enhancement of large scale peripheral features while preserving high spatial frequencies in the fovea.Office of Naval Research (N00014-95-I-0409

View point robust visual search technique

In this thesis, we have explored visual search techniques for images taken from diferent view points and have tried to enhance the matching capability under view point changes. We have proposed the Homography based back-projection as post-processing stage of Compact Descriptors for Visual Search (CDVS), the new MPEG standard; moreover, we have deined the aine adapted scale space based aine detection, which steers the Gaussian scale space to capture the features from aine transformed images; we have also developed the corresponding gradient based aine descriptor. Using these proposed techniques, the image retrieval robustness to aine transformations has been signiicantly improved. The irst chapter of this thesis introduces the background on visual search. In the second chapter, we propose a homography based back-projection used as the postprocessing stage of CDVS to improve the resilience to view point changes. The theory behind this proposal is that each perspective projection of the image of 2D object can be simulated as an aine transformation. Each pair of aine transformations are mathematically related by homography matrix. Given that matrix, the image can be back-projected to simulate the image of another view point. In this way, the real matched images can then be declared as matching because the perspective distortion has been reduced by the back-projection. An accurate homography estimation from the images of diferent view point requires at least 4 correspondences, which could be ofered by the CDVS pipeline. In this way, the homography based back-projection can be used to scrutinize the images with not enough matched keypoints. If they contain some homography relations, the perspective distortion can then be reduced exploiting the few provided correspondences. In the experiment, this technique has been proved to be quite efective especially to the 2D object images. The third chapter introduces the scale space, which is also the kernel to the feature detection for the scale invariant visual search techniques. Scale space, which is made by a series of Gaussian blurred images, represents the image structures at diferent level of details. The Gaussian smoothed images in the scale space result in feature detection being not invariant to aine transformations. That is the reason why scale invariant visual search techniques are sensitive to aine transformations. Thus, in this chapter, we propose an aine adapted scale space, which employs the aine steered Gaussian ilters to smooth the images. This scale space is lexible to diferent aine transformations and it well represents the image structures from diferent view points. With the help of this structure, the features from diferent view points can be well captured. In practice, the scale invariant visual search techniques have employed a pyramid structure to speed up the construction. By employing the aine Gaussian scale space principles, we also propose two structures to build the aine Gaussian scale space. The structure of aine Gaussian scale space is similar to the pyramid structure because of the similiar sampling and cascading iii properties. Conversely, the aine Laplacian of Gaussian (LoG) structure is completely diferent. The Laplacian operator, under aine transformation, is hard to be aine deformed. Diferently from a simple Laplacian operation on the scale space to build the general LoG construction, the aine LoG can only be obtained by aine LoG convolution and the cascade implementations on the aine scale space. Using our proposed structures, both the aine Gaussian scale space and aine LoG can be constructed. We have also explored the aine scale space implementation in frequency domain. In the second chapter, we will also explore the spectrum of Gaussian image smoothing under the aine transformation, and propose two structures. General speaking, the implementation in frequency domain is more robust to aine transformations at the expense of a higher computational complexity. It makes sense to adopt an aine descriptor for the aine invariant visual search. In the fourth chapter, we will propose an aine invariant feature descriptor based on aine gradient. Currently, the state of the art feature descriptors, including SIFT and Gradient location and orientation histogram (GLOH), are based on the histogram of image gradient around the detected features. If the image gradient is calculated as the diference of the adjacent pixels, it will not be aine invariant. Thus in that chapter, we irst propose an aine gradient which will contribute the aine invariance to the descriptor. This aine gradient will be calculated directly by the derivative of the aine Gaussian blurred images. To simplify the processing, we will also create the corresponding aine Gaussian derivative ilters for diferent detected scales to quickly generate the aine gradient. With this aine gradient, we can apply the same scheme of SIFT descriptor to generate the gradient histogram. By normalizing the histogram, the aine descriptor can then be formed. This aine descriptor is not only aine invariant but also rotation invariant, because the direction of the area to form the histogram is determined by the main direction of the gradient around the features. In practice, this aine descriptor is fully aine invariant and its performance for image matching is extremely good. In the conclusions chapter, we draw some conclusions and describe some future work

An Autoencoder-Based Image Descriptor for Image Matching and Retrieval

Local image features are used in many computer vision applications. Many point detectors and descriptors have been proposed in recent years; however, creation of effective descriptors is still a topic of research. The Scale Invariant Feature Transform (SIFT) developed by David Lowe is widely used in image matching and image retrieval. SIFT detects interest points in an image based on Scale-Space analysis, which is invariant to change in image scale. A SIFT descriptor contains gradient information about an image patch centered at a point of interest. SIFT is found to provide a high matching rate, is robust to image transformations; however, it is found to be slow in image matching/retrieval. Autoencoder is a method for representation learning and is used in this project to construct a low-dimensional representation of a high-dimensional data while preserving the structure and geometry of the data. In many computer vision tasks, the high dimensionality of input data means a high computational cost. The main motivation in this project is to improve the speed and the distinctness of SIFT descriptors. To achieve this, a new descriptor is proposed that is based on Autoencoder. Our newly generated descriptors can reduce the size and complexity of SIFT descriptors, reducing the time required in image matching and image retrieval

Image hierarchy in gaussian scale space

We investigate the topological structure of an image and the hierarchical relationship between local and global structures provided by spatial gradients at different levels of scale in the Gaussian scale space. The gradient field curves link stationary points of an image, including a local minimum at infinity, and construct the topological structure of the image. The evolution of the topological structure with respect to scale is analyzed using pseudograph representation. The hierarchical relationships among the structures at different scales are expressed as trajectories of the stationary points in the scale space, which we call the stationary curves. Each top point of the local extremum curve generically has a specific gradient field curve, which we call the antidirectional figure-flow curve. The antidirectional figure-flow curve connects the top-point and another local extremum to which the toppoint is subordinate. A point at infinity can also be connected to the top points of local minimum curves. These hierarchical relationships among the stationary points are expressed as a tree. This tree expresses a hierarchical structure of dominant parts. We clarify the graphical grammar for the construction of this tree in the Gaussian scale space. Furthermore, we show a combinatorial structure of singular points in the Gaussian scale space using conformal mapping from Euclidean space to the spherical surface. We define segment edges as a zero-crossing set in the Gaussian scale space using the singular points. An image in the Gaussian scale space is the convolution of the image and the Gaussian kernel. The Gaussian kernel of an appropriate variance is a typical presmoothing operator for segmentation. The variance is heuristically selected using statistics of images such as the noise distribution in images. The variance of the kernel is determined using the singular-point configuration in the Gaussian scale space, since singular points in the Gaussian scale space allow the extraction of the dominant parts of an image. This scale-selection strategy derives the hierarchical structure of the segments. Unsupervised segmentation methods, however, have difficulty in distinguishing valid segments associated with the objects from invalid random segments due to noise. By showing that the number of invalid segments monotonically decreases with increasing scale, we characterize the valid and invalid segments in the Gaussian scale space. This property allows us to identify the valid segments from coarse to fine and does us to prevent undersegmentation and oversegmentation. Finally, we develop principal component analysis (PCA) of a point cloud on the basis of the scale-space representation of its probability density function. We explain the geometric features of a point cloud in the Gaussian scale space and observe reduced dimensionality with respect to the loss of information. Furthermore, we introduce a hierarchical clustering of the point cloud and analyze the statistical significance of the clusters and their subspaces. Moreover, we present a mathematical framework of the scale-based PCA, which derives a statistically reasonable criterion for choosing the number of components to retain or reduce the dimensionality of a point cloud. Finally, we also develop a segmentation algorithm using configurations of singular points in the Gaussian scale space

Geometrical Optics Formalism to Model Contrast in Dark-Field X-ray Microscopy

Dark-field X-ray microscopy is a new full-field imaging technique that nondestructively maps the structure and local strain inside deeply embedded crystalline elements in three dimensions. Placing an objective lens in the diffracted beam generates a magnified projection image of a local volume. We provide a general formalism based on geometrical optics for the diffraction imaging, valid for any crystallographic space group. This allows simulation of diffraction images based on micro-mechanical models. We present example simulations with the formalism, demonstrating how it may be used to design new experiments or interpret existing ones. In particular, we show how modifications to the experimental design may tailor the reciprocal-space resolution function to map specific components of the deformation gradient tensor. The formalism supports multi-length scale experiments, as it enables DFXM to be interfaced with 3DXRD. The formalism is demonstrated by comparison to experimental images of the strain field around a straight dislocation

Evidence for a Supermassive Black Hole in the S0 Galaxy NGC 3245

The S0 galaxy NGC 3245 contains a circumnuclear disk of ionized gas and dust with a radius of 1.1" (110 pc), making it an ideal target for dynamical studies with the Hubble Space Telescope (HST). We have obtained spectra of the nuclear disk with the Space Telescope Imaging Spectrograph, using a 0.2" wide slit at five parallel positions. Measurements of the Hα and [N II] emission lines are used to map out the kinematic structure of the disk in unprecedented detail. The data reveal a rotational velocity field with a steep velocity gradient across the innermost 0.4". We construct dynamical models for a thin gas disk in circular rotation, using HST optical images to map out the gravitational potential due to stars. Our modeling code includes the blurring due to the telescope point-spread function and the nonzero slit width, as well as the instrumental shift in measured wavelength for light entering the slit off-center, so as to simulate the data as closely as possible. The Hα+[N II] surface brightness measured from an HST narrowband image is folded into the models, and we demonstrate that many of the apparent small-scale irregularities in the observed velocity curves are the result of the patchy distribution of emission-line surface brightness. Over most of the disk, the models are able to fit the observed radial velocity curves closely, although there are localized regions within the disk that appear to be kinematically disturbed relative to the overall rotational pattern. The velocity dispersion of [N II] λ6584 rises from σ~50 km/s in the outer disk to ~160 km/s at the nucleus, and most of this line width cannot be attributed to rotational or instrumental broadening. To account for the possible dynamical effect of the intrinsic velocity dispersion in the gas, we also calculate models that include a correction for asymmetric drift. This correction increases the derived black hole mass by 12% but leads to slightly poorer fits to the data. A central dark mass of (2.1+/-0.5)×10^8 Msolar is required for the models to reproduce the steep central velocity gradient. This value for the central mass is consistent with recently discovered correlations between black hole mass and bulge velocity dispersion.Peer reviewe

Real-Time Anisotropic Diffusion using Space-Variant Vision

Many computer and robot vision applications require multi-scale image analysis. Classically, this has been accomplished through the use of a linear scale-space, which is constructed by convolution of visual input with Gaussian kernels of varying size (scale). This has been shown to be equivalent to the solution of a linear diffusion equation on an infinite domain, as the Gaussian is the Green's function of such a system (Koenderink, 1984). Recently, much work has been focused on the use of a variable conductance function resulting in anisotropic diffusion described by a nonlinear partial differential equation (PDF). The use of anisotropic diffusion with a conductance coefficient which is a decreasing function of the gradient magnitude has been shown to enhance edges, while decreasing some types of noise (Perona and Malik, 1987). Unfortunately, the solution of the anisotropic diffusion equation requires the numerical integration of a nonlinear PDF which is a costly process when carried out on a fixed mesh such as a typical image. In this paper we show that the complex log transformation, variants of which are universally used in mammalian retino-cortical systems, allows the nonlinear diffusion equation to be integrated at exponentially enhanced rates due to the non-uniform mesh spacing inherent in the log domain. The enhanced integration rates, coupled with the intrinsic compression of the complex log transformation, yields a seed increase of between two and three orders of magnitude, providing a means of performing real-time image enhancement using anisotropic diffusion.Office of Naval Research (N00014-95-I-0409