Wide-Angle Foveation for All-Purpose Use

This paper proposes a model of a wide-angle space-variant image that provides a guide for designing a fovea sensor. First, an advanced wide-angle foveated (AdWAF) model is formulated, taking all-purpose use into account. This proposed model uses both Cartesian (linear) coordinates and logarithmic coordinates in both planar projection and spherical projection. Thus, this model divides its wide-angle field of view into four areas, such that it can represent an image by various types of lenses, flexibly. The first simulation compares with other lens models, in terms of image height and resolution. The result shows that the AdWAF model can reduce image data by 13.5%, compared to a log-polar lens model, both having the same resolution in the central field of view. The AdWAF image is remapped from an actual input image by the prototype fovea lens, a wide-angle foveated (WAF) lens, using the proposed model. The second simulation compares with other foveation models used for the existing log-polar chip and vision system. The third simulation estimates a scale-invariant property by comparing with the existing fovea lens and the log-polar lens. The AdWAF model gives its planar logarithmic part a complete scale-invariant property, while the fovea lens has 7.6% error at most in its spherical logarithmic part. The fourth simulation computes optical flow in order to examine the unidirectional property when the fovea sensor by the AdWAF model moves, compared to the pinhole camera. The result obtained by using a concept of a virtual cylindrical screen indicates that the proposed model has advantages in terms of computation and application of the optical flow when the fovea sensor moves forward

An Adaptable Foveating Vision Chip

Published versio

Eccentricity estimator for wide-angle fovea sensor by FMI descriptor approach

This paper proposes a method for estimating eccentricity that corresponds to an incident angle to a fovea sensor. The proposed method applies Fourier-Mellin Invariant descriptor for estimating rotation, scale, and translation, by taking both geometrical distortion and non-uniform resolution of a space-variant image by the fovea sensor into account. The following 2 points are focused in this paper. One is to use multi-resolution images computed by discrete wavelet transform for reducing noise caused by foveation properly. Another is to use a variable window function (although the window function is generally used for reducing DFT leakage caused by both ends of a signal.) for changing an effective field of view (FOV) in order not to sacrifice high accuracy. The simulation compares the root mean square (RMS) of the foveation noise between uniform and non-uniform resolutions, when a resolution level and a FOV level are changed, respectively. Experimental results show that the proposed method is consistent with the wide-angle space-variant image by the fovea sensor, i.e., it does not sacrifice high accuracy in the central FOV

Image Extraction by Wide Angle Foveated Lens for Overt-Attention

This paper defines Wide Angle Foveated (WAF) imaging. A proposed model combines Cartesian coordinate system, a log-polar coordinate system, and a unique camera model composed of planar projection and spherical projection for all-purpose use of a single imaging device. The central field-of-view (FOV) and intermediate FOV are given translation-invariance and, rotation and scale-invariance for pattern recognition, respectively. Further, the peripheral FOV is more useful for camera’s view direction control, because its image height is linear to an incident angle to the camera model’s optical center point. Thus, this imaging model improves its usability especially when a camera is dynamically moved, that is, overt-attention. Moreover, simulation results of image extraction show advantages of the proposed model, in view of its magnification factor of the central FOV, accuracy of scale-invariance and flexibility to describe other WAF vision sensors

A biologically inspired computational vision front-end based on a self-organised pseudo-randomly tessellated artificial retina

This paper considers the construction of a biologically inspired front-end for computer vision based on an artificial retina pyramid with a self-organised pseudo-randomly tessellated receptive field tessellation. The organisation of photoreceptors and receptive fields in biological retinae locally resembles a hexagonal mosaic, whereas globally these are organised with a very densely tessellated central foveal region which seamlessly merges into an increasingly sparsely tessellated periphery. In contrast, conventional computer vision approaches use a rectilinear sampling tessellation which samples the whole field of view with uniform density. Scale-space interest points which are suitable for higher level attention and reasoning tasks are efficiently extracted by our vision front-end by performing hierarchical feature extraction on the pseudo-randomly spaced visual information. All operations were conducted on a geometrically irregular foveated representation (data structure for visual information) which is radically different to the uniform rectilinear arrays used in conventional computer vision

Tracking in a space variant active vision system

Without the ability to foveate on and maintain foveation, active vision for applications such as surveillance, object recognition and object tracking are difficult to build. Although foveation in cartesian coordinates is being actively pursued by many, multi-resolution high accuracy foveation in log polar space has not been given much attention. This paper addresses the use of foveation to track a single object as well as multiple objects for a simulated space variant active vision system. Complex logarithmic mapping is chosen firstly because it provides high resolution and wide angle viewing. Secondly, the spatially variant structure of log polar space leads to an object increasing in size as it moves towards the fovea. This is important as we know which object is closer to the fovea at any instant in time.<br /

A Foveated Silicon Retina for Two-Dimensional Tracking

A silicon retina chip with a central foveal region for smooth-pursuit tracking and a peripheral region for saccadic target acquisition is presented. The foveal region contains a 9 x 9 dense array of large dynamic range photoreceptors and edge detectors. Two-dimensional direction of foveal motion is computed outside the imaging array. The peripheral region contains a sparse array of 19 x 17 similar, but larger, photoreceptors with in-pixel edge and temporal ON-set detection. The coordinates of moving or flashing targets are computed with two one-dimensional centroid localization circuits located on the outskirts of the peripheral region. The chip is operational for ambient intensities ranging over six orders of magnitude, targets contrast as low as 10%, foveal speed ranging from 1.5 to 10K pixels/s, and peripheral ON-set frequencies from \u3c0.1 to 800 kHz. The chip is implemented in 2-μm N well CMOS process and consumes 15 mW (V dd = 4 V) in normal indoor light (25 μW/cm2). It has been used as a person tracker in a smart surveillance system and a road follower in an autonomous navigation system

Machine Vision System to Induct Binocular Wide-Angle Foveated Information into Both the Human and Computers - Feature Generation Algorithm based on DFT for Binocular Fixation

This paper introduces a machine vision system, which is suitable for cooperative works between the human and computer. This system provides images inputted from a stereo camera head not only to the processor but also to the user’s sight as binocular wide-angle foveated (WAF) information, thus it is applicable for Virtual Reality (VR) systems such as tele-existence or training experts. The stereo camera head plays a role to get required input images foveated by special wide-angle optics under camera view direction control and 3D head mount display (HMD) displays fused 3D images to the user. Moreover, an analog video signal processing device much inspired from a structure of the human visual system realizes a unique way to provide WAF information to plural processors and the user. Therefore, this developed vision system is also much expected to be applicable for the human brain and vision research, because the design concept is to mimic the human visual system. Further, an algorithm to generate features using Discrete Fourier Transform (DFT) for binocular fixation in order to provide well-fused 3D images to 3D HMD is proposed. This paper examines influences of applying this algorithm to space variant images such as WAF images, based on experimental results

Foveated Sampling Architectures for CMOS Image Sensors

Electronic imaging technologies are faced with the challenge of power consumption when transmitting large amounts of image data from the acquisition imager to the display or processing devices. This is especially a concern for portable applications, and becomes more prominent in increasingly high-resolution, high-frame rate imagers. Therefore, new sampling techniques are needed to minimize transmitted data, while maximizing the conveyed image information. From this point of view, two approaches have been proposed and implemented in this thesis: A system-level approach, in which the classical 1D row sampling CMOS imager is modified to a 2D ring sampling pyramidal architecture, using the same standard three transistor (3T) active pixel sensor (APS). A device-level approach, in which the classical orthogonal architecture has been preserved while altering the APS device structure, to design an expandable multiresolution image sensor. A new scanning scheme has been suggested for the pyramidal image sensor, resulting in an intrascene foveated dynamic range (FDR) similar in profile to that of the human eye. In this scheme, the inner rings of the imager have a higher dynamic range than the outer rings. The pyramidal imager transmits the sampled image through 8 parallel output channels, allowing higher frame rates. The human eye is known to have less sensitivity to oblique contrast. Using this fact on the typical oblique distribution of fixed pattern noise, we demonstrate lower perception of this noise than the orthogonal FPN distribution of classical CMOS imagers. The multiresolution image sensor principle is based on averaging regions of low interest from frame-sampled image kernels. One pixel is read from each kernel while keeping pixels in the region of interest at their high resolution. This significantly reduces the transferred data and increases the frame rate. Such architecture allows for programmability and expandability of multiresolution imaging applications