Analog hardware for detecting discontinuities in early vision

The detection of discontinuities in motion, intensity, color, and depth is a well-studied but difficult problem in computer vision [6]. We discuss the first hardware circuit that explicitly implements either analog or binary line processes in a deterministic fashion. Specifically, we show that the processes of smoothing (using a first-order or membrane type of stabilizer) and of segmentation can be implemented by a single, two-terminal nonlinear voltage-controlled resistor, the “resistive fuse”; and we derive its current-voltage relationship from a number of deterministic approximations to the underlying stochastic Markov random fields algorthms. The concept that the quadratic variation functionals of early vision can be solved via linear resistive networks minimizing power dissipation [37] can be extended to non-convex variational functionals with analog or binary line processes being solved by nonlinear resistive networks minimizing the electrical co-content. We have successfully designed, tested, and demonstrated an analog CMOS VLSI circuit that contains a 1D resistive network of fuses implementing piecewise smooth surface interpolation. We furthermore demonstrate the segmenting abilities of these analog and deterministic “line processes” by numerically simulating the nonlinear resistive network computing optical flow in the presence of motion discontinuities. Finally, we discuss various circuit implementations of the optical flow computation using these circuits

Computing motion in the primate's visual system

Computing motion on the basis of the time-varying image intensity is a difficult problem for both artificial and biological vision systems. We will show how one well-known gradient-based computer algorithm for estimating visual motion can be implemented within the primate's visual system. This relaxation algorithm computes the optical flow field by minimizing a variational functional of a form commonly encountered in early vision, and is performed in two steps. In the first stage, local motion is computed, while in the second stage spatial integration occurs. Neurons in the second stage represent the optical flow field via a population-coding scheme, such that the vector sum of all neurons at each location codes for the direction and magnitude of the velocity at that location. The resulting network maps onto the magnocellular pathway of the primate visual system, in particular onto cells in the primary visual cortex (V1) as well as onto cells in the middle temporal area (MT). Our algorithm mimics a number of psychophysical phenomena and illusions (perception of coherent plaids, motion capture, motion coherence) as well as electrophysiological recordings. Thus, a single unifying principle ‘the final optical flow should be as smooth as possible’ (except at isolated motion discontinuities) explains a large number of phenomena and links single-cell behavior with perception and computational theory

Continuous-time segmentation networks

Segmentation is a basic problem in computer vision. The tiny-tanh network, a continuous-time network that segments scenes based upon intensity, motion, or depth is introduced. The tiny- tanh algorithm maps naturally to analog circuitry since it was inspired by previous experiments with analog VLSI segmentation hardware. A convex Lyapunov energy is utilized so that the system does not get stuck in local minima. No annealing algorithms of any kind are necessary- -a sharp contrast to previous software/hardware solutions of this problem

Computation of Smooth Optical Flow in a Feedback Connected Analog Network

In 1986, Tanner and Mead \cite{Tanner_Mead86} implemented an interesting constraint satisfaction circuit for global motion sensing in aVLSI. We report here a new and improved aVLSI implementation that provides smooth optical flow as well as global motion in a two dimensional visual field. The computation of optical flow is an ill-posed problem, which expresses itself as the aperture problem. However, the optical flow can be estimated by the use of regularization methods, in which additional constraints are introduced in terms of a global energy functional that must be minimized. We show how the algorithmic constraints of Horn and Schunck \cite{Horn_Schunck81} on computing smooth optical flow can be mapped onto the physical constraints of an equivalent electronic network

Toward color image segmentation in analog VLSI: Algorithm and hardware

Standard techniques for segmenting color images are based on finding normalized RGB discontinuities, color histogramming, or clustering techniques in RGB or CIE color spaces. The use of the psychophysical variable hue in HSI space has not been popular due to its numerical instability at low saturations. In this article, we propose the use of a simplified hue description suitable for implementation in analog VLSI. We demonstrate that if theintegrated white condition holds, hue is invariant to certain types of highlights, shading, and shadows. This is due to theadditive/shift invariance property, a property that other color variables lack. The more restrictive uniformly varying lighting model associated with themultiplicative/scale invariance property shared by both hue and normalized RGB allows invariance to transparencies, and to simple models of shading and shadows. Using binary hue discontinuities in conjunction with first-order type of surface interpolation, we demonstrate these invariant properties and compare them against the performance of RGB, normalized RGB, and CIE color spaces. We argue that working in HSI space offers an effective method for segmenting scenes in the presence of confounding cues due to shading, transparency, highlights, and shadows. Based on this work, we designed and fabricated for the first time an analog CMOS VLSI circuit with on-board phototransistor input that computes normalized color and hue

Optical Flow and Surface Interpolation in Resistive Networks: Algorithms and Analog VLSI Chips

To us, and to other biological organisms, vision seems effortless. We open our eyes and we "see" the world in all its color, brightness, and movement. Flies, frogs, cats, and humans can all equally well perceive a rapidly changing environment and act on it. Yet, we have great difficulties when trying to endow our machines with similar abilities. In this article, we describe recent developments in the theory of early vision that led from the formulation of the motion problem as an ill-posed one to its solution by minimizing certain "cost" functions. These cost or energy functions can be mapped onto simple analog and digital resistive networks. For instance, as detailed in this chapter, we can compute the optical flow by injecting currents into resistive networks and recording the resulting stationary voltage distribution at each node. These networks, which are implemented in subthreshold, analog, complementary metal oxide semiconductor (CMOS) very large scale integrated (VLSI) circuits, are very attractive for their technological potential

Negative fuse network

The authors describe the negative fuse, an analog model which encourages boundary completion in early vision regularization algorithms. This algorithm is an extension of the successful implementation of line processes in analog VLSI using the resistive fuse (Harris et al.). The negative fuse provides for true negative resistance regions for the enhancement of edges, making long connected edges more likely to occur. This model has a natural mapping into inexpensive, fast, low-power analog hardware. The authors discuss the performance of a negative fuse element fabricated in VLSI and show simulations of network performance on digitized camera images