Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

The representation of the visual field on the sub-cortical centers of the cat and rabbit

In the visual system the primary problem has been the identification of the visual pathway - the fiber systems along which the impulses arising in the eyes pass to the various subcortical levels and finally to the cortex - their precise localization, boundaries, and the extent of the terminal areas of the pathway. The second problem has been to determine whether, and to what extent, the original spatial and dimensional relationships present in visual space and in the retina are preserved in the pathways and centers, and if they are, how and where the various distinct areas of the retina, i.e. the fovea, the extra-foveal peripheral quadrants, the horizontal and vertical meridians, were represented in the various levels of the visual pathway

Neuromorphic stereo vision: A survey of bio-inspired sensors and algorithms

Any visual sensor, whether artificial or biological, maps the 3D-world on a 2D-representation. The missing dimension is depth and most species use stereo vision to recover it. Stereo vision implies multiple perspectives and matching, hence it obtains depth from a pair of images. Algorithms for stereo vision are also used prosperously in robotics. Although, biological systems seem to compute disparities effortless, artificial methods suffer from high energy demands and latency. The crucial part is the correspondence problem; finding the matching points of two images. The development of event-based cameras, inspired by the retina, enables the exploitation of an additional physical constraint—time. Due to their asynchronous course of operation, considering the precise occurrence of spikes, Spiking Neural Networks take advantage of this constraint. In this work, we investigate sensors and algorithms for event-based stereo vision leading to more biologically plausible robots. Hereby, we focus mainly on binocular stereo vision

Factors which determine visual acuity

The object of this study has been the investigation of the cortical and subcortical projection of the rabbits' visual field, with particular emphasis on the magnification that certain regions of the visual field undergo in the process

Aerospace Medicine and Biology. A continuing bibliography with indexes

This bibliography lists 244 reports, articles, and other documents introduced into the NASA scientific and technical information system in February 1981. Aerospace medicine and aerobiology topics are included. Listings for physiological factors, astronaut performance, control theory, artificial intelligence, and cybernetics are included

VLSI analogs of neuronal visual processing: a synthesis of form and function

This thesis describes the development and testing of a simple visual system fabricated using complementary metal-oxide-semiconductor (CMOS) very large scale integration (VLSI) technology. This visual system is composed of three subsystems. A silicon retina, fabricated on a single chip, transduces light and performs signal processing in a manner similar to a simple vertebrate retina. A stereocorrespondence chip uses bilateral retinal input to estimate the location of objects in depth. A silicon optic nerve allows communication between chips by a method that preserves the idiom of action potential transmission in the nervous system. Each of these subsystems illuminates various aspects of the relationship between VLSI analogs and their neurobiological counterparts. The overall synthetic visual system demonstrates that analog VLSI can capture a significant portion of the function of neural structures at a systems level, and concomitantly, that incorporating neural architectures leads to new engineering approaches to computation in VLSI. The relationship between neural systems and VLSI is rooted in the shared limitations imposed by computing in similar physical media. The systems discussed in this text support the belief that the physical limitations imposed by the computational medium significantly affect the evolving algorithm. Since circuits are essentially physical structures, I advocate the use of analog VLSI as powerful medium of abstraction, suitable for understanding and expressing the function of real neural systems. The working chip elevates the circuit description to a kind of synthetic formalism. The behaving physical circuit provides a formal test of theories of function that can be expressed in the language of circuits

Application of holography to flow visualization within rotating compressor blade row

Two holographic interblade row flow visualization systems were designed to determine the three-dimensional shock patterns and velocity distributions within the rotating blade row of a transonic fan rotor, utilizing the techniques of pulsed laser transmission holography. Both single- and double-exposure bright field holograms and dark field scattered-light holograms were successfully recorded. Two plastic windows were installed in the rotor tip casing and outer casing forward of the rotor to view the rotor blade passage. The viewing angle allowed detailed investigation of the leading edge shocks and shocks in the midspan damper area; limited details of the trailing edge shocks also were visible. A technique was devised for interpreting the reconstructed holograms by constructing three dimensional models that allowed identification of the major shock systems. The models compared favorably with theoretical predictions and results of the overall and blade element data. Most of the holograms were made using the rapid double-pulse technique