The Role of Vergence Micromovements on Depth Perception

A new approach in stereo vision is proposed which recovers 3D depth information using continuous vergence angle control with simultaneous local correspondence response. This technique relates elements with the same relative position in the left and right images for a continuous sequence of vergence angles. the approach considers the extremely fine vergence movements about a given fixation point within the depth of field boundaries. It allows the recovery of 3D depth information given the knowledge of the system\u27s geometry and a sequence of pairs [αi, Ci], where αi is the ith vergence angle and Ci is the ith matrix of correspondence responses. The approach has several advantages over the current ones. First, due to its local operation characteristics, the resulting algorithms can be implemented in a modular hardware scheme. Second, unlike currently used algorithms, there is no need to compute depth from disparity values; at the cost of the acquisition of a sequence of images during the micromovements. The approach also greatly reduces the errors in stereo due to the sensor quantization. Last, and most important of all, the approach is supported by experimental results from physiology and psychophysics. Physiological results show that the human eye performs fine movements during the process of fixation on a single point, which are collectively called physiological nystagmus. One such movement, called binocular flicks, happens in opposing directions and produces convergence/divergence of the eyes. These are the micromovements that we suppose are the basis for depth perception. Therefore, the approach proposes a functional correlation between these vergence micromovements, depth perception, stereo acuity and stereo fusion

Motion-stereo mechanisms sensitive to inter-ocular phase

AbstractWe measured depth from interocular delay (The Pulfrich effect) using a dynamic random-dot pattern, consisting of a spatially-random noise field, the individual elements of which were sinusoidally-modulated in luminance over time. When an interocular phase difference in the flicker was introduced the display appeared to rotate in depth around a vertical axis like a transparent textured cylinder. The threshold phase lag was in the region of 5–10° in different observers, which translated into a non-constant, decreasing interocular delay (ms) as the flicker frequency was increased. We conclude that phase, not delay, is the critical parameter in determining the detection of depth. Threshold signal/noise ratios were measured at different delays to determine the optimum phase difference, which was found to be in the region 60–90°. However, delays centred around 180° were less detectable than those around zero, ruling out a quadrature input to the stereo-motion mechanisms. We show that depth-from-phase is a natural consequence of paired monocularly motion-direction sensitive neurones. Complex energy-detecting neurones are not required to explain the findings

Real Time Tracking of Moving Objects with an Active Camera

This article is concerned with the design and implementation of a system for real time monocular tracking of a moving object using the two degrees of freedom of a camera platform. Figure-ground segregation is based on motion without making any a priori assumptions about the object form. Using only the first spatiotemporal image derivatives subtraction of the normal optical flow induced by camera motion yields the object image motion. Closed-loop control is achieved by combining a stationary Kalman estimator with an optimal Linear Quadratic Regulator. The implementation on a pipeline architecture enables a servo rate of 25 Hz. We study the effects of time-recursive filtering and fixed-point arithmetic in image processing and we test the performance of the control algorithm on controlled motion of objects

NETRA: Interactive Display for Estimating Refractive Errors and Focal Range

We introduce an interactive, portable, and inexpensive solution for estimating refractive errors in the human eye. While expensive optical devices for automatic estimation of refractive correction exist, our goal is to greatly simplify the mechanism by putting the human subject in the loop. Our solution is based on a high-resolution programmable display and combines inexpensive optical elements, interactive GUI, and computational reconstruction. The key idea is to interface a lenticular view-dependent display with the human eye in close range - a few millimeters apart. Via this platform, we create a new range of interactivity that is extremely sensitive to parameters of the human eye, like refractive errors, focal range, focusing speed, lens opacity, etc. We propose several simple optical setups, verify their accuracy, precision, and validate them in a user study.Alfred P. Sloan Foundation (Research Fellowship

Automating Active Stereo Vision Calibration Process with Cobots

Collaborative robots help the academia and industry to accelerate the work by introducing a new concept of cooperation between human and robot. In this paper, a calibration process for an active stereo vision rig has been automated to accelerate the task and improve the quality of the calibration. As illustrated in this paper by using Baxter Robot, the calibration process has been done faster by three times in comparison to the manual calibration that depends on the human. The quality of the calibration was improved by 120% when the Baxter robot was used

Recent Progress in Image Deblurring

This paper comprehensively reviews the recent development of image deblurring, including non-blind/blind, spatially invariant/variant deblurring techniques. Indeed, these techniques share the same objective of inferring a latent sharp image from one or several corresponding blurry images, while the blind deblurring techniques are also required to derive an accurate blur kernel. Considering the critical role of image restoration in modern imaging systems to provide high-quality images under complex environments such as motion, undesirable lighting conditions, and imperfect system components, image deblurring has attracted growing attention in recent years. From the viewpoint of how to handle the ill-posedness which is a crucial issue in deblurring tasks, existing methods can be grouped into five categories: Bayesian inference framework, variational methods, sparse representation-based methods, homography-based modeling, and region-based methods. In spite of achieving a certain level of development, image deblurring, especially the blind case, is limited in its success by complex application conditions which make the blur kernel hard to obtain and be spatially variant. We provide a holistic understanding and deep insight into image deblurring in this review. An analysis of the empirical evidence for representative methods, practical issues, as well as a discussion of promising future directions are also presented.Comment: 53 pages, 17 figure

Deformable Beamsplitters: Enhancing Perception with Wide Field of View, Varifocal Augmented Reality Displays

An augmented reality head-mounted display with full environmental awareness could present data in new ways and provide a new type of experience, allowing seamless transitions between real life and virtual content. However, creating a light-weight, optical see-through display providing both focus support and wide field of view remains a challenge. This dissertation describes a new dynamic optical element, the deformable beamsplitter, and its applications for wide field of view, varifocal, augmented reality displays. Deformable beamsplitters combine a traditional deformable membrane mirror and a beamsplitter into a single element, allowing reflected light to be manipulated by the deforming membrane mirror, while transmitted light remains unchanged. This research enables both single element optical design and correct focus while maintaining a wide field of view, as demonstrated by the description and analysis of two prototype hardware display systems which incorporate deformable beamsplitters. As a user changes the depth of their gaze when looking through these displays, the focus of virtual content can quickly be altered to match the real world by simply modulating air pressure in a chamber behind the deformable beamsplitter; thus ameliorating vergence–accommodation conflict. Two user studies verify the display prototypes’ capabilities and show the potential of the display in enhancing human performance at quickly perceiving visual stimuli. This work shows that near-eye displays built with deformable beamsplitters allow for simple optical designs that enable wide field of view and comfortable viewing experiences with the potential to enhance user perception.Doctor of Philosoph

Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world