Motionless active depth from defocus system using smart optics for camera autofocus applications

This paper describes a motionless active Depth from Defocus (DFD) system design suited for long working range camera autofocus applications. The design consists of an active illumination module that projects a scene illuminating coherent conditioned optical radiation pattern which maintains its sharpness over multiple axial distances allowing an increased DFD working distance range. The imager module of the system responsible for the actual DFD operation deploys an electronically controlled variable focus lens (ECVFL) as a smart optic to enable a motionless imager design capable of effective DFD operation. An experimental demonstration is conducted in the laboratory which compares the effectiveness of the coherent conditioned radiation module versus a conventional incoherent active light source, and demonstrates the applicability of the presented motionless DFD imager design. The fast response and no-moving-parts features of the DFD imager design are especially suited for camera scenarios where mechanical motion of lenses to achieve autofocus action is challenging, for example, in the tiny camera housings in smartphones and tablets. Applications for the proposed system include autofocus in modern day digital cameras

"Plug-and-Play" Edge-Preserving Regularization

In many inverse problems it is essential to use regularization methods that preserve edges in the reconstructions, and many reconstruction models have been developed for this task, such as the Total Variation (TV) approach. The associated algorithms are complex and require a good knowledge of large-scale optimization algorithms, and they involve certain tolerances that the user must choose. We present a simpler approach that relies only on standard computational building blocks in matrix computations, such as orthogonal transformations, preconditioned iterative solvers, Kronecker products, and the discrete cosine transform -- hence the term "plug-and-play." We do not attempt to improve on TV reconstructions, but rather provide an easy-to-use approach to computing reconstructions with similar properties.Comment: 14 pages, 7 figures, 3 table

Focal Surface Projection: Extending Projector Depth-of-Field Using a Phase-Only Spatial Light Modulator

We present a focal surface projection to solve the narrow depth-of-field problem in projection mapping applications. We apply a phase-only spatial light modulator to realize nonuniform focusing distances, whereby the projected contents appear focused on a surface with considerable depth variations. The feasibility of the proposed technique was validated through a physical experiment

Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping

Kageyama Y., Iwai D., Sato K.. Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping. IEEE Transactions on Visualization and Computer Graphics , (2024); https://doi.org/10.1109/TVCG.2024.3354957.Dynamic Projection Mapping (DPM) necessitates geometric compensation of the projection image based on the position and orientation of moving objects. Additionally, the projector's shallow depth of field results in pronounced defocus blur even with minimal object movement. Achieving delay-free DPM with high image quality requires real-time implementation of geometric compensation and projector deblurring. To meet this demand, we propose a framework comprising two neural components: one for geometric compensation and another for projector deblurring. The former component warps the image by detecting the optical flow of each pixel in both the projection and captured images. The latter component performs real-time sharpening as needed. Ideally, our network's parameters should be trained on data acquired in an actual environment. However, training the network from scratch while executing DPM, which demands real-time image generation, is impractical. Therefore, the network must undergo pre-training. Unfortunately, there are no publicly available large real datasets for DPM due to the diverse image quality degradation patterns. To address this challenge, we propose a realistic synthetic data generation method that numerically models geometric distortion and defocus blur in real-world DPM. Through exhaustive experiments, we have confirmed that the model trained on the proposed dataset achieves projector deblurring in the presence of geometric distortions with a quality comparable to state-of-the-art methods

An Efficient Method for Projection-Based Image Deblurring

Abstract Previously lot of problems are encountered in image processing -image deblurring. From the origination of the image deblurring to its broad applications in enormous number of areas today, the deblurring approaches have evolved with time and forked into many different and fascinating branches. In this paper, we propose a method for reducing out-of-focus blur caused by projector projection. We estimate the Point-Spread-Function (PSF) in the image projected onto the screen by using a camera that captures the projector screen. According to the estimated PSF, the original image is pre-corrected, so that the projected image can be deblurred. Proposed system can successfully reduce the effects of out-offocus projection blur, even though the screen image includes spatially varying blur without projecting the feature image

Correcting for optical aberrations using multilayer displays

Optical aberrations of the human eye are currently corrected using eyeglasses, contact lenses, or surgery. We describe a fourth option: modifying the composition of displayed content such that the perceived image appears in focus, after passing through an eye with known optical defects. Prior approaches synthesize pre-filtered images by deconvolving the content by the point spread function of the aberrated eye. Such methods have not led to practical applications, due to severely reduced contrast and ringing artifacts. We address these limitations by introducing multilayer pre-filtering, implemented using stacks of semi-transparent, light-emitting layers. By optimizing the layer positions and the partition of spatial frequencies between layers, contrast is improved and ringing artifacts are eliminated. We assess design constraints for multilayer displays; autostereoscopic light field displays are identified as a preferred, thin form factor architecture, allowing synthetic layers to be displaced in response to viewer movement and refractive errors. We assess the benefits of multilayer pre-filtering versus prior light field pre-distortion methods, showing pre-filtering works within the constraints of current display resolutions. We conclude by analyzing benefits and limitations using a prototype multilayer LCD.National Science Foundation (U.S.) (Grant IIS-1116452)Alfred P. Sloan Foundation (Research Fellowship)United States. Defense Advanced Research Projects Agency (Young Faculty Award)Vodafone (Firm) (Wireless Innovation Award

Coded Aperture Projection

In computer vision, optical defocus is often described as convolution with a filter kernel that corresponds to an image of the aperture being used by the imaging device. The degree of defocus correlates to the scale of the kernel. Convolving an image with the inverse aperture kernel will digitally sharpen the image and consequently compensate optical defocus. This is referred to as deconvolution or inverse filtering. In frequency domain, the reciprocal of the filter kernel is its inverse, and deconvolution reduces to a division. Low magnitudes in the Fourier transform of the aperture image, however, lead to intensity values in spatial domain that exceed the displayable range. Therefore, the corresponding frequencies are not considered, which then results in visible ringing artifacts in the final projection. This is the main limitation of previous approaches, since in frequency domain the Gaussian PSF of spherical apertures does contain a large fraction of low Fourier magnitudes. Applying only small kernel scales will reduce the number of low Fourier magnitudes (and consequently the ringing artifacts) -- but will also lead only to minor focus improvements. To overcome this problem, we apply a coded aperture whose Fourier transform has less low magnitudes initially. Consequently, more frequencies are retained and more image details are reconstructed

Improving Depth Perception in Immersive Media Devices by Addressing Vergence-Accommodation Conflict

: Recently, immersive media devices have seen a boost in popularity. However, many problems still remain. Depth perception is a crucial part of how humans behave and interact with their environment. Convergence and accommodation are two physiological mechanisms that provide important depth cues. However, when humans are immersed in virtual environments, they experience a mismatch between these cues. This mismatch causes users to feel discomfort while also hindering their ability to fully perceive object distances. To address the conflict, we have developed a technique that encompasses inverse blurring into immersive media devices. For the inverse blurring, we utilize the classical Wiener deconvolution approach by proposing a novel technique that is applied without the need for an eye-tracker and implemented in a commercial immersive media device. The technique's ability to compensate for the vergence-accommodation conflict was verified through two user studies aimed at reaching and spatial awareness, respectively. The two studies yielded a statistically significant 36% and 48% error reduction in user performance to estimate distances, respectively. Overall, the work done demonstrates how visual stimuli can be modified to allow users to achieve a more natural perception and interaction with the virtual environment

V-cycle optimal convergence for certain (multilevel) structured linear systems

In this paper we are interested in the solution by multigrid strategies of multilevel linear systems whose coefficient matrices belong to the circulant, Hartley, or \u3c4 algebras or to the Toeplitz class and are generated by (the Fourier expansion of) a nonnegative multivariate polynomial f. It is well known that these matrices are banded and have eigenvalues equally distributed as f, so they are ill-conditioned whenever f takes the zero value; they can even be singular and need a low-rank correction. We prove the V-cycle multigrid iteration to have a convergence rate independent of the dimension even in presence of ill-conditioning. If the (multilevel) coefficient matrix has partial dimension nr at level r, r = 1, . . . ,d, then the size of the algebraic system is N(n) = \u3a0r=1 d nr, O(N(n)) operations are required by our technique, and therefore the corresponding method is optimal. Some numerical experiments concerning linear systems arising in applications, such as elliptic PDEs with mixed boundary conditions and image restoration problems, are considered and discussed.cussed