Live User-guided Intrinsic Video For Static Scenes

We present a novel real-time approach for user-guided intrinsic decomposition of static scenes captured by an RGB-D sensor. In the first step, we acquire a three-dimensional representation of the scene using a dense volumetric reconstruction framework. The obtained reconstruction serves as a proxy to densely fuse reflectance estimates and to store user-provided constraints in three-dimensional space. User constraints, in the form of constant shading and reflectance strokes, can be placed directly on the real-world geometry using an intuitive touch-based interaction metaphor, or using interactive mouse strokes. Fusing the decomposition results and constraints in three-dimensional space allows for robust propagation of this information to novel views by re-projection.We leverage this information to improve on the decomposition quality of existing intrinsic video decomposition techniques by further constraining the ill-posed decomposition problem. In addition to improved decomposition quality, we show a variety of live augmented reality applications such as recoloring of objects, relighting of scenes and editing of material appearance

Statistical and systematic uncertainties in pixel-based source reconstruction algorithms for gravitational lensing

Gravitational lens modeling of spatially resolved sources is a challenging inverse problem with many observational constraints and model parameters. We examine established pixel-based source reconstruction algorithms for de-lensing the source and constraining lens model parameters. Using test data for four canonical lens configurations, we explore statistical and systematic uncertainties associated with gridding, source regularisation, interpolation errors, noise, and telescope pointing. Specifically, we compare two gridding schemes in the source plane: a fully adaptive grid that follows the lens mapping but is irregular, and an adaptive Cartesian grid. We also consider regularisation schemes that minimise derivatives of the source (using two finite difference methods) and introduce a scheme that minimises deviations from an analytic source profile. Careful choice of gridding and regularisation can reduce "discreteness noise" in the $\chi^2$ surface that is inherent in the pixel-based methodology. With a gridded source, some degree of interpolation is unavoidable, and errors due to interpolation need to be taken into account (especially for high signal-to-noise data). Different realisations of the noise and telescope pointing lead to slightly different values for lens model parameters, and the scatter between different "observations" can be comparable to or larger than the model uncertainties themselves. The same effects create scatter in the lensing magnification at the level of a few percent for a peak signal-to-noise ratio of 10, which decreases as the data quality improves.Comment: 20 pages, 18 figures, accepted to MNRAS, see http://physics.rutgers.edu/~tagoreas/papers/ for high resolution image

Regularized Newton Methods for X-ray Phase Contrast and General Imaging Problems

Like many other advanced imaging methods, x-ray phase contrast imaging and tomography require mathematical inversion of the observed data to obtain real-space information. While an accurate forward model describing the generally nonlinear image formation from a given object to the observations is often available, explicit inversion formulas are typically not known. Moreover, the measured data might be insufficient for stable image reconstruction, in which case it has to be complemented by suitable a priori information. In this work, regularized Newton methods are presented as a general framework for the solution of such ill-posed nonlinear imaging problems. For a proof of principle, the approach is applied to x-ray phase contrast imaging in the near-field propagation regime. Simultaneous recovery of the phase- and amplitude from a single near-field diffraction pattern without homogeneity constraints is demonstrated for the first time. The presented methods further permit all-at-once phase contrast tomography, i.e. simultaneous phase retrieval and tomographic inversion. We demonstrate the potential of this approach by three-dimensional imaging of a colloidal crystal at 95 nm isotropic resolution.Comment: (C)2016 Optical Society of America. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modifications of the content of this paper are prohibite

Adaptive foveated single-pixel imaging with dynamic super-sampling

As an alternative to conventional multi-pixel cameras, single-pixel cameras enable images to be recorded using a single detector that measures the correlations between the scene and a set of patterns. However, to fully sample a scene in this way requires at least the same number of correlation measurements as there are pixels in the reconstructed image. Therefore single-pixel imaging systems typically exhibit low frame-rates. To mitigate this, a range of compressive sensing techniques have been developed which rely on a priori knowledge of the scene to reconstruct images from an under-sampled set of measurements. In this work we take a different approach and adopt a strategy inspired by the foveated vision systems found in the animal kingdom - a framework that exploits the spatio-temporal redundancy present in many dynamic scenes. In our single-pixel imaging system a high-resolution foveal region follows motion within the scene, but unlike a simple zoom, every frame delivers new spatial information from across the entire field-of-view. Using this approach we demonstrate a four-fold reduction in the time taken to record the detail of rapidly evolving features, whilst simultaneously accumulating detail of more slowly evolving regions over several consecutive frames. This tiered super-sampling technique enables the reconstruction of video streams in which both the resolution and the effective exposure-time spatially vary and adapt dynamically in response to the evolution of the scene. The methods described here can complement existing compressive sensing approaches and may be applied to enhance a variety of computational imagers that rely on sequential correlation measurements.Comment: 13 pages, 5 figure

Process of image super-resolution

In this paper we explain a process of super-resolution reconstruction allowing to increase the resolution of an image.The need for high-resolution digital images exists in diverse domains, for example the medical and spatial domains. The obtaining of high-resolution digital images can be made at the time of the shooting, but it is often synonymic of important costs because of the necessary material to avoid such costs, it is known how to use methods of super-resolution reconstruction, consisting from one or several low resolution images to obtain a high-resolution image. The american patent US 9208537 describes such an algorithm. A zone of one low-resolution image is isolated and categorized according to the information contained in pixels forming the borders of the zone. The category of it zone determines the type of interpolation used to add pixels in aforementioned zone, to increase the neatness of the images. It is also known how to reconstruct a low-resolution image there high-resolution image by using a model of super-resolution reconstruction whose learning is based on networks of neurons and on image or a picture library. The demand of chinese patent CN 107563965 and the scientist publication "Pixel Recursive Super Resolution", R. Dahl, M. Norouzi, J. Shlens propose such methods. The aim of this paper is to demonstrate that it is possible to reconstruct coherent human faces from very degraded pixelated images with a very fast algorithm, more faster than compressed sensing (CS), easier to compute and without deep learning, so without important technology resources, i.e. a large database of thousands training images (see arXiv:2003.13063). This technological breakthrough has been patented in 2018 with the demand of French patent FR 1855485 (https://patents.google.com/patent/FR3082980A1, see the HAL reference https://hal.archives-ouvertes.fr/hal-01875898v1).Comment: 19 pages, 10 figure

Accelerated Cardiac Diffusion Tensor Imaging Using Joint Low-Rank and Sparsity Constraints

Objective: The purpose of this manuscript is to accelerate cardiac diffusion tensor imaging (CDTI) by integrating low-rankness and compressed sensing. Methods: Diffusion-weighted images exhibit both transform sparsity and low-rankness. These properties can jointly be exploited to accelerate CDTI, especially when a phase map is applied to correct for the phase inconsistency across diffusion directions, thereby enhancing low-rankness. The proposed method is evaluated both ex vivo and in vivo, and is compared to methods using either a low-rank or sparsity constraint alone. Results: Compared to using a low-rank or sparsity constraint alone, the proposed method preserves more accurate helix angle features, the transmural continuum across the myocardium wall, and mean diffusivity at higher acceleration, while yielding significantly lower bias and higher intraclass correlation coefficient. Conclusion: Low-rankness and compressed sensing together facilitate acceleration for both ex vivo and in vivo CDTI, improving reconstruction accuracy compared to employing either constraint alone. Significance: Compared to previous methods for accelerating CDTI, the proposed method has the potential to reach higher acceleration while preserving myofiber architecture features which may allow more spatial coverage, higher spatial resolution and shorter temporal footprint in the future.Comment: 11 pages, 16 figures, published on IEEE Transactions on Biomedical Engineerin