2,420 research outputs found
Depth Super-Resolution Meets Uncalibrated Photometric Stereo
A novel depth super-resolution approach for RGB-D sensors is presented. It
disambiguates depth super-resolution through high-resolution photometric clues
and, symmetrically, it disambiguates uncalibrated photometric stereo through
low-resolution depth cues. To this end, an RGB-D sequence is acquired from the
same viewing angle, while illuminating the scene from various uncalibrated
directions. This sequence is handled by a variational framework which fits
high-resolution shape and reflectance, as well as lighting, to both the
low-resolution depth measurements and the high-resolution RGB ones. The key
novelty consists in a new PDE-based photometric stereo regularizer which
implicitly ensures surface regularity. This allows to carry out depth
super-resolution in a purely data-driven manner, without the need for any
ad-hoc prior or material calibration. Real-world experiments are carried out
using an out-of-the-box RGB-D sensor and a hand-held LED light source.Comment: International Conference on Computer Vision (ICCV) Workshop, 201
MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework
We propose MeshfreeFlowNet, a novel deep learning-based super-resolution
framework to generate continuous (grid-free) spatio-temporal solutions from the
low-resolution inputs. While being computationally efficient, MeshfreeFlowNet
accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet
allows for: (i) the output to be sampled at all spatio-temporal resolutions,
(ii) a set of Partial Differential Equation (PDE) constraints to be imposed,
and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal
domains owing to its fully convolutional encoder. We empirically study the
performance of MeshfreeFlowNet on the task of super-resolution of turbulent
flows in the Rayleigh-Benard convection problem. Across a diverse set of
evaluation metrics, we show that MeshfreeFlowNet significantly outperforms
existing baselines. Furthermore, we provide a large scale implementation of
MeshfreeFlowNet and show that it efficiently scales across large clusters,
achieving 96.80% scaling efficiency on up to 128 GPUs and a training time of
less than 4 minutes.Comment: Supplementary Video: https://youtu.be/mjqwPch9gDo. Accepted to SC2
Real Time Turbulent Video Perfecting by Image Stabilization and Super-Resolution
Image and video quality in Long Range Observation Systems (LOROS) suffer from
atmospheric turbulence that causes small neighbourhoods in image frames to
chaotically move in different directions and substantially hampers visual
analysis of such image and video sequences. The paper presents a real-time
algorithm for perfecting turbulence degraded videos by means of stabilization
and resolution enhancement. The latter is achieved by exploiting the turbulent
motion. The algorithm involves generation of a reference frame and estimation,
for each incoming video frame, of a local image displacement map with respect
to the reference frame; segmentation of the displacement map into two classes:
stationary and moving objects and resolution enhancement of stationary objects,
while preserving real motion. Experiments with synthetic and real-life
sequences have shown that the enhanced videos, generated in real time, exhibit
substantially better resolution and complete stabilization for stationary
objects while retaining real motion.Comment: Submitted to The Seventh IASTED International Conference on
Visualization, Imaging, and Image Processing (VIIP 2007) August, 2007 Palma
de Mallorca, Spai
Physics-Informed Computer Vision: A Review and Perspectives
Incorporation of physical information in machine learning frameworks are
opening and transforming many application domains. Here the learning process is
augmented through the induction of fundamental knowledge and governing physical
laws. In this work we explore their utility for computer vision tasks in
interpreting and understanding visual data. We present a systematic literature
review of formulation and approaches to computer vision tasks guided by
physical laws. We begin by decomposing the popular computer vision pipeline
into a taxonomy of stages and investigate approaches to incorporate governing
physical equations in each stage. Existing approaches in each task are analyzed
with regard to what governing physical processes are modeled, formulated and
how they are incorporated, i.e. modify data (observation bias), modify networks
(inductive bias), and modify losses (learning bias). The taxonomy offers a
unified view of the application of the physics-informed capability,
highlighting where physics-informed learning has been conducted and where the
gaps and opportunities are. Finally, we highlight open problems and challenges
to inform future research. While still in its early days, the study of
physics-informed computer vision has the promise to develop better computer
vision models that can improve physical plausibility, accuracy, data efficiency
and generalization in increasingly realistic applications
A novel method for surface exploration: Super-resolution restoration of Mars repeat-pass orbital imagery
Higher resolution imaging data of planetary surfaces is considered desirable by the international community of planetary scientists interested in improving understanding of surface formation processes. However, given various physical constraints from the imaging instruments through to limited bandwidth of transmission one needs to trade-off spatial resolution against bandwidth. Even given optical communications, future imaging systems are unlikely to be able to resolve features smaller than 25 cm on most planetary bodies, such as Mars. In this paper, we propose a novel super-resolution restoration technique, called Gotcha-PDE-TV (GPT), taking advantage of the non-redundant sub-pixel information contained in multiple raw orbital images in order to restore higher resolution imagery. We demonstrate optimality of this technique in planetary image super-resolution restoration with example processing of 8 repeat-pass 25 cm HiRISE images covering the MER-A Spirit rover traverse in Gusev crater to resolve a 5 cm resolution of the area. We assess the “true” resolution of the 5 cm super-resolution restored images using contemporaneous rover Navcam imagery on the surface and an inter-comparison of landmarks in the two sets of imagery
Physics-informed Deep Super-resolution for Spatiotemporal Data
High-fidelity simulation of complex physical systems is exorbitantly
expensive and inaccessible across spatiotemporal scales. Recently, there has
been an increasing interest in leveraging deep learning to augment scientific
data based on the coarse-grained simulations, which is of cheap computational
expense and retains satisfactory solution accuracy. However, the major existing
work focuses on data-driven approaches which rely on rich training datasets and
lack sufficient physical constraints. To this end, we propose a novel and
efficient spatiotemporal super-resolution framework via physics-informed
learning, inspired by the independence between temporal and spatial derivatives
in partial differential equations (PDEs). The general principle is to leverage
the temporal interpolation for flow estimation, and then introduce
convolutional-recurrent neural networks for learning temporal refinement.
Furthermore, we employ the stacked residual blocks with wide activation and
sub-pixel layers with pixelshuffle for spatial reconstruction, where feature
extraction is conducted in a low-resolution latent space. Moreover, we consider
hard imposition of boundary conditions in the network to improve reconstruction
accuracy. Results demonstrate the superior effectiveness and efficiency of the
proposed method compared with baseline algorithms through extensive numerical
experiments
- …