708 research outputs found
End-to-end Projector Photometric Compensation
Projector photometric compensation aims to modify a projector input image
such that it can compensate for disturbance from the appearance of projection
surface. In this paper, for the first time, we formulate the compensation
problem as an end-to-end learning problem and propose a convolutional neural
network, named CompenNet, to implicitly learn the complex compensation
function. CompenNet consists of a UNet-like backbone network and an autoencoder
subnet. Such architecture encourages rich multi-level interactions between the
camera-captured projection surface image and the input image, and thus captures
both photometric and environment information of the projection surface. In
addition, the visual details and interaction information are carried to deeper
layers along the multi-level skip convolution layers. The architecture is of
particular importance for the projector compensation task, for which only a
small training dataset is allowed in practice. Another contribution we make is
a novel evaluation benchmark, which is independent of system setup and thus
quantitatively verifiable. Such benchmark is not previously available, to our
best knowledge, due to the fact that conventional evaluation requests the
hardware system to actually project the final results. Our key idea, motivated
from our end-to-end problem formulation, is to use a reasonable surrogate to
avoid such projection process so as to be setup-independent. Our method is
evaluated carefully on the benchmark, and the results show that our end-to-end
learning solution outperforms state-of-the-arts both qualitatively and
quantitatively by a significant margin.Comment: To appear in the 2019 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR). Source code and dataset are available at
https://github.com/BingyaoHuang/compenne
Detection of Non-Stationary Photometric Perturbations on Projection Screens
Interfaces based on projection screens have become increasingly
more popular in recent years, mainly due to the large screen size and resolution
that they provide, as well as their stereo-vision capabilities. This work shows a
local method for real-time detection of non-stationary photometric perturbations
in projected images by means of computer vision techniques. The method is
based on the computation of differences between the images in the projector’s
frame buffer and the corresponding images on the projection screen observed
by the camera. It is robust under spatial variations in the intensity of light
emitted by the projector on the projection surface and also robust under
stationary photometric perturbations caused by external factors. Moreover, we
describe the experiments carried out to show the reliability of the method
Multi-View Neural Surface Reconstruction with Structured Light
Three-dimensional (3D) object reconstruction based on differentiable
rendering (DR) is an active research topic in computer vision. DR-based methods
minimize the difference between the rendered and target images by optimizing
both the shape and appearance and realizing a high visual reproductivity.
However, most approaches perform poorly for textureless objects because of the
geometrical ambiguity, which means that multiple shapes can have the same
rendered result in such objects. To overcome this problem, we introduce active
sensing with structured light (SL) into multi-view 3D object reconstruction
based on DR to learn the unknown geometry and appearance of arbitrary scenes
and camera poses. More specifically, our framework leverages the
correspondences between pixels in different views calculated by structured
light as an additional constraint in the DR-based optimization of implicit
surface, color representations, and camera poses. Because camera poses can be
optimized simultaneously, our method realizes high reconstruction accuracy in
the textureless region and reduces efforts for camera pose calibration, which
is required for conventional SL-based methods. Experiment results on both
synthetic and real data demonstrate that our system outperforms conventional
DR- and SL-based methods in a high-quality surface reconstruction, particularly
for challenging objects with textureless or shiny surfaces.Comment: Accepted by BMVC 202
INFORMATION TECHNOLOGY FOR NEXT-GENERATION OF SURGICAL ENVIRONMENTS
Minimally invasive surgeries (MIS) are fundamentally constrained by image quality,access to the operative field, and the visualization environment on which thesurgeon relies for real-time information. Although invasive access benefits the patient,it also leads to more challenging procedures, which require better skills andtraining. Endoscopic surgeries rely heavily on 2D interfaces, introducing additionalchallenges due to the loss of depth perception, the lack of 3-Dimensional imaging,and the reduction of degrees of freedom.By using state-of-the-art technology within a distributed computational architecture,it is possible to incorporate multiple sensors, hybrid display devices, and3D visualization algorithms within a exible surgical environment. Such environmentscan assist the surgeon with valuable information that goes far beyond what iscurrently available. In this thesis, we will discuss how 3D visualization and reconstruction,stereo displays, high-resolution display devices, and tracking techniques arekey elements in the next-generation of surgical environments
A Multi-Projector Calibration Method for Virtual Reality Simulators with Analytically Defined Screens
The geometric calibration of projectors is a demanding task, particularly for the industry of virtual reality simulators. Different methods have been developed during the last decades to retrieve the intrinsic and extrinsic parameters of projectors, most of them being based on planar homographies and some requiring an extended calibration process. The aim of our research work is to design a fast and user-friendly method to provide multi-projector calibration on analytically defined screens, where a sample is shown for a virtual reality Formula 1 simulator that has a cylindrical screen. The proposed method results from the combination of surveying, photogrammetry and image processing approaches, and has been designed by considering the spatial restrictions of virtual reality simulators. The method has been validated from a mathematical point of view, and the complete system which is currently installed in a shopping mall in Spain has been tested by different users
- …