792 research outputs found
Wireless Software Synchronization of Multiple Distributed Cameras
We present a method for precisely time-synchronizing the capture of image
sequences from a collection of smartphone cameras connected over WiFi. Our
method is entirely software-based, has only modest hardware requirements, and
achieves an accuracy of less than 250 microseconds on unmodified commodity
hardware. It does not use image content and synchronizes cameras prior to
capture. The algorithm operates in two stages. In the first stage, we designate
one device as the leader and synchronize each client device's clock to it by
estimating network delay. Once clocks are synchronized, the second stage
initiates continuous image streaming, estimates the relative phase of image
timestamps between each client and the leader, and shifts the streams into
alignment. We quantitatively validate our results on a multi-camera rig imaging
a high-precision LED array and qualitatively demonstrate significant
improvements to multi-view stereo depth estimation and stitching of dynamic
scenes. We release as open source 'libsoftwaresync', an Android implementation
of our system, to inspire new types of collective capture applications.Comment: Main: 9 pages, 10 figures. Supplemental: 3 pages, 5 figure
Towards High-Frequency Tracking and Fast Edge-Aware Optimization
This dissertation advances the state of the art for AR/VR tracking systems by
increasing the tracking frequency by orders of magnitude and proposes an
efficient algorithm for the problem of edge-aware optimization.
AR/VR is a natural way of interacting with computers, where the physical and
digital worlds coexist. We are on the cusp of a radical change in how humans
perform and interact with computing. Humans are sensitive to small
misalignments between the real and the virtual world, and tracking at
kilo-Hertz frequencies becomes essential. Current vision-based systems fall
short, as their tracking frequency is implicitly limited by the frame-rate of
the camera. This thesis presents a prototype system which can track at orders
of magnitude higher than the state-of-the-art methods using multiple commodity
cameras. The proposed system exploits characteristics of the camera
traditionally considered as flaws, namely rolling shutter and radial
distortion. The experimental evaluation shows the effectiveness of the method
for various degrees of motion.
Furthermore, edge-aware optimization is an indispensable tool in the computer
vision arsenal for accurate filtering of depth-data and image-based rendering,
which is increasingly being used for content creation and geometry processing
for AR/VR. As applications increasingly demand higher resolution and speed,
there exists a need to develop methods that scale accordingly. This
dissertation proposes such an edge-aware optimization framework which is
efficient, accurate, and algorithmically scales well, all of which are much
desirable traits not found jointly in the state of the art. The experiments
show the effectiveness of the framework in a multitude of computer vision tasks
such as computational photography and stereo.Comment: PhD thesi
Rolling Shutter Stereo
A huge fraction of cameras used nowadays is based on
CMOS sensors with a rolling shutter that exposes the image
line by line. For dynamic scenes/cameras this introduces
undesired effects like stretch, shear and wobble. It has been
shown earlier that rotational shake induced rolling shutter
effects in hand-held cell phone capture can be compensated
based on an estimate of the camera rotation. In contrast, we
analyse the case of significant camera motion, e.g. where
a bypassing streetlevel capture vehicle uses a rolling shutter
camera in a 3D reconstruction framework. The introduced
error is depth dependent and cannot be compensated
based on camera motion/rotation alone, invalidating also
rectification for stereo camera systems. On top, significant
lens distortion as often present in wide angle cameras intertwines
with rolling shutter effects as it changes the time
at which a certain 3D point is seen. We show that naive
3D reconstructions (assuming global shutter) will deliver
biased geometry already for very mild assumptions on vehicle
speed and resolution. We then develop rolling shutter
dense multiview stereo algorithms that solve for time of exposure
and depth at the same time, even in the presence of
lens distortion and perform an evaluation on ground truth
laser scan models as well as on real street-level data
- …