660 research outputs found
Towards High-Frequency Tracking and Fast Edge-Aware Optimization
This dissertation advances the state of the art for AR/VR tracking systems by
increasing the tracking frequency by orders of magnitude and proposes an
efficient algorithm for the problem of edge-aware optimization.
AR/VR is a natural way of interacting with computers, where the physical and
digital worlds coexist. We are on the cusp of a radical change in how humans
perform and interact with computing. Humans are sensitive to small
misalignments between the real and the virtual world, and tracking at
kilo-Hertz frequencies becomes essential. Current vision-based systems fall
short, as their tracking frequency is implicitly limited by the frame-rate of
the camera. This thesis presents a prototype system which can track at orders
of magnitude higher than the state-of-the-art methods using multiple commodity
cameras. The proposed system exploits characteristics of the camera
traditionally considered as flaws, namely rolling shutter and radial
distortion. The experimental evaluation shows the effectiveness of the method
for various degrees of motion.
Furthermore, edge-aware optimization is an indispensable tool in the computer
vision arsenal for accurate filtering of depth-data and image-based rendering,
which is increasingly being used for content creation and geometry processing
for AR/VR. As applications increasingly demand higher resolution and speed,
there exists a need to develop methods that scale accordingly. This
dissertation proposes such an edge-aware optimization framework which is
efficient, accurate, and algorithmically scales well, all of which are much
desirable traits not found jointly in the state of the art. The experiments
show the effectiveness of the framework in a multitude of computer vision tasks
such as computational photography and stereo.Comment: PhD thesi
Measuring the Angular Velocity of a Propeller with Video Camera Using Electronic Rolling Shutter
Noncontact measurement for rotational motion has advantages over the traditional method which measures rotational motion by means of installing some devices on the object, such as a rotary encoder. Cameras can be employed as remote monitoring or inspecting sensors to measure the angular velocity of a propeller because of their commonplace availability, simplicity, and potentially low cost. A defect of the measurement with cameras is to process the massive data generated by cameras. In order to reduce the collected data from the camera, a camera using ERS (electronic rolling shutter) is applied to measure angular velocities which are higher than the speed of the camera. The effect of rolling shutter can induce geometric distortion in the image, when the propeller rotates during capturing an image. In order to reveal the relationship between the angular velocity and the image distortion, a rotation model has been established. The proposed method was applied to measure the angular velocities of the two-blade propeller and the multiblade propeller. The experimental results showed that this method could detect the angular velocities which were higher than the camera speed, and the accuracy was acceptable
Direct Sparse Odometry with Rolling Shutter
Neglecting the effects of rolling-shutter cameras for visual odometry (VO)
severely degrades accuracy and robustness. In this paper, we propose a novel
direct monocular VO method that incorporates a rolling-shutter model. Our
approach extends direct sparse odometry which performs direct bundle adjustment
of a set of recent keyframe poses and the depths of a sparse set of image
points. We estimate the velocity at each keyframe and impose a
constant-velocity prior for the optimization. In this way, we obtain a near
real-time, accurate direct VO method. Our approach achieves improved results on
challenging rolling-shutter sequences over state-of-the-art global-shutter VO
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
Revisiting Rolling Shutter Bundle Adjustment: Toward Accurate and Fast Solution
We propose a robust and fast bundle adjustment solution that estimates the
6-DoF pose of the camera and the geometry of the environment based on
measurements from a rolling shutter (RS) camera. This tackles the challenges in
the existing works, namely relying on additional sensors, high frame rate video
as input, restrictive assumptions on camera motion, readout direction, and poor
efficiency. To this end, we first investigate the influence of normalization to
the image point on RSBA performance and show its better approximation in
modelling the real 6-DoF camera motion. Then we present a novel analytical
model for the visual residual covariance, which can be used to standardize the
reprojection error during the optimization, consequently improving the overall
accuracy. More importantly, the combination of normalization and covariance
standardization weighting in RSBA (NW-RSBA) can avoid common planar degeneracy
without needing to constrain the filming manner. Besides, we propose an
acceleration strategy for NW-RSBA based on the sparsity of its Jacobian matrix
and Schur complement. The extensive synthetic and real data experiments verify
the effectiveness and efficiency of the proposed solution over the
state-of-the-art works. We also demonstrate the proposed method can be easily
implemented and plug-in famous GSSfM and GSSLAM systems as completed RSSfM and
RSSLAM solutions
- …