4,696 research outputs found
Virtual camera synthesis for soccer game replays
International audienceIn this paper, we present a set of tools developed during the creation of a platform that allows the automatic generation of virtual views in a live soccer game production. Observing the scene through a multi-camera system, a 3D approximation of the players is computed and used for the synthesis of virtual views. The system is suitable both for static scenes, to create bullet time effects, and for video applications, where the virtual camera moves as the game plays
Selecting surface features for accurate multi-camera surface reconstruction
This paper proposes a novel feature detector for selecting local textures that are suitable for accurate multi-camera surface reconstruction, and in particular planar patch fitting techniques. This approach is in contrast to conventional feature detectors, which focus on repeatability under scale and affine transformations rather than suitability for multi-camera reconstruction techniques. The proposed detector selects local textures that are sensitive to affine transformations, which is a fundamental requirement for accurate patch fitting. The proposed detector is evaluated against the SIFT detector on a synthetic dataset and the fitted patches are compared against ground truth. The experiments show that patches originating from the proposed detector are fitted more accurately to the visible surfaces than those originating from SIFT keypoints. In addition, the detector is evaluated on a performance capture studio dataset to show the real-world application of the proposed detector
Selecting surface features for accurate multi-camera surface reconstruction
This paper proposes a novel feature detector for selecting local textures that are suitable for accurate multi-camera surface reconstruction, and in particular planar patch fitting techniques. This approach is in contrast to conventional feature detectors, which focus on repeatability under scale and affine transformations rather than suitability for multi-camera reconstruction techniques. The proposed detector selects local textures that are sensitive to affine transformations, which is a fundamental requirement for accurate patch fitting. The proposed detector is evaluated against the SIFT detector on a synthetic dataset and the fitted patches are compared against ground truth. The experiments show that patches originating from the proposed detector are fitted more accurately to the visible surfaces than those originating from SIFT keypoints. In addition, the detector is evaluated on a performance capture studio dataset to show the real-world application of the proposed detector
Mirrored Light Field Video Camera Adapter
This paper proposes the design of a custom mirror-based light field camera
adapter that is cheap, simple in construction, and accessible. Mirrors of
different shape and orientation reflect the scene into an upwards-facing camera
to create an array of virtual cameras with overlapping field of view at
specified depths, and deliver video frame rate light fields. We describe the
design, construction, decoding and calibration processes of our mirror-based
light field camera adapter in preparation for an open-source release to benefit
the robotic vision community.Comment: tech report, v0.5, 15 pages, 6 figure
Comparative Study of Model-Based and Learning-Based Disparity Map Fusion Methods
Creating an accurate depth map has several, valuable applications including augmented/virtual reality, autonomous navigation, indoor/outdoor mapping, object segmentation, and aerial topography. Current hardware solutions for precise 3D scanning are relatively expensive. To combat hardware costs, software alternatives based on stereoscopic images have previously been proposed. However, software solutions are less accurate than hardware solutions, such as laser scanning, and are subject to a variety of irregularities. Notably, disparity maps generated from stereo images typically fall short in cases of occlusion, near object boundaries, and on repetitive texture regions or texture-less regions. Several post-processing methods are examined in an effort to combine strong algorithm results and alleviate erroneous disparity regions. These methods include basic statistical combinations, histogram-based voting, edge detection guidance, support vector machines (SVMs), and bagged trees. Individual errors and average errors are compared between the newly introduced fusion methods and the existing disparity algorithms. Several acceptable solutions are identified to bridge the gap between 3D scanning and stereo imaging. It is shown that fusing disparity maps can result in lower error rates than individual algorithms across the dataset while maintaining a high level of robustness
Stereo Reconstruction using Induced Symmetry and 3D scene priors
Tese de doutoramento em Engenharia Electrotécnica e de
Computadores apresentada Ă Faculdade de CiĂȘncias e Tecnologia da Universidade de CoimbraRecuperar a geometria 3D a partir de dois vistas, conhecida como reconstrução
estéreo, é um dos tópicos mais antigos e mais investigado em visão por computador.
A computação de modelos 3D do ambiente Ă© Ăștil para uma grande nĂșmero de
aplicaçÔes, desde a robĂłticaâ, passando pela sua utilização do consumidor comum,
atĂ© a procedimentos mĂ©dicos. O princĂpio para recuperar a estrutura 3D cena Ă©
bastante simples, no entanto, existem algumas situaçÔes que complicam consideravelmente o processo de reconstrução. Objetos que contĂȘm estruturas pouco
texturadas ou repetitivas, e superfĂcies com bastante inclinação ainda colocam em
dificuldade os algoritmos state-of-the-art.
Esta tese de doutoramento aborda estas questÔes e apresenta um novo framework
estéreo que é completamente diferente das abordagens convencionais. Propomos
a utilização de simetria em vez de foto-similaridade para avaliar a verosimilhança
de pontos em duas imagens distintas serem uma correspondĂȘncia. O framework
Ă© chamado SymStereo, e baseia-se no efeito de espelhagem que surge sempre
que uma imagem Ă© mapeada para a outra cĂąmera usando a homografia induzida
por um plano de corte virtual que intersecta a baseline. ExperiĂȘncias em estĂ©reo
denso comprovam que as nossas funçÔes de custo baseadas em simetria se comparam
favoravelmente com os custos baseados em foto-consistĂȘncia de melhor
desempenho. Param alĂ©m disso, investigamos a possibilidade de realizar Stereo-Rangefinding, que consiste em usar estĂ©reo passivo para recuperar exclusivamente a profundidade ao longo de um plano de varrimento. ExperiĂȘncias abrangentes fornecem evidĂȘncia de que estĂ©reo baseada em simetria induzida Ă© especialmente eficaz para esta finalidade.
Como segunda linha de investigação, propomos superar os problemas descritos
anteriormente usando informação a priori sobre o ambiente 3D, com o objectivo de
aumentar a robustez do processo de reconstrução. Para tal, apresentamos uma nova abordagem global para detectar pontos de desvanecimento e grupos de direcçÔes de desvanecimento mutuamente ortogonais em ambientes Manhattan. ExperiĂȘncias quer em imagens sintĂ©ticas quer em imagens reais demonstram que os nossos algoritmos superaram os mĂ©todos state-of-the-art, mantendo a computação aceitĂĄvel. AlĂ©m disso, mostramos pela primeira vez resultados na detecção simultĂąnea de mĂșltiplas configuraçÔes de Manhattan. Esta informação a priori sobre a estrutura da cena Ă© depois usada numa pipeline de reconstrução que gera modelos piecewise planares de ambientes urbanos a partir de duas vistas calibradas. A nossa
formulação combina SymStereo e o algoritmo de clustering PEARL [3], e alterna
entre um passo de otimização discreto, que funde hipĂłteses de superfĂcies planares
e descarta detecçÔes com pouco suporte, e uma etapa de otimização contĂnua, que
refina as poses dos planos. ExperiĂȘncias com pares estĂ©reo de ambientes interiores
e exteriores confirmam melhorias significativas sobre métodos state-of-the-art
relativamente a precisĂŁo e robustez.
Finalmente, e como terceira contribuição para melhorar a visão estéreo na
presença de superfĂcies inclinadas, estendemos o recente framework de agregação estĂ©reo baseada em histogramas [4]. O algoritmo original utiliza janelas de suporte fronto-paralelas para a agregação de custo, o que leva a resultados imprecisos
na presença de superfĂcies com inclinação significativa. NĂłs abordamos o
problema considerando hipóteses de orientação discretas. Os resultados experimentais obtidos comprovam a eficåcia do método, permitindo melhorar a precisção
de correspondĂȘncia, preservando simultaneamente uma baixa complexidade computacional.Recovering the 3D geometry from two or more views, known as stereo reconstruction,
is one of the earliest and most investigated topics in computer vision. The
computation of 3D models of an environment is useful for a very large number of applications, ranging from robotics, consumer utilization to medical procedures. The principle to recover the 3D scene structure is quite simple, however, there are some issues that considerable complicate the reconstruction process. Objects containing complicated structures, including low and repetitive textures, and highly slanted surfaces still pose difficulties to state-of-the-art algorithms.
This PhD thesis tackles this issues and introduces a new stereo framework that
is completely different from conventional approaches. We propose to use symmetry
instead of photo-similarity for assessing the likelihood of two image locations
being a match. The framework is called SymStereo, and is based on the mirroring
effect that arises whenever one view is mapped into the other using the homography induced by a virtual cut plane that intersects the baseline. Extensive experiments in dense stereo show that our symmetry-based cost functions compare favorably against the best performing photo-similarity matching costs. In addition, we investigate the possibility of accomplishing Stereo-Rangefinding that consists in using
passive stereo to exclusively recover depth along a scan plane. Thorough experiments
provide evidence that Stereo from Induced Symmetry is specially well suited
for this purpose.
As a second research line, we propose to overcome the previous issues using
priors about the 3D scene for increasing the robustness of the reconstruction process.
For this purpose, we present a new global approach for detecting vanishing
points and groups of mutually orthogonal vanishing directions in man-made environments. Experiments in both synthetic and real images show that our algorithms
outperform the state-of-the-art methods while keeping computation tractable. In
addition, we show for the first time results in simultaneously detecting multiple
Manhattan-world configurations. This prior information about the scene structure
is then included in a reconstruction pipeline that generates piece-wise planar
models of man-made environments from two calibrated views. Our formulation
combines SymStereo and PEARL clustering [3], and alternates between a discrete
optimization step, that merges planar surface hypotheses and discards detections
with poor support, and a continuous optimization step, that refines the plane poses.
Experiments with both indoor and outdoor stereo pairs show significant improvements
over state-of-the-art methods with respect to accuracy and robustness.
Finally, and as a third contribution to improve stereo matching in the presence
of surface slant, we extend the recent framework of Histogram Aggregation
[4]. The original algorithm uses a fronto-parallel support window for cost aggregation, leading to inaccurate results in the presence of significant surface slant. We
address the problem by considering discrete orientation hypotheses. The experimental
results prove the effectiveness of the approach, which enables to improve
the matching accuracy while preserving a low computational complexity
- âŠ