Cross-calibration of Time-of-flight and Colour Cameras

Time-of-flight cameras provide depth information, which is complementary to the photometric appearance of the scene in ordinary images. It is desirable to merge the depth and colour information, in order to obtain a coherent scene representation. However, the individual cameras will have different viewpoints, resolutions and fields of view, which means that they must be mutually calibrated. This paper presents a geometric framework for this multi-view and multi-modal calibration problem. It is shown that three-dimensional projective transformations can be used to align depth and parallax-based representations of the scene, with or without Euclidean reconstruction. A new evaluation procedure is also developed; this allows the reprojection error to be decomposed into calibration and sensor-dependent components. The complete approach is demonstrated on a network of three time-of-flight and six colour cameras. The applications of such a system, to a range of automatic scene-interpretation problems, are discussed.Comment: 18 pages, 12 figures, 3 table

Camera re-calibration after zooming based on sets of conics

We describe a method to compute the internal parameters (focal and principal points) of a camera with known position and orientation, based on the observation of two or more conics on a known plane. The conics can even be degenerate (e.g. pairs of lines). The proposed method can be used to re-estimate the internal parameters of a fully calibrated camera after zooming to a new, unknown, focal length. It also allows estimating the internal parameters when a second, fully calibrated camera observes the same conics. The parameters estimated through the proposed method are coherent with the output of more traditional procedures that require a higher number of calibration images. A deep analysis of the geometrical configurations that influence the proposed method is also reported

Image-Based View Synthesis

We present a new method for rendering novel images of flexible 3D objects from a small number of example images in correspondence. The strength of the method is the ability to synthesize images whose viewing position is significantly far away from the viewing cone of the example images ("view extrapolation"), yet without ever modeling the 3D structure of the scene. The method relies on synthesizing a chain of "trilinear tensors" that governs the warping function from the example images to the novel image, together with a multi-dimensional interpolation function that synthesizes the non-rigid motions of the viewed object from the virtual camera position. We show that two closely spaced example images alone are sufficient in practice to synthesize a significant viewing cone, thus demonstrating the ability of representing an object by a relatively small number of model images --- for the purpose of cheap and fast viewers that can run on standard hardware

Advanced Strategies for Robot Manipulators

Amongst the robotic systems, robot manipulators have proven themselves to be of increasing importance and are widely adopted to substitute for human in repetitive and/or hazardous tasks. Modern manipulators are designed complicatedly and need to do more precise, crucial and critical tasks. So, the simple traditional control methods cannot be efficient, and advanced control strategies with considering special constraints are needed to establish. In spite of the fact that groundbreaking researches have been carried out in this realm until now, there are still many novel aspects which have to be explored

Minimal Solutions to Geometric Problems with Multiple Cameras or Multiple Sensor Modalities

Tese de doutoramento em Engenharia Electrotécnica e de Computadores, no ramo de Especialização em Automação e Robótica, apresentada ao Departamento de Engenharia Electrotécnica e de Computadores da Universidade de CoimbraThis thesis addresses minimal problems that involve multiple cameras or a combination of cameras with other sensors, particularly focusing on four cases: extrinsic calibration between a camera and a laser rangefinder (LRF); full calibration of an ultrasound array (US) with a camera; full calibration of a camera within a calibrated network; relative pose between axial systems. The first problem (LRF-Camera) is highly important in the context of mobile robotics in order to fuse the information of an LRF and a Camera in localization maps. The second problem (US-Camera) is becoming increasingly relevant in the context of medical imaging to perform guided intervention and 3D reconstruction with US probes. Both these problems use a planar calibration target to obtain a minimal solution from 3 and 4 correspondences respectively. They are formulated as the registration between planes detected by the camera and lines detected by either the LRF or the US. The third problem (Camera-Network) is concerned with two application scenarios: addition of a new camera to a calibrated network, and tracking of a hand-held camera within the field of view of a calibrated network. The last problem (Axial System) has its main application in motion estimation of stereo camera pairs. Both these problems introduce a 5-dimensional linear subspace to model line incidence relations of an axial system, of which a pair of calibrated cameras is a particular example. In the Camera-Network problem a generalized fundamental matrix is derived to obtain a 11-correspondence minimal solution. In the Axial System problem a generalized essential matrix is derived to obtain a 10-correspondence non-minimal solution. Although it should be possible to solve this last problem with as few as 6 correspondences, the proposed solution is the closest to minimal in the literature. Additionally this thesis addresses the use of the RANSAC framework in the context of the problems mentioned above. While RANSAC is the most widely used method in computer vision for robust estimation when minimal solutions are available, it cannot be applied directly to some of the problems discussed here. A new framework -- multi-RANSAC -- is presented as an adaptation of RANSAC to problems with multiple sampling datasets. Problems with multiple cameras or multiple sensors often fall in this category and thus this new framework can greatly improve their results. Its applicability is demonstrated in both the US-Camera and the Camera-Network problems.Esta tese aborda os problemas mínimos no contexto de visão por computador, isto é, problemas com o mesmo número de restrições e de parâmetros desconhecidos, para os quais existe um conjunto finito e discreto de soluções. A tese foca-se em particular nos seguintes problemas: calibração extrínseca entre uma câmara e um sensor laser rangefinder (LRF); calibração completa de uma sonda ultrasom (US) com uma câmara; calibração completa de uma câmara dentro de uma rede calibrada; estimação de pose relativa entre sistema axiais. O primeiro problema (LRF-Camera) é extremamente importante no contexto de robótica móvel para fundir a informação de um sensor LRF e uma câmara em mapas de localização. O segundo problema (US-Camera) está-se a tornar cada vez mais relevante no contexto de imagiologia médica para realizar intervenções guiadas e reconstrução 3D com sondas ecográficas. Ambos os problemas usam um alvo de calibração planar para obter uma solução mínima usando 3 e 4 correspondências respectivamente, e são formulados como o registo 3D entre planos detectados pela câmara e linhas detectadas pelo LRF ou US. O terceiro problema (Camera-Network) tem duas aplicações em mente: a introdução de uma nova câmara numa rede calibrada, e o seguimento de uma câmara guiada manualmente dentro do campo de visão de uma rede calibrada. O último problema (Axial System) tem a sua maior aplicação na estimação de pose relativa entre pares de câmaras estéreo. Em ambos os problemas é introduzido um subespaço linear em 5 dimensões que modela as relações de incidência de linhas num sistema axial, do qual as câmaras estéreo são um caso particular. No problema Camera- Network é introduzida uma generalização da matriz fundamental que permite obter uma solução mínima com 11 correspondências. No problema Axial System é introduzida uma generalização da matrix essencial que permite obter uma solução não mínima com 10 correspondências. Apesar de ser possível, em teoria, resolver este último problema com apenas 6 correspondências, a solução apresentada nesta tese usa um menor número de correspondências que as alternativas existentes. Adicionalmente esta tese aborda o uso de RANSAC no contexto dos problemas anteriormente descritos. O RANSAC é o estimador robusto mais utilizado em visão por computador quando existem soluções mínimas para um determinado problema, no entanto não pode ser aplicado directamente em algumas das aplicações aqui descritas. Um novo método é proposto – multiset-RANSAC – que adapta o RANSAC para situações que envolvem a amostragem de múltiplos conjuntos de dados. Os problemas com múltiplas câmaras ou múltiplos sensores encontram-se mutas vezes nesta categoria, tornando o multiset-RANSAC numa ferramenta que pode melhorar bastante os resultados em alguns dos problemas focados nesta tese. A utilidade deste método é demonstrada nos problemas US-Camera e Camera-Network