Desarrollo y evaluación de estrategias para aplicaciones de percepción 3D usando cámaras de rango

La evolución de sensores de profundidad de bajo coste en los últimos años ha permitido un uso más generalizado de las cámaras de rango. Su aplicación, gracias a que proporcionan información de profundidad en tiempo real, se ha ampliado a diversos ámbitos de percepción de objetos donde es necesario el conocimiento del entorno en tres dimensiones (3D) para poder interaccionar con él. Por este motivo, existe una demanda creciente de sistemas de percepción visual capaces de explotar la información procedente de los dispositivos de rango. Sin embargo, la mayoría de los enfoques existentes presentan limitaciones en entornos poco estructurados, comunes en el mundo real. El objetivo de esta tesis es contribuir al progreso de sistemas eficaces de percepción 3D de objetos en este tipo de entornos, abordando diversos retos aún sin resolver como la falta de información a priori o ruido en las imágenes. Para ello se ha diseñado una metodología que permite desarrollar estrategias para entornos complejos con el objetivo de dar solución a los desafíos de percepción planteados, explotando al máximo las características de la escena. Para validar la estrategia planteada se ha seguido una metodología guiada por escenarios, por lo que para su evaluación se han seleccionado tres entornos con diferente complejidad, y lo suficientemente significativos a la vez que complementarios entre sí. Cada contexto se enmarca dentro de un grupo de escenarios semiestructurados de percepción visual, clasificados según condiciones ambientales así como grado de información conocida a priori. Para cada escenario se propone una estrategia, la cual se desarrolla y se evalúa exhaustivamente en un contexto experimental, seleccionando el hardware así como el software a implementar más adecuado a los requisitos y necesidades del entorno. El enfoque descrito en esta tesis permite reducir la incertidumbre centrándose en un determinado contexto para plantear la estrategia a seguir, reduciendo el espacio de estados sin afectar a la tarea a desempeñar.The evolution of low cost depth sensors in recent years has allowed a more widespread use of range cameras. They provide depth information in real time so their application has been extended to various areas of object perception where three-dimensional (3D) knowledge of the environment is mandatory to interact with it. For this reason, there is a growing demand for visual perception systems to exploit the information from range devices. However, most existing approaches are limited in unstructured environments, which are common in the real world. The objective of this thesis is to contribute to the progress of 3D object perception in these environments, addressing open challenges like lack of prior information or picture noise. A methodology to develop strategies for complex environment has been designed to solve the challenges, exploiting the characteristics of the scene. The strategy is validated following a methodology based on scenarios, so three environments have been selected with different complexity. These are sufficiently significant and complementary. Each context is part of a group of perception unstructured scenarios, classified by environmental conditions and the degree of information known a priori. For each scenario, a strategy is proposed, which is developed and evaluated thoroughly in an experimental context. This has also taken into account that hardware and software should be appropriated to the requirements and environmental needs. The approach described in this thesis decreases uncertainty by focusing on a particular context. Furthermore the state space is reduced without affecting the task to be performed.Esta tesis ha sido parcialmente financiada por el proyecto europeo HANDLE, dentro del Séptimo Programa Marco de la Comunidad Europea en virtud del acuerdo de subvención ICT 231640. También ha sido parcialmente financiada por una beca de Formación de Personal Investigador de la Universidad Carlos III de Madrid (PIF-UC3M ref. 03-1213).European Community's Seventh Framework ProgramPrograma Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: José María Sebastián y Zúñiga.- Secretario: Juan Antonio Escalera Piña.- Vocal: Óscar Reinoso Garcí

Visual object perception in unstructured environments

As robotic systems move from well-controlled settings to increasingly unstructured environments, they are required to operate in highly dynamic and cluttered scenarios. Finding an object, estimating its pose, and tracking its pose over time within such scenarios are challenging problems. Although various approaches have been developed to tackle these problems, the scope of objects addressed and the robustness of solutions remain limited. In this thesis, we target a robust object perception using visual sensory information, which spans from the traditional monocular camera to the more recently emerged RGB-D sensor, in unstructured environments. Toward this goal, we address four critical challenges to robust 6-DOF object pose estimation and tracking that current state-of-the-art approaches have, as yet, failed to solve. The first challenge is how to increase the scope of objects by allowing visual perception to handle both textured and textureless objects. A large number of 3D object models are widely available in online object model databases, and these object models provide significant prior information including geometric shapes and photometric appearances. We note that using both geometric and photometric attributes available from these models enables us to handle both textured and textureless objects. This thesis presents our efforts to broaden the spectrum of objects to be handled by combining geometric and photometric features. The second challenge is how to dependably estimate and track the pose of an object despite the clutter in backgrounds. Difficulties in object perception rise with the degree of clutter. Background clutter is likely to lead to false measurements, and false measurements tend to result in inaccurate pose estimates. To tackle significant clutter in backgrounds, we present two multiple pose hypotheses frameworks: a particle filtering framework for tracking and a voting framework for pose estimation. Handling of object discontinuities during tracking, such as severe occlusions, disappearances, and blurring, presents another important challenge. In an ideal scenario, a tracked object is visible throughout the entirety of tracking. However, when an object happens to be occluded by other objects or disappears due to the motions of the object or the camera, difficulties ensue. Because the continuous tracking of an object is critical to robotic manipulation, we propose to devise a method to measure tracking quality and to re-initialize tracking as necessary. The final challenge we address is performing these tasks within real-time constraints. Our particle filtering and voting frameworks, while time-consuming, are composed of repetitive, simple and independent computations. Inspired by that observation, we propose to run massively parallelized frameworks on a GPU for those robotic perception tasks which must operate within strict time constraints.Ph.D