Dynamic Programming and Skyline Extraction in Catadioptric Infrared Images

International audienceUnmanned Aerial Vehicles (UAV) are the subject of an increasing interest in many applications and a key requirement for autonomous navigation is the attitude/position stabilization of the vehicle. Some previous works have suggested using catadioptric vision, instead of traditional perspective cameras, in order to gather much more information from the environment and therefore improve the robustness of the UAV attitude/position estimation. This paper belongs to a series of recent publications of our research group concerning catadioptric vision for UAVs. Currently, we focus on the extraction of skyline in catadioptric images since it provides important information about the attitude/position of the UAV. For example, the DEM-based methods can match the extracted skyline with a Digital Elevation Map (DEM) by process of registration, which permits to estimate the attitude and the position of the camera. Like any standard cameras, catadioptric systems cannot work in low luminosity situations because they are based on visible light. To overcome this important limitation, in this paper, we propose using a catadioptric infrared camera and extending one of our methods of skyline detection towards catadioptric infrared images. The task of extracting the best skyline in images is usually converted in an energy minimization problem that can be solved by dynamic programming. The major contribution of this paper is the extension of dynamic programming for catadioptric images using an adapted neighborhood and an appropriate scanning direction. Finally, we present some experimental results to demonstrate the validity of our approach

A Parametric Algorithm for Skyline Extraction

International audienceThis paper is dedicated to the problem of automatic skyline extraction in digital images. The study is motivated by the needs, expressed by urbanists, to describe in terms of geometrical features, the global shape created by man-made buildings in urban areas. Skyline extraction has been widely studied for navigation of Unmanned Aerial Vehicles (drones) or for geolocalization, both in natural and urban contexts. In most of these studies, the skyline is defined by the limit between sky and ground objects, and can thus be resumed to the sky segmentation problem in images. In our context, we need a more generic definition of skyline, which makes its extraction more complex and even variable. The skyline can be extracted for different depths, depending on the interest of the user (far horizon, intermediate buildings, near constructions , ...), and thus requires a human interaction. The main steps of our method are as follows: we use a Canny filter to extract edges and allow the user to interact with filter's parameters. With a high sensitivity , all the edges will be detected, whereas with lower values, only most contrasted contours will be kept by the filter. From the obtained edge map, an upper envelope is extracted, which is a disconnected approximation of the skyline. A graph is then constructed and a shortest path algorithm is used to link discontinuities. Our approach has been tested on several public domain urban and natural databases, and have proven to give better results that previously published methods

Airborne vision-based attitude estimation and localisation

Vision plays an integral part in a pilot's ability to navigate and control an aircraft. Therefore Visual Flight Rules have been developed around the pilot's ability to see the environment outside of the cockpit in order to control the attitude of the aircraft, to navigate and to avoid obstacles. The automation of these processes using a vision system could greatly increase the reliability and autonomy of unmanned aircraft and flight automation systems. This thesis investigates the development and implementation of a robust vision system which fuses inertial information with visual information in a probabilistic framework with the aim of aircraft navigation. The horizon appearance is a strong visual indicator of the attitude of the aircraft. This leads to the first research area of this thesis, visual horizon attitude determination. An image processing method was developed to provide high performance horizon detection and extraction from camera imagery. A number of horizon models were developed to link the detected horizon to the attitude of the aircraft with varying degrees of accuracy. The second area investigated in this thesis was visual localisation of the aircraft. A terrain-aided horizon model was developed to estimate the position, altitude as well as attitude of the aircraft. This gives rough positions estimates with highly accurate attitude information. The visual localisation accuracy was improved by incorporating ground feature-based map-aided navigation. Road intersections were detected using a developed image processing algorithm and then they were matched to a database to provide positional information. The developed vision system show comparable performance to other non-vision-based systems while removing the dependence on external systems for navigation. The vision system and techniques developed in this thesis helps to increase the autonomy of unmanned aircraft and flight automation systems for manned flight

Holistic methods for visual navigation of mobile robots in outdoor environments

Differt D. Holistic methods for visual navigation of mobile robots in outdoor environments. Bielefeld: Universität Bielefeld; 2017

Robust density modelling using the student's t-distribution for human action recognition

The extraction of human features from videos is often inaccurate and prone to outliers. Such outliers can severely affect density modelling when the Gaussian distribution is used as the model since it is highly sensitive to outliers. The Gaussian distribution is also often used as base component of graphical models for recognising human actions in the videos (hidden Markov model and others) and the presence of outliers can significantly affect the recognition accuracy. In contrast, the Student's t-distribution is more robust to outliers and can be exploited to improve the recognition rate in the presence of abnormal data. In this paper, we present an HMM which uses mixtures of t-distributions as observation probabilities and show how experiments over two well-known datasets (Weizmann, MuHAVi) reported a remarkable improvement in classification accuracy. © 2011 IEEE

UAV vision system: Application in electric line following and 3D reconstruction of associated terrain

Abstract. In this work, a set of vision techniques applied to a UAV (Unmanned Aerial Vehicle) images is presented. The techniques are used to detect electrical lines and towers which are used in vision based navigation and for 3D associated terrain reconstruction. The developed work aims to be a previous stage for autonomous electrical infrastructure inspection. This work is divided in four stages: power line detection, transmission tower detection, UAV navigation and 3D reconstruction of associated terrain. In the first stage, a study of algorithms for line detection was performed. After that, an algorithm for line detection called CBS (Circle Based Search) which presented good results with azimuthal images was developed. This method offers a shorter response time in comparison with the Hough transform and the LSD (Line Segment Detector) algorithm, and a similar response to EDLines which is one of the fastest and most trustful algorithms for line detection. Given that most of the works related with line detection are focused in straight lines, an algorithm for catenary detection based on a concatenation process was developed. This algorithm was validated using real power line inspection images with catenaries. Additionally, in this work a tower detection method based on a feature descriptor with the capacity of detecting towers in times close to 100 ms was developed. Navigation over power lines by using UAVs requires a lot of tests because of the risk of failures and accidents. For this reason, a virtual environment for real time UAV simulation of visual navigation was developed by using ROS (Robot Operative System), which is open source. An onboard visual navigation system for UAV was also developed. This system allows the UAV to navigate following a power line in real sceneries by using the developed techniques. In the last part a 3D tower reconstruction that uses images obtained with UAVs is presented.}, keywordenglish = {line detection, inspection, navigation, tower detection, onboard vision system, UAV.Este trabajo presenta un conjunto de técnicas de visión aplicadas a imágenes adquiridas mediante UAVs (vehículos aéreos no tripulados). Las técnicas se usan para la detección de líneas y torres eléctricas las cuales son usadas en un proceso de navegación basada en vision y para la reconstrucción de terreno asociado en 3D. El proyecto está planteado como una fase previa a un proceso de inspección de infraestructura electrica. El trabajo se encuentra dividido en cuatro partes: la detección de líneas de transmisión eléctrica, la detección de torres de transmisión, la navegación de UAVs y la reconstrucción tridimensional de objetos tales como torres de transmisión. En primer lugar se realizó un estudio de los algoritmos para la detección de líneas en imágenes. Posteriormente se desarrolló un algoritmo para la detección de líneas llamado CBS (Búsqueda basada en círculos), el cual tiene buenos resultados en imágenes azimutales de líneas eléctricas. Este método ofrece un tiempo de respuesta más corto que la transformada de Houg o el algoritmo LSD (line segment detector), y un tiempo similar a EDLines el cual es uno de los algoritmos más rápidos y confiables para detectar líneas. Debido a que la mayoría de trabajos relacionados con detección de líneas se enfocan en líneas rectas, se desarrolló un algoritmo para detectar catenarias que cuenta con un proceso de concatenación de segmentos, esta técnica fue validada con imágenes de catenarias obtenidas en inspecciones reales de infraestructura eléctrica. Adicionalmente se desarrolló un algoritmo basado en descriptores de características para la detección de torres de transmisión con la intención de facilitar los procesos de navegación e inspección. El proceso desarrollado ha permitido detectar torres en videos en tiempos cercanos a 100 ms. La navegación sobre líneas eléctricas mediante UAVs requiere una gran cantidad de pruebas debido al riesgo de fallos y accidentes, por esto se realizó un ambiente virtual para la simulación en tiempo real de técnicas de navegación basadas en características visuales haciendo uso del entorno de ROS (Robot Operative System), el cual es de código abierto. Se desarrollo un sistema de navegación a bordo de un UAV el cual permitio obtener resultados de navegación autónoma en el seguimiento de líneas en escenarios reales usando las técnicas desarrolladas. En la parte final del trabajo se realizó una reconstrucción 3D de torres electricas haciendo uso de imagenes adquiridas con UAVs.Doctorad

Virtual Heritage: new technologies for edutainment

Cultural heritage represents an enormous amount of information and knowledge. Accessing this treasure chest allows not only to discover the legacy of physical and intangible attributes of the past but also to provide a better understanding of the present. Museums and cultural institutions have to face the problem of providing access to and communicating these cultural contents to a wide and assorted audience, meeting the expectations and interests of the reference end-users and relying on the most appropriate tools available. Given the large amount of existing tangible and intangible heritage, artistic, historical and cultural contents, what can be done to preserve and properly disseminate their heritage significance? How can these items be disseminated in the proper way to the public, taking into account their enormous heterogeneity? Answering this question requires to deal as well with another aspect of the problem: the evolution of culture, literacy and society during the last decades of 20th century. To reflect such transformations, this period witnessed a shift in the museum’s focus from the aesthetic value of museum artifacts to the historical and artistic information they encompass, and a change into the museums’ role from a mere "container" of cultural objects to a "narrative space" able to explain, describe, and revive the historical material in order to attract and entertain visitors. These developments require creating novel exhibits, able to tell stories about the objects and enabling visitors to construct semantic meanings around them. The objective that museums presently pursue is reflected by the concept of Edutainment, Education + Entertainment. Nowadays, visitors are not satisfied with ‘learning something’, but would rather engage in an ‘experience of learning’, or ‘learning for fun’, being active actors and players in their own cultural experience. As a result, institutions are faced with several new problems, like the need to communicate with people from different age groups and different cultural backgrounds, the change in people attitude due to the massive and unexpected diffusion of technology into everyday life, the need to design the visit by a personal point of view, leading to a high level of customization that allows visitors to shape their path according to their characteristics and interests. In order to cope with these issues, I investigated several approaches. In particular, I focused on Virtual Learning Environments (VLE): real-time interactive virtual environments where visitors can experience a journey through time and space, being immersed into the original historical, cultural and artistic context of the work of arts on display. VLE can strongly help archivists and exhibit designers, allowing to create new interesting and captivating ways to present cultural materials. In this dissertation I will tackle many of the different dimensions related to the creation of a cultural virtual experience. During my research project, the entire pipeline involved into the development and deployment of VLE has been investigated. The approach followed was to analyze in details the main sub-problems to face, in order to better focus on specific issues. Therefore, I first analyzed different approaches to an effective recreation of the historical and cultural context of heritage contents, which is ultimately aimed at an effective transfer of knowledge to the end-users. In particular, I identified the enhancement of the users’ sense of presence in VLE as one of the main tools to reach this objective. Presence is generally expressed as the perception of 'being there', i.e. the subjective belief of users that they are in a certain place, even if they know that the experience is mediated by the computer. Presence is related to the number of senses involved by the VLE and to the quality of the sensorial stimuli. But in a cultural scenario, this is not sufficient as the cultural presence plays a relevant role. Cultural presence is not just a feeling of 'being there' but of being - not only physically, but also socially, culturally - 'there and then'. In other words, the VLE must be able to transfer not only the appearance, but also all the significance and characteristics of the context that makes it a place and both the environment and the context become tools capable of transferring the cultural significance of a historic place. The attention that users pay to the mediated environment is another aspect that contributes to presence. Attention is related to users’ focalization and concentration and to their interests. Thus, in order to improve the involvement and capture the attention of users, I investigated in my work the adoption of narratives and storytelling experiences, which can help people making sense of history and culture, and of gamification approaches, which explore the use of game thinking and game mechanics in cultural contexts, thus engaging users while disseminating cultural contents and, why not?, letting them have fun during this process. Another dimension related to the effectiveness of any VLE is also the quality of the user experience (UX). User interaction, with both the virtual environment and its digital contents, is one of the main elements affecting UX. With respect to this I focused on one of the most recent and promising approaches: the natural interaction, which is based on the idea that persons need to interact with technology in the same way they are used to interact with the real world in everyday life. Then, I focused on the problem of presenting, displaying and communicating contents. VLE represent an ideal presentation layer, being multiplatform hypermedia applications where users are free to interact with the virtual reconstructions by choosing their own visiting path. Cultural items, embedded into the environment, can be accessed by users according to their own curiosity and interests, with the support of narrative structures, which can guide them through the exploration of the virtual spaces, and conceptual maps, which help building meaningful connections between cultural items. Thus, VLE environments can even be seen as visual interfaces to DBs of cultural contents. Users can navigate the VE as if they were browsing the DB contents, exploiting both text-based queries and visual-based queries, provided by the re-contextualization of the objects into their original spaces, whose virtual exploration can provide new insights on specific elements and improve the awareness of relationships between objects in the database. Finally, I have explored the mobile dimension, which became absolutely relevant in the last period. Nowadays, off-the-shelf consumer devices as smartphones and tablets guarantees amazing computing capabilities, support for rich multimedia contents, geo-localization and high network bandwidth. Thus, mobile devices can support users in mobility and detect the user context, thus allowing to develop a plethora of location-based services, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits and cultural or tourist sites according to visitors’ personal interest and curiosity