Vision-Based Georeferencing of GPR in Urban Areas

Ground Penetrating Radar (GPR) surveying is widely used to gather accurate knowledge about the geometry and position of underground utilities. The sensor arrays need to be coupled to an accurate positioning system, like a geodetic-grade Global Navigation Satellite System (GNSS) device. However, in urban areas this approach is not always feasible because GNSS accuracy can be substantially degraded due to the presence of buildings, trees, tunnels, etc. In this work, a photogrammetric (vision-based) method for GPR georeferencing is presented. The method can be summarized in three main steps: tie point extraction from the images acquired during the survey, computation of approximate camera extrinsic parameters and finally a refinement of the parameter estimation using a rigorous implementation of the collinearity equations. A test under operational conditions is described, where accuracy of a few centimeters has been achieved. The results demonstrate that the solution was robust enough for recovering vehicle trajectories even in critical situations, such as poorly textured framed surfaces, short baselines, and low intersection angles

A Review of Sensor Technologies for Perception in Automated Driving

After more than 20 years of research, ADAS are common in modern vehicles available in the market. Automated Driving systems, still in research phase and limited in their capabilities, are starting early commercial tests in public roads. These systems rely on the information provided by on-board sensors, which allow to describe the state of the vehicle, its environment and other actors. Selection and arrangement of sensors represent a key factor in the design of the system. This survey reviews existing, novel and upcoming sensor technologies, applied to common perception tasks for ADAS and Automated Driving. They are put in context making a historical review of the most relevant demonstrations on Automated Driving, focused on their sensing setup. Finally, the article presents a snapshot of the future challenges for sensing technologies and perception, finishing with an overview of the commercial initiatives and manufacturers alliances that will show future market trends in sensors technologies for Automated Vehicles.This work has been partly supported by ECSEL Project ENABLE- S3 (with grant agreement number 692455-2), by the Spanish Government through CICYT projects (TRA2015- 63708-R and TRA2016-78886-C3-1-R)

Multi-Object Tracking System based on LiDAR and RADAR for Intelligent Vehicles applications

El presente Trabajo Fin de Grado tiene como objetivo el desarrollo de un Sistema de Detección y Multi-Object Tracking 3D basado en la fusión sensorial de LiDAR y RADAR para aplicaciones de conducción autónoma basándose en algoritmos tradicionales de Machine Learning. La implementación realizada está basada en Python, ROS y cumple requerimientos de tiempo real. En la etapa de detección de objetos se utiliza el algoritmo de segmentación del plano RANSAC, para una posterior extracción de Bounding Boxes mediante DBSCAN. Una Late Sensor Fusion mediante Intersection over Union 3D y un sistema de tracking BEV-SORT completan la arquitectura propuesta.This Final Degree Project aims to develop a 3D Multi-Object Tracking and Detection System based on the Sensor Fusion of LiDAR and RADAR for autonomous driving applications based on traditional Machine Learning algorithms. The implementation is based on Python, ROS and complies with real-time requirements. In the Object Detection stage, the RANSAC plane segmentation algorithm is used, for a subsequent extraction of Bounding Boxes using DBSCAN. A Late Sensor Fusion using Intersection over Union 3D and a BEV-SORT tracking system complete the proposed architecture.Grado en Ingeniería en Electrónica y Automática Industria

Outdoor-Indoor tracking systems through geomatic techniques: data analysis for marketing and safety management

Negli ultimi decenni, l'utilizzo di sistemi di gestione delle informazioni nel trattamento dei dati edilizi ha portato a cambiamenti radicali nei metodi di produzione, documentazione e archiviazione dei dati. Dato il crescente interesse per i dati e la loro gestione, l'obiettivo di questa tesi è quello di creare un flusso di lavoro efficace e chiaro a partire dai rilievi geomatici in un'ottica di miglioramento dei dati raccolti sul territorio, sugli edifici circostanti e su quelli relativi al comportamento umano, in modo che possano essere meglio sfruttati e integrati in modelli di gestione intelligenti. Come primo passo, questa tesi mira a comprendere i limiti dell'interoperabilità e dell'integrazione dei dati nei GIS. Per promuovere l'interoperabilità dei dati GIS, è necessario analizzare i metodi di conversione nei diversi modelli di archiviazione dei dati, come CityGML e IndoorGML, definendo un dominio ontologico. Questo ha portato alla creazione di un nuovo modello arricchito, basato sulle connessioni tra i diversi elementi del modello urbano in GIS. Il secondo passo consiste nel raccogliere tutti i dati tradotti in un database a grafo sfruttando il web semantico. Il risultato offrirà vantaggi sostanziali durante l'intero ciclo di vita del progetto. Questa metodologia può essere applicata anche al patrimonio culturale, dove la gestione delle informazioni gioca un ruolo fondamentale. Un altro lavoro di ricerca è stato quello di sviluppare un sistema di gestione SMART per le attività di conservazione dei borghi storici attraverso la gestione di tipologie eterogenee di dati, dal rilievo alla documentazione tecnica. Il flusso di lavoro è stato strutturato come segue: (i) acquisizione dei dati; (ii) modellazione 3D; (iii) modellazione della conoscenza; (iv) modellazione della gestione SMART. Questa ricerca apre la strada allo sviluppo di una piattaforma web in cui importare i dati GIS per un approccio di digital twin. Tutte le ricerche svolte fino a questo punto sono state finalizzate a comprendere la capacità di creare modelli e sistemi informativi intelligenti per capire la fattibilità di ospitare dati eterogenei che potrebbero essere inclusi in futuro. Il passo successivo consiste nel comprendere il comportamento umano in uno spazio. Finora sono pochi i lavori di ricerca che si occupano di sistemi di mappatura e posizionamento che tengono conto sia degli spazi esterni che di quelli interni. Questo argomento, anche se ha pochi articoli di ricerca, rappresenta un aspetto cruciale per molte ragioni, soprattutto quando si tratta di gestire la sicurezza degli edifici danneggiati. Angelats e il suo gruppo di ricerca al CTTC hanno lavorato su questo aspetto, fornendo un sistema in grado di seguire in tempo reale le persone dall'esterno all'interno di spazi chiusi e viceversa. L'uso di sensori GNSS combinato con l'odometria inerziale visiva fornisce una traiettoria continua senza perdere il percorso seguito dall'utente monitorato. Una parte di questa tesi si è concentrata sul miglioramento della traiettoria finale ottenuta con il sistema appena descritto, effettuando test sulla traiettoria esterna del GNSS per capire il comportamento della traiettoria quando si avvicina agli edifici o quando l'utente si sposta in indoor. L'ultimo aspetto su cui si concentrerà la tesi è il tracciamento delle persone in ambienti chiusi. Il comportamento umano è al centro di numerosi studi in diversi campi, come quello scientifico, sociale ed economico. A differenza del precedente caso di studio sul tracciamento delle persone in aree esterne/interne, l'obiettivo è stato quello di raccogliere informazioni sul posizionamento dinamico delle persone in ambienti indoor, sulla base del segnale WiFi. Verrà effettuata una breve analisi dei dati per dimostrare il corretto funzionamento del sistema, per sottolineare l'importanza della conoscenza dei dati e l'uso che se ne può fare.In the last decades, the use of information management systems in the building data processing led to radical changes to the methods of data production, documentation and archiving. Given the ever-increasing interest in data and their management, the aim of this thesis is to create an effective and clear workflow starting from geomatic surveys in a perspective of improving the collected data on the territory, surrounding buildings and those related to human behaviour so they can be better exploited and integrated into smart management models As first step this thesis aims to understand the limits of data interoperability and integration in GIS filed. Before that, the data must be collected as raw data, then processed and interpret in order to obtain information. At the end of this first stage, when the information is well organized and can be well understanded and used it becomes knowledge. To promote the interoperability of GIS data, it is necessary at first to analyse methods of conversion in different data storage models such as CityGML and IndoorGML, defining an ontological domain. This has led to the creation of a new enriched model, based on connections among the different elements of the urban model in GIS environment, and to the possibility to formulate queries based on these relations. The second step consists in collecting all data translated into a specific format that fill a graph database in a semantic web environment, while maintaining those relationships. The outcome will offer substantial benefits during the entire project life cycle. This methodology can also be applied to cultural heritage where the information management plays a key role. Another research work, was to develop a SMART management system for preservation activities of historical villages through the management of heterogeneous types of data, from the survey to the technical documentation. The workflow was structured as follows: (i) Data acquisition; (ii) 3D modelling; (iii) Knowledge modelling; (iv) SMART management modelling. This research paves the way to develop a web platform where GIS data would be imported for a digital twin approach. All the research done up to this point was to understand the capability of creating smart information models and systems in order to understand the feasibility to host heterogeneous data that may be included in the future. The next step consist of understanding human behaviour in a space. So far only a few research papers are addressed towards mapping and positioning systems taking into account both outdoor and indoor spaces. This topic, even though it has few research articles, represents a crucial aspect for many reasons, especially when it comes to safety management of damaged building. Angelats and his research team at CTTC have been working on this aspect providing a system able to track in real time people from outdoor to indoor areas and vice-versa. The use of GNSS sensors combined with Visual Inertial Odometry provide a continuous trajectory without losing the path followed by the monitored user. A part of this thesis focused on enhancing the final trajectory obtained with the described system above, carrying out tests on the outdoor trajectory of GNSS in order to understand behaviour of the trajectory when it gets close to buildings or when the user moves indoor. The last aspect this thesis will focus on is the tracking of people indoor. Human behaviour is at the centre of several studies in different fields such as scientific subjects, social and economics. Differently from the previous case study of tracking people in outdoor/indoor areas, the scope was to collect information about the dynamic indoor positioning of people, based on the WiFi signal. A brief analysis of the data will be made to demonstrate the correct functioning of the system, to emphasise the importance of data knowledge and the use that can be made of it

Percepção do ambiente urbano e navegação usando visão robótica : concepção e implementação aplicado à veículo autônomo

Orientadores: Janito Vaqueiro Ferreira, Alessandro Corrêa VictorinoTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia MecânicaResumo: O desenvolvimento de veículos autônomos capazes de se locomover em ruas urbanas pode proporcionar importantes benefícios na redução de acidentes, no aumentando da qualidade de vida e também na redução de custos. Veículos inteligentes, por exemplo, frequentemente baseiam suas decisões em observações obtidas a partir de vários sensores tais como LIDAR, GPS e câmeras. Atualmente, sensores de câmera têm recebido grande atenção pelo motivo de que eles são de baixo custo, fáceis de utilizar e fornecem dados com rica informação. Ambientes urbanos representam um interessante mas também desafiador cenário neste contexto, onde o traçado das ruas podem ser muito complexos, a presença de objetos tais como árvores, bicicletas, veículos podem gerar observações parciais e também estas observações são muitas vezes ruidosas ou ainda perdidas devido a completas oclusões. Portanto, o processo de percepção por natureza precisa ser capaz de lidar com a incerteza no conhecimento do mundo em torno do veículo. Nesta tese, este problema de percepção é analisado para a condução nos ambientes urbanos associado com a capacidade de realizar um deslocamento seguro baseado no processo de tomada de decisão em navegação autônoma. Projeta-se um sistema de percepção que permita veículos robóticos a trafegar autonomamente nas ruas, sem a necessidade de adaptar a infraestrutura, sem o conhecimento prévio do ambiente e considerando a presença de objetos dinâmicos tais como veículos. Propõe-se um novo método baseado em aprendizado de máquina para extrair o contexto semântico usando um par de imagens estéreo, a qual é vinculada a uma grade de ocupação evidencial que modela as incertezas de um ambiente urbano desconhecido, aplicando a teoria de Dempster-Shafer. Para a tomada de decisão no planejamento do caminho, aplica-se a abordagem dos tentáculos virtuais para gerar possíveis caminhos a partir do centro de referencia do veículo e com base nisto, duas novas estratégias são propostas. Em primeiro, uma nova estratégia para escolher o caminho correto para melhor evitar obstáculos e seguir a tarefa local no contexto da navegação hibrida e, em segundo, um novo controle de malha fechada baseado na odometria visual e o tentáculo virtual é modelado para execução do seguimento de caminho. Finalmente, um completo sistema automotivo integrando os modelos de percepção, planejamento e controle são implementados e validados experimentalmente em condições reais usando um veículo autônomo experimental, onde os resultados mostram que a abordagem desenvolvida realiza com sucesso uma segura navegação local com base em sensores de câmeraAbstract: The development of autonomous vehicles capable of getting around on urban roads can provide important benefits in reducing accidents, in increasing life comfort and also in providing cost savings. Intelligent vehicles for example often base their decisions on observations obtained from various sensors such as LIDAR, GPS and Cameras. Actually, camera sensors have been receiving large attention due to they are cheap, easy to employ and provide rich data information. Inner-city environments represent an interesting but also very challenging scenario in this context, where the road layout may be very complex, the presence of objects such as trees, bicycles, cars might generate partial observations and also these observations are often noisy or even missing due to heavy occlusions. Thus, perception process by nature needs to be able to deal with uncertainties in the knowledge of the world around the car. While highway navigation and autonomous driving using a prior knowledge of the environment have been demonstrating successfully, understanding and navigating general inner-city scenarios with little prior knowledge remains an unsolved problem. In this thesis, this perception problem is analyzed for driving in the inner-city environments associated with the capacity to perform a safe displacement based on decision-making process in autonomous navigation. It is designed a perception system that allows robotic-cars to drive autonomously on roads, without the need to adapt the infrastructure, without requiring previous knowledge of the environment and considering the presence of dynamic objects such as cars. It is proposed a novel method based on machine learning to extract the semantic context using a pair of stereo images, which is merged in an evidential grid to model the uncertainties of an unknown urban environment, applying the Dempster-Shafer theory. To make decisions in path-planning, it is applied the virtual tentacle approach to generate possible paths starting from ego-referenced car and based on it, two news strategies are proposed. First one, a new strategy to select the correct path to better avoid obstacles and to follow the local task in the context of hybrid navigation, and second, a new closed loop control based on visual odometry and virtual tentacle is modeled to path-following execution. Finally, a complete automotive system integrating the perception, path-planning and control modules are implemented and experimentally validated in real situations using an experimental autonomous car, where the results show that the developed approach successfully performs a safe local navigation based on camera sensorsDoutoradoMecanica dos Sólidos e Projeto MecanicoDoutor em Engenharia Mecânic

3D Positioning system with optical sensors using encoding techniques

Esta tesis doctoral se centra en el desarrollo y la mejora de los Sistemas de Posicionamiento Locales (LPS) en interiores, los cuales se utilizan en entornos no compatibles con señales GNSS (Global Navigation Satellite Systems) para localizar, seguir y guiar a personas, objetos o vehículos. Se han realizado numerosos estudios para llevar a cabo un sistema de posicionamiento en entornos interiores, donde las personas pasan aproximadamente el 80% de su tiempo. Algunas de las técnicas propuestas emplean diversas señales, como acústicas, de radiofrecuencia, mecánicas u ópticas, entre otras. Por su bajo coste, facilidad de integración en el entorno de trabajo y ausencia de riesgos para la salud, la tecnología óptica es una alternativa viable que ha comenzado a expandirse rápidamente. Esta tesis aporta propuestas que permiten establecer las bases para el desarrollo de un LPS óptico basado en técnicas de codificación y sensores QADA. Se han propuesto dos diseños: un LPS orientado a la privacidad, basado en un conjunto de cuatro LEDs transmisores, aunque fácilmente extensible a más emisores, que actúan como balizas en ubicaciones conocidas y un único sensor QADA que actúa como el receptor a posicionar; y un LPS centralizado basado en un conjunto de transmisores móviles y al menos dos receptores QADA colocados en ubicaciones conocidas. Se han estudiado los módulos transmisor y receptor. En concreto, se propone un esquema de codificación para la emisión del transmisor, que proporciona capacidad de acceso múltiple, así como robustez frente a bajas relaciones señal a ruido y condiciones adversas como los efectos de multicamino y cerca-lejos. Además, para mejorar las prestaciones de la propuesta sin aumentar significativamente el tiempo de emisión, se han analizado diferentes secuencias y sus longitudes, como los códigos LS (Loosely Synchronized) o las secuencias pseudoaleatorias (Kasami). Por otro lado, el módulo receptor está compuesto por un sensor QADA, una apertura cuadrada y una etapa de filtrado para reducir las interferencias no deseadas. El sensor QADA y la apertura se han modelado para, en primer lugar, analizar la influencia de la longitud de la apertura en la linealidad de las ecuaciones de estimación del punto imagen y, en segundo lugar, determinar los parámetros intrínsecos que modelan el receptor (longitud, altura, desalineación y descentrado de la apertura respecto al sensor QADA), de forma que se pueda implementar un algoritmo de calibración para mejorar la precisión del sistema propuesto. El LPS tiene como objetivo estimar la posición 3D de un objeto estático o en movimiento. Para ello, se diseñan varios algoritmos basados en técnicas de triangulación con determinación de ángulos de llegada (AoA) y técnicas homograficas que resuelven el problema de la perspectiva de n puntos (PnP) del sistema pin-hole propuesto. Todas las propuestas han sido verificadas mediante simulaciones y pruebas experimentales en una gran variedad de situaciones: utilizando luz visible o infrarroja, secuencias LS o Kasami, diferentes longitudes de apertura, distintas distancias entre transmisores y receptores, diferentes algoritmos de posicionamiento y varias rotaciones del receptor. Finalmente, las pruebas experimentales han demostrado que es posible posicionar con errores de menos de 5 centímetros

UAV or Drones for Remote Sensing Applications in GPS/GNSS Enabled and GPS/GNSS Denied Environments

The design of novel UAV systems and the use of UAV platforms integrated with robotic sensing and imaging techniques, as well as the development of processing workflows and the capacity of ultra-high temporal and spatial resolution data, have enabled a rapid uptake of UAVs and drones across several industries and application domains.This book provides a forum for high-quality peer-reviewed papers that broaden awareness and understanding of single- and multiple-UAV developments for remote sensing applications, and associated developments in sensor technology, data processing and communications, and UAV system design and sensing capabilities in GPS-enabled and, more broadly, Global Navigation Satellite System (GNSS)-enabled and GPS/GNSS-denied environments.Contributions include:UAV-based photogrammetry, laser scanning, multispectral imaging, hyperspectral imaging, and thermal imaging;UAV sensor applications; spatial ecology; pest detection; reef; forestry; volcanology; precision agriculture wildlife species tracking; search and rescue; target tracking; atmosphere monitoring; chemical, biological, and natural disaster phenomena; fire prevention, flood prevention; volcanic monitoring; pollution monitoring; microclimates; and land use;Wildlife and target detection and recognition from UAV imagery using deep learning and machine learning techniques;UAV-based change detection

Exploration, navigation and localization for mobile robots.

he main goal of this thesis is the advancement of the state of the art in mobile robot autonomy. In order to achieve this objective, several contributions have been presented that tackle well defined problems in the areas of localization, navigation and exploration. The very first contribution is focused on the task of robustly finding the localization of a mobile robot in an outdoor environment. Specifically, the presented technique introduces a key methodolgy to perform sensor fusion of a global localization sensor so ubiquitous as a GPS device, within the context of a particle filter based Monte Carlo localization system. We focus on the management of multiple sensor data sources under noisy and conflicting readings. This strategy allows for a reduced uncertainty in the robot pose estimation, as well as improved robustness of the system. The second contribution presents a completely integrated navigation system running within a constrained and highly dynamic platform like a quadrotor, applied to full 3D environments. The navigation stack comprises a Simultaneous Localization and Mapping (SLAM) system for RGB-D cameras that provides both the robot pose and an obstacle map of the environment, as well as a 4D path planner capable of finding obstacle free and kinematically feasible trajectories for the quadrotor to navigate this environment. The third contribution introduces a novel approach for autonomous exploration of unknown environments with robust homing. We present a technique to predict possible environment structures in the unseen parts of the robot's surroundings based on previously explored environments. We exploit this belief to predict possible loop closures that the robot may experience when exploring an unknown part of the scene. This allows the robot to actively reduce the uncertainty in its belief through its exploration actions. Also, we introduce a robust homing system that addresses the problem of returning a robot operating in an unknown environment to its starting position even if the underlying SLAM system fails. All contributions where designed, implemented and tested on real autonomous robots: a self-driving car, a micro aerial vehicle and an underground exploration platform

Precise positioning in real-time using GPS-RTK signal for visually impaired people navigation system

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University, 24/9/2010.This thesis presents the research carried out to investigate and achieve highly reliable and accurate navigation system of guidance for visually impaired pedestrians. The main aim with this PhD project has been to identify the limits and insufficiencies in utilising Network Real-Time Kinematic Global Navigation Satellite Systems (NRTK GNSS) and its augmentation techniques within the frame of pedestrian applications in a variety of environments and circumstances. Moreover, the system can be used in many other applications, including unmanned vehicles, military applications, police, etc. NRTK GNSS positioning is considered to be a superior solution in comparison to the conventional standalone Global Positioning System (GPS) technique whose accuracy is highly affected by the distance dependent errors such as satellite orbital and atmospheric biases. Nevertheless, NRTK GNSS positioning is particularly constrained by wireless data link coverage, delays of correction and transmission and completeness, GPS and GLONASS signal availability, etc., which could downgrade the positioning quality of the NRTK results. This research is based on the dual frequency NRTK GNSS (GPS and GLONASS). Additionally, it is incorporated into several positioning and communication methods responsible for data correction while providing the position solutions, in which all identified contextual factors and application requirements are accounted. The positioning model operates through client-server based architecture consisted of a Navigation Service Centre (NSC) and a Mobile Navigation Unit (MNU). Hybrid functional approaches were consisting of several processing procedures allowing the positioning model to operate in position determination modes. NRTK GNSS and augmentation service is used if enough navigation information was available at the MNU using its local positioning device (GPS/GLONASS receiver).The positioning model at MNU was experimentally evaluated and centimetric accuracy was generally attained during both static and kinematic tests in various environments (urban, suburban and rural). This high accuracy was merely affected by some level of unavailability mainly caused by GPS and GLONASS signal blockage. Additionally, the influence of the number of satellites in view, dilution of precision (DOP) and age corrections (AoC) over the accuracy and stability of the NRTK GNSS solution was also investigated during this research and presented in the thesis. This positioning performance has outperformed the existing GPS service. In addition, utilising a simulation evaluation facility the positioning model at MNU performance was quantified with reference to a hybrid positioning service that will be offered by future Galileo Open Service (OS) along with GPS. However, a significant difference in terms of the service availability for the advantage of the hybrid system was experienced in all remaining scenarios and environments more especially the urban areas due to surrounding obstacles and conditions. As an outcome of this research a new and precise positioning model was proposed. The adaptive framework is understood as approaching an integration of the available positioning technology into the context of surrounding wireless communication for a maintainable performance. The positioning model has the capability of delivering indeed accurate, precise and consistent position solutions, and thus is fulfilling the requirements of visually impaired people navigation application, as identified in the adaptive framework