Robust Multi-sensor Data Fusion for Practical Unmanned Surface Vehicles (USVs) Navigation

The development of practical Unmanned Surface Vehicles (USVs) are attracting increasing attention driven by their assorted military and commercial application potential. However, addressing the uncertainties presented in practical navigational sensor measurements of an USV in maritime environment remain the main challenge of the development. This research aims to develop a multi-sensor data fusion system to autonomously provide an USV reliable navigational information on its own positions and headings as well as to detect dynamic target ships in the surrounding environment in a holistic fashion. A multi-sensor data fusion algorithm based on Unscented Kalman Filter (UKF) has been developed to generate more accurate estimations of USV’s navigational data considering practical environmental disturbances. A novel covariance matching adaptive estimation algorithm has been proposed to deal with the issues caused by unknown and varying sensor noise in practice to improve system robustness. Certain measures have been designed to determine the system reliability numerically, to recover USV trajectory during short term sensor signal loss, and to autonomously detect and discard permanently malfunctioned sensors, and thereby enabling potential sensor faults tolerance. The performance of the algorithms have been assessed by carrying out theoretical simulations as well as using experimental data collected from a real-world USV projected collaborated with Plymouth University. To increase the degree of autonomy of USVs in perceiving surrounding environments, target detection and prediction algorithms using an Automatic Identification System (AIS) in conjunction with a marine radar have been proposed to provide full detections of multiple dynamic targets in a wider coverage range, remedying the narrow detection range and sensor uncertainties of the AIS. The detection algorithms have been validated in simulations using practical environments with water current effects. The performance of developed multi-senor data fusion system in providing reliable navigational data and perceiving surrounding environment for USV navigation have been comprehensively demonstrated

Decentralized Riemannian Particle Filtering with Applications to Multi-Agent Localization

The primary focus of this research is to develop consistent nonlinear decentralized particle filtering approaches to the problem of multiple agent localization. A key aspect in our development is the use of Riemannian geometry to exploit the inherently non-Euclidean characteristics that are typical when considering multiple agent localization scenarios. A decentralized formulation is considered due to the practical advantages it provides over centralized fusion architectures. Inspiration is taken from the relatively new field of information geometry and the more established research field of computer vision. Differential geometric tools such as manifolds, geodesics, tangent spaces, exponential, and logarithmic mappings are used extensively to describe probabilistic quantities. Numerous probabilistic parameterizations were identified, settling on the efficient square-root probability density function parameterization. The square-root parameterization has the benefit of allowing filter calculations to be carried out on the well studied Riemannian unit hypersphere. A key advantage for selecting the unit hypersphere is that it permits closed-form calculations, a characteristic that is not shared by current solution approaches. Through the use of the Riemannian geometry of the unit hypersphere, we are able to demonstrate the ability to produce estimates that are not overly optimistic. Results are presented that clearly show the ability of the proposed approaches to outperform current state-of-the-art decentralized particle filtering methods. In particular, results are presented that emphasize the achievable improvement in estimation error, estimator consistency, and required computational burden

Contributions to improve the technologies supporting unmanned aircraft operations

Mención Internacional en el título de doctorUnmanned Aerial Vehicles (UAVs), in their smaller versions known as drones, are becoming increasingly important in today's societies. The systems that make them up present a multitude of challenges, of which error can be considered the common denominator. The perception of the environment is measured by sensors that have errors, the models that interpret the information and/or define behaviors are approximations of the world and therefore also have errors. Explaining error allows extending the limits of deterministic models to address real-world problems. The performance of the technologies embedded in drones depends on our ability to understand, model, and control the error of the systems that integrate them, as well as new technologies that may emerge. Flight controllers integrate various subsystems that are generally dependent on other systems. One example is the guidance systems. These systems provide the engine's propulsion controller with the necessary information to accomplish a desired mission. For this purpose, the flight controller is made up of a control law for the guidance system that reacts to the information perceived by the perception and navigation systems. The error of any of the subsystems propagates through the ecosystem of the controller, so the study of each of them is essential. On the other hand, among the strategies for error control are state-space estimators, where the Kalman filter has been a great ally of engineers since its appearance in the 1960s. Kalman filters are at the heart of information fusion systems, minimizing the error covariance of the system and allowing the measured states to be filtered and estimated in the absence of observations. State Space Models (SSM) are developed based on a set of hypotheses for modeling the world. Among the assumptions are that the models of the world must be linear, Markovian, and that the error of their models must be Gaussian. In general, systems are not linear, so linearization are performed on models that are already approximations of the world. In other cases, the noise to be controlled is not Gaussian, but it is approximated to that distribution in order to be able to deal with it. On the other hand, many systems are not Markovian, i.e., their states do not depend only on the previous state, but there are other dependencies that state space models cannot handle. This thesis deals a collection of studies in which error is formulated and reduced. First, the error in a computer vision-based precision landing system is studied, then estimation and filtering problems from the deep learning approach are addressed. Finally, classification concepts with deep learning over trajectories are studied. The first case of the collection xviiistudies the consequences of error propagation in a machine vision-based precision landing system. This paper proposes a set of strategies to reduce the impact on the guidance system, and ultimately reduce the error. The next two studies approach the estimation and filtering problem from the deep learning approach, where error is a function to be minimized by learning. The last case of the collection deals with a trajectory classification problem with real data. This work completes the two main fields in deep learning, regression and classification, where the error is considered as a probability function of class membership.Los vehículos aéreos no tripulados (UAV) en sus versiones de pequeño tamaño conocidos como drones, van tomando protagonismo en las sociedades actuales. Los sistemas que los componen presentan multitud de retos entre los cuales el error se puede considerar como el denominador común. La percepción del entorno se mide mediante sensores que tienen error, los modelos que interpretan la información y/o definen comportamientos son aproximaciones del mundo y por consiguiente también presentan error. Explicar el error permite extender los límites de los modelos deterministas para abordar problemas del mundo real. El rendimiento de las tecnologías embarcadas en los drones, dependen de nuestra capacidad de comprender, modelar y controlar el error de los sistemas que los integran, así como de las nuevas tecnologías que puedan surgir. Los controladores de vuelo integran diferentes subsistemas los cuales generalmente son dependientes de otros sistemas. Un caso de esta situación son los sistemas de guiado. Estos sistemas son los encargados de proporcionar al controlador de los motores información necesaria para cumplir con una misión deseada. Para ello se componen de una ley de control de guiado que reacciona a la información percibida por los sistemas de percepción y navegación. El error de cualquiera de estos sistemas se propaga por el ecosistema del controlador siendo vital su estudio. Por otro lado, entre las estrategias para abordar el control del error se encuentran los estimadores en espacios de estados, donde el filtro de Kalman desde su aparición en los años 60, ha sido y continúa siendo un gran aliado para los ingenieros. Los filtros de Kalman son el corazón de los sistemas de fusión de información, los cuales minimizan la covarianza del error del sistema, permitiendo filtrar los estados medidos y estimarlos cuando no se tienen observaciones. Los modelos de espacios de estados se desarrollan en base a un conjunto de hipótesis para modelar el mundo. Entre las hipótesis se encuentra que los modelos del mundo han de ser lineales, markovianos y que el error de sus modelos ha de ser gaussiano. Generalmente los sistemas no son lineales por lo que se realizan linealizaciones sobre modelos que a su vez ya son aproximaciones del mundo. En otros casos el ruido que se desea controlar no es gaussiano, pero se aproxima a esta distribución para poder abordarlo. Por otro lado, multitud de sistemas no son markovianos, es decir, sus estados no solo dependen del estado anterior, sino que existen otras dependencias que los modelos de espacio de estados no son capaces de abordar. Esta tesis aborda un compendio de estudios sobre los que se formula y reduce el error. En primer lugar, se estudia el error en un sistema de aterrizaje de precisión basado en visión por computador. Después se plantean problemas de estimación y filtrado desde la aproximación del aprendizaje profundo. Por último, se estudian los conceptos de clasificación con aprendizaje profundo sobre trayectorias. El primer caso del compendio estudia las consecuencias de la propagación del error de un sistema de aterrizaje de precisión basado en visión artificial. En este trabajo se propone un conjunto de estrategias para reducir el impacto sobre el sistema de guiado, y en última instancia reducir el error. Los siguientes dos estudios abordan el problema de estimación y filtrado desde la perspectiva del aprendizaje profundo, donde el error es una función que minimizar mediante aprendizaje. El último caso del compendio aborda un problema de clasificación de trayectorias con datos reales. Con este trabajo se completan los dos campos principales en aprendizaje profundo, regresión y clasificación, donde se plantea el error como una función de probabilidad de pertenencia a una clase.I would like to thank the Ministry of Science and Innovation for granting me the funding with reference PRE2018-086793, associated to the project TEC2017-88048-C2-2-R, which provide me the opportunity to carry out all my PhD. activities, including completing an international research internship.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: Antonio Berlanga de Jesús.- Secretario: Daniel Arias Medina.- Vocal: Alejandro Martínez Cav

World Modeling for Intelligent Autonomous Systems

The functioning of intelligent autonomous systems requires constant situation awareness and cognition analysis. Thus, it needs a memory structure that contains a description of the surrounding environment (world model) and serves as a central information hub. This book presents a row of theoretical and experimental results in the field of world modeling. This includes areas of dynamic and prior knowledge modeling, information fusion, management and qualitative/quantitative information analysis

World Modeling for Intelligent Autonomous Systems

The functioning of intelligent autonomous systems requires constant situation awareness and cognition analysis. Thus, it needs a memory structure that contains a description of the surrounding environment (world model) and serves as a central information hub. This book presents a row of theoretical and experimental results in the field of world modeling. This includes areas of dynamic and prior knowledge modeling, information fusion, management and qualitative/quantitative information analysis

Perception and intelligent localization for autonomous driving

Mestrado em Engenharia de Computadores e TelemáticaVisão por computador e fusão sensorial são temas relativamente recentes, no entanto largamente adoptados no desenvolvimento de robôs autónomos que exigem adaptabilidade ao seu ambiente envolvente. Esta dissertação foca-se numa abordagem a estes dois temas para alcançar percepção no contexto de condução autónoma. O uso de câmaras para atingir este fim é um processo bastante complexo. Ao contrário dos meios sensoriais clássicos que fornecem sempre o mesmo tipo de informação precisa e atingida de forma determinística, as sucessivas imagens adquiridas por uma câmara estão repletas da mais variada informação e toda esta ambígua e extremamente difícil de extrair. A utilização de câmaras como meio sensorial em robótica é o mais próximo que chegamos na semelhança com aquele que é o de maior importância no processo de percepção humana, o sistema de visão. Visão por computador é uma disciplina científica que engloba àreas como: processamento de sinal, inteligência artificial, matemática, teoria de controlo, neurobiologia e física. A plataforma de suporte ao estudo desenvolvido no âmbito desta dissertação é o ROTA (RObô Triciclo Autónomo) e todos os elementos que consistem o seu ambiente. No contexto deste, são descritas abordagens que foram introduzidas com fim de desenvolver soluções para todos os desafios que o robô enfrenta no seu ambiente: detecção de linhas de estrada e consequente percepção desta, detecção de obstáculos, semáforos, zona da passadeira e zona de obras. É também descrito um sistema de calibração e aplicação da remoção da perspectiva da imagem, desenvolvido de modo a mapear os elementos percepcionados em distâncias reais. Em consequência do sistema de percepção, é ainda abordado o desenvolvimento de auto-localização integrado numa arquitectura distribuída incluindo navegação com planeamento inteligente. Todo o trabalho desenvolvido no decurso da dissertação é essencialmente centrado no desenvolvimento de percepção robótica no contexto de condução autónoma.Computer vision and sensor fusion are subjects that are quite recent, however widely adopted in the development of autonomous robots that require adaptability to their surrounding environment. This thesis gives an approach on both in order to achieve perception in the scope of autonomous driving. The use of cameras to achieve this goal is a rather complex subject. Unlike the classic sensorial devices that provide the same type of information with precision and achieve this in a deterministic way, the successive images acquired by a camera are replete with the most varied information, that this ambiguous and extremely dificult to extract. The use of cameras for robotic sensing is the closest we got within the similarities with what is of most importance in the process of human perception, the vision system. Computer vision is a scientific discipline that encompasses areas such as signal processing, artificial intelligence, mathematics, control theory, neurobiology and physics. The support platform in which the study within this thesis was developed, includes ROTA (RObô Triciclo Autónomo) and all elements comprising its environment. In its context, are described approaches that introduced in the platform in order to develop solutions for all the challenges facing the robot in its environment: detection of lane markings and its consequent perception, obstacle detection, trafic lights, crosswalk and road maintenance area. It is also described a calibration system and implementation for the removal of the image perspective, developed in order to map the elements perceived in actual real world distances. As a result of the perception system development, it is also addressed self-localization integrated in a distributed architecture that allows navigation with long term planning. All the work developed in the course of this work is essentially focused on robotic perception in the context of autonomous driving

Discrete Time Systems

Discrete-Time Systems comprehend an important and broad research field. The consolidation of digital-based computational means in the present, pushes a technological tool into the field with a tremendous impact in areas like Control, Signal Processing, Communications, System Modelling and related Applications. This book attempts to give a scope in the wide area of Discrete-Time Systems. Their contents are grouped conveniently in sections according to significant areas, namely Filtering, Fixed and Adaptive Control Systems, Stability Problems and Miscellaneous Applications. We think that the contribution of the book enlarges the field of the Discrete-Time Systems with signification in the present state-of-the-art. Despite the vertiginous advance in the field, we also believe that the topics described here allow us also to look through some main tendencies in the next years in the research area

Cooperative Localization in Mobile Underwater Acoustic Sensor Networks

Die großflächige Erkundung und Überwachung von Tiefseegebieten gewinnt mehr und mehr an Bedeutung für Industrie und Wissenschaft. Diese schwer zugänglichen Areale in der Tiefsee können nur mittels Teams unbemannter Tauchbote effizient erkundet werden. Aufgrund der hohen Kosten, war bisher ein Einsatz von mehreren autonomen Unterwasserfahrzeugen (AUV) wirtschaftlich undenkbar, wodurch AUV-Teams nur in Simulationen erforscht werden konnten. In den letzten Jahren konnte jedoch eine Entwicklung hin zu günstigeren und robusteren AUVs beobachtet werden. Somit wird der Einsatz von AUV-Teams in Zukunft zu einer realen Option. Die wachsende Nachfrage nach Technologien zur Unterwasseraufklärung und Überwachung konnte diese Entwicklung noch zusätzlich beschleunigen. Eine der größten technischen Hürden für tief tauchende AUVs ist die Unterwasserlokalisierug. Satelitengestützte Navigation ist in der Tiefe nicht möglich, da Radiowellen bereits nach wenigen Metern im Wasser stark an Intensität verlieren. Daher müssen neue Ansätze für die Unterwasserlokalisierung entwickelt werden die sich auch für Fahrzeugenverbände skalieren lassen. Der Einsatz von AUV-Teams ermöglicht nicht nur völlig neue Möglichkeiten der Kooperation, sondern erlaubt auch jedem einzelnen AUV von den Navigationsdaten der anderen Fahrzeuge im Verband zu profitieren, um die eigene Lokalisierung zu verbessern. In dieser Arbeit wird ein kooperativer Lokalisierungsansatz vorgestellt, welcher auf dem Nachrichtenaustausch durch akustische Ultra-Short Base-Line (USBL) Modems basiert. Ein akustisches Modem ermöglicht die Übertragung von Datenpaketen im Wasser, wärend ein USBL-Sensor die Richtung einer akustischen Quelle bestimmen kann. Durch die Kombination von Modem und Sensor entsteht ein wichtiges Messinstrument für die Unterwasserlokalisierung. Wenn ein Fahrzeug ein Datenpaket mit seiner eignen Position aussendet, können andere Fahrzeuge mit einem USBL-Modem diese Nachricht empfangen. In Verbindung mit der Richtungsmessung zur Quelle, können diese Daten von einem Empfangenden AUV verwendet werden, um seine eigene Positionsschatzung zu verbessern. Diese Arbeit schlägt einen Ansatz zur Fusionierung der empfangenen Nachricht mit der Richtungsmessung vor, welcher auch die jeweiligen Messungenauigkeiten berücksichtigt. Um die Messungenauigkeit des komplexen USBL-Sensors bestimmen zu können, wurde zudem ein detailliertes Sensormodell entwickelt. Zunächst wurden existierende Ansätze zur kooperativen Lokalisierung (CL) untersucht, um daraus eine Liste von erwünschten Eigenschaften für eine CL abzuleiten. Darauf aufbauend wurde der Deep-Sea Network Lokalisation (DNL) Ansatz entwickelt. Bei DNL handelt es sich um eine CL Methode, bei der die Skalierbarkeit sowie die praktische Anwendbarkeit im Fokus stehen. DNL ist als eine Zwischenschicht konzipiert, welche USBL-Modem und Navigationssystem miteinander verbindet. Es werden dabei Messwerte und Kommunikationsdaten des USBL zu einer Standortbestimmung inklusive Richtungsschätzung fusioniert und an das Navigationssystem weiter geleitet, ähnlich einem GPS-Sensor. Die Funktionalität von USBL-Modell und DNL konnten evaluiert werden anhand von Messdaten aus Seeerprobungen in der Ostsee sowie im Mittelatlantik. Die Qualität einer CL hangt häufig von vielen unterschiedlichen Faktoren ab. Die Netzwerktopologie muss genauso berücksichtig werden wie die Lokalisierungsfähigkeiten jedes einzelnen Teilnehmers. Auch das Kommunikationsverhalten der einzelnen Teilnehmer bestimmt, welche Informationen im Netzwerk vorhanden sind und hat somit einen starken Einfluss auf die CL. Um diese Einflussfaktoren zu untersuchen, wurden eine Reihe von Szenarien simuliert, in denen Kommunikationsverhalten und Netzwerktopologie für eine Gruppe von AUVs variiert wurden. In diesen Experimenten wurden die AUVs durch ein Oberflächenfahrzeug unterstützt, welches seine geo-referenzierte Position über DNL an die getauchten Fahrzeuge weiter leitete. Anhand der untersuchten Topologie können die Experimente eingeteilt werden in Single-Hop und Multi-Hop. Single-Hop bedeutet, dass jedes AUV sich in der Sendereichweite des Oberflächenfahrzeugs befindet und dessen Positionsdaten auf direktem Wege erhält. Wie die Ergebnisse der Single-Hop Experimente zeigen, kann der Lokalisierungsfehler der AUVs eingegrenzt werden, wenn man DNL verwendet. Dabei korreliert der Lokalisierungsfehler mit der kombinierten Ungenauigkeit von USBL-Messung und Oberflächenfahrzeugposition. Bei den Multi-Hop Experimenten wurde die Topologie so geändert, dass sich nur eines der AUVs in direkter Sendereichweite des Oberflächenfahrzeugs befindet. Dieses AUV verbessert seine Position mit den empfangen Daten des Oberflächenfahrzeugs und sendet wiederum seine verbesserte Position an die anderen AUVs. Auch hier konnte gezeigt werden, dass sich der Lokalisierungfehler der Gruppe mit DNL einschränken lässt. Ändert man nun das Schema der Kommunikation so, dass alle AUVs zyklisch ihre Position senden, zeigte sich eine Verschlechterung der Lokalisierungsqualität der Gruppe. Dieses unerwartet Ergebnis konnte auf einen Teil des DNL-Algorithmus zurück geführt werden. Da die verwendete USBL-Klasse nur die Richtung eines Signals misst, nicht jedoch die Entfernung zum Sender, wird in der DNL-Schicht eine Entfernungsschatzung vorgenommen. Wenn die Kommunikation nicht streng unidirektional ist, entsteht eine Ruckkopplungsschleife, was zu fehlerhaften Entfernungsschatzungen führt. Im letzten Experiment wird gezeigt wie sich dieses Problem vermeiden lasst, mithilfe einer relativ neue USBL-Klasse, die sowohl Richtung als auch Entfernung zum Sender misst. Die zwei wesentlichen Beiträge dieser Arbeit sind das USBL-Model zum einen und zum Anderen, der neue kooperative Lokalisierungsansatz DNL. Mithilfe des Sensormodels lassen sich nicht nur Messabweichungen einer USBL-Messung bestimmen, es kann auch dazu genutzt werden, einige Fehlereinflüsse zu korrigieren. Mit DNL wurde eine skalierbare CL-Methode entwickelt, die sich gut für den den Einsatz bei mobilen Unterwassersensornetzwerken eignet. Durch das Konzept als Zwischenschicht, lasst sich DNL einfach in bestehende Navigationslösungen integrieren, um die Langzeitstabilität der Navigation für große Verbände von tiefgetauchten Fahrzeugen zu gewährleisten. Sowohl USBL-Model als auch DNL sind dabei so ressourcenschonend, dass sie auf dem Computer eines Standard USBL laufen können, ohne die ursprüngliche Funktionalität einzuschränken, was den praktischen Einsatz zusätzlich vereinfacht

Cooperative Perception for Social Driving in Connected Vehicle Traffic

The development of autonomous vehicle technology has moved to the center of automotive research in recent decades. In the foreseeable future, road vehicles at all levels of automation and connectivity will be required to operate safely in a hybrid traffic where human operated vehicles (HOVs) and fully and semi-autonomous vehicles (AVs) coexist. Having an accurate and reliable perception of the road is an important requirement for achieving this objective. This dissertation addresses some of the associated challenges via developing a human-like social driver model and devising a decentralized cooperative perception framework. A human-like driver model can aid the development of AVs by building an understanding of interactions among human drivers and AVs in a hybrid traffic, therefore facilitating an efficient and safe integration. The presented social driver model categorizes and defines the driver\u27s psychological decision factors in mathematical representations (target force, object force, and lane force). A model predictive control (MPC) is then employed for the motion planning by evaluating the prevailing social forces and considering the kinematics of the controlled vehicle as well as other operating constraints to ensure a safe maneuver in a way that mimics the predictive nature of the human driver\u27s decision making process. A hierarchical model predictive control structure is also proposed, where an additional upper level controller aggregates the social forces over a longer prediction horizon upon the availability of an extended perception of the upcoming traffic via vehicular networking. Based on the prediction of the upper level controller, a sequence of reference lanes is passed to a lower level controller to track while avoiding local obstacles. This hierarchical scheme helps reduce unnecessary lane changes resulting in smoother maneuvers. The dynamic vehicular communication environment requires a robust framework that must consistently evaluate and exploit the set of communicated information for the purpose of improving the perception of a participating vehicle beyond the limitations. This dissertation presents a decentralized cooperative perception framework that considers uncertainties in traffic measurements and allows scalability (for various settings of traffic density, participation rate, etc.). The framework utilizes a Bhattacharyya distance filter (BDF) for data association and a fast covariance intersection fusion scheme (FCI) for the data fusion processes. The conservatism of the covariance intersection fusion scheme is investigated in comparison to the traditional Kalman filter (KF), and two different fusion architectures: sensor-to-sensor and sensor-to-system track fusion are evaluated. The performance of the overall proposed framework is demonstrated via Monte Carlo simulations with a set of empirical communications models and traffic microsimulations where each connected vehicle asynchronously broadcasts its local perception consisting of estimates of the motion states of self and neighboring vehicles along with the corresponding uncertainty measures of the estimates. The evaluated framework includes a vehicle-to-vehicle (V2V) communication model that considers intermittent communications as well as a model that takes into account dynamic changes in an individual vehicle’s sensors’ FoV in accordance with the prevailing traffic conditions. The results show the presence of optimality in participation rate, where increasing participation rate beyond a certain level adversely affects the delay in packet delivery and the computational complexity in data association and fusion processes increase without a significant improvement in the achieved accuracy via the cooperative perception. In a highly dense traffic environment, the vehicular network can often be congested leading to limited bandwidth availability at high participation rates of the connected vehicles in the cooperative perception scheme. To alleviate the bandwidth utilization issues, an information-value discriminating networking scheme is proposed, where each sender broadcasts selectively chosen perception data based on the novelty-value of information. The potential benefits of these approaches include, but are not limited to, the reduction of bandwidth bottle-necking and the minimization of the computational cost of data association and fusion post processing of the shared perception data at receiving nodes. It is argued that the proposed information-value discriminating communication scheme can alleviate these adverse effects without sacrificing the fidelity of the perception