Towards autonomous robotic systems: seamless localization and trajectory planning in dynamic environments

Evolucionar hacia una sociedad más automatizada y robotizada en la que podamos convivir con sistemas robóticos que desempeñen tareas poco atractivas o peligrosas para el ser humano, supone plantearnos, entre otras cuestiones, qué soluciones existen actualmente y cuáles son las mejoras a incorporar a las mismas. La mayoría de aplicaciones ya desarrolladas son soluciones robustas y adecuadas para el fin que se diseñan. Sin embargo, muchas de las técnicas implantadas podrían funcionar de manera más eficiente o bien adaptarse a otras necesidades. Asimismo, en la mayoría de aplicaciones robóticas adquiere importancia el contexto en el que desempeñan su función. Hay entornos estructurados y fáciles de modelar, mientras que otros apenas presentan características utilizables para obtener información de los mismos.Esta tesis se centra en dos de las funciones básicas que debe tener cualquier sistema robótico autónomo para desplazarse de forma robusta en cualquier tipo de entorno: la localización y el cálculo de trayectorias seguras. Además, los escenarios en los que se desea poner en práctica la investigación son complejos: un parque industrial con zonas cuyas características de entorno (usualmente geométricas) son utilizadas para que un robot se localice, varían; y entornos altamente ocupados por otros agentes móviles, como el vestíbulo de un teatro, en los que se debe considerar las características dinámicas de los demás para calcular un movimiento que sea seguro tanto para el robot como para los demás agentes.La información que se puede percibir de los escenarios con ambientes no homogéneos, por ejemplo de interior y exterior, suele ser de características diferentes. Cuando la información que se dispone del entorno proviene de sensores diferentes hay que definir un método que integre las medidas para tener una estimación de la localización del robot en todo momento. El tema de la localización se ha investigado intensamente y existen soluciones robustas en interior y exterior, pero no tanto en zonas mixtas. En las zonas de transición interior-exterior y viceversa es necesario utilizar sensores que funcionan correctamente en ambas zonas, realizando una integración sensorial durante la transición para evitar discontinuidades en la localización o incluso que el robot se pierda. De esta manera la navegación autónoma, dependiente de la correcta localización, funcionará sin discontinuidades ni movimientos bruscos.En entornos dinámicos es esencial definir una forma de representar la información que refleje su naturaleza cambiante. Por ello, se han definido en la literatura diferentes modelos que representan el dinamismo del entorno, y que permiten desarrollar una planificación de trayectorias directamente sobre las variables que controlan el movimiento del robot, en nuestro caso, las velocidades angular y lineal para un robot diferencial. Los planificadores de trayectorias y navegadores diseñados para entornos estáticos no funcionan correctamente en escenarios dinámicos, ya que son puramente reactivos. Es necesario tener en cuenta la predicción del movimiento de los obstáculos móviles para planificar trayectorias seguras sin colisión. Los temas abordados y las contribuciones aportadas en esta tesis son:• Diseño de un sistema de localización continua en entornos de interior y exterior, poniendo especial interés en la fusión de las medidas obtenidas de diferentes sensores durante las transiciones interior-exterior, aspecto poco abordado en la literatura. De esta manera se obtiene una estimación acotada de la localización durante toda la navegación del robot. Además, la localización se integra con una técnica reactiva de navegación, construyendo un sistema completo de navegación. El sistema integrado se ha evaluado en un escenario real de un parque industrial, para una aplicación logística en la que las transiciones interior-exterior y viceversa suponían un problema fundamental a resolver.• Definición de un modelo para representar el entorno dinámico del robot, llamado Dynamic Obstacle Velocity-Time Space (DOVTS). En este modelo aparecen representadas las velocidades permitidas y prohibidas para que el robot evite las colisiones con los obstáculos de alrededor. Este modelo puede ser utilizado por algoritmos de navegación ya existentes, y sirve de base para las nuevas técnicas de navegación desarrolladas en la tesis y explicadas en los siguientes puntos. • Desarrollo de una técnica de planificación y navegación basada en el modelo DOVTS. En este modelo se identifica un conjunto de situaciones relativas entre el robot y los obstáculos. A cada situación se asocia una estrategia de navegación, que considera la seguridad del robot para evitar colisiones, a la vez que intenta minimizar el tiempo al objetivo.• Implementación de una técnica de planificación y navegación basada en el modelo DOVTS, que utiliza explícitamente la información del tiempo para la planificación del movimiento. Se desarrolla un algoritmo A*-like que planifica los movimientos de los siguientes instantes, incrementando la maniobrabilidad del robot para la evitación de obstáculos respecto al método del anterior punto, a costa de un mayor tiempo de cómputo. Se analizan las diferencias en el comportamiento global del robot con respecto a la técnica anterior.Los diferentes aspectos que se han investigado en esta tesis tratan de avanzar en el objetivo de conseguir robots autónomos que puedan adaptarse a nuestra vida cotidiana en escenarios que son típicamente dinámicos de una forma natural y segura.<br /

Spatial Road Representation for Driving in Complex Scenes by Interpretation of Traffic Behavior

Casapietra E. Spatial Road Representation for Driving in Complex Scenes by Interpretation of Traffic Behavior. Bielefeld: Universität Bielefeld; 2019.The detection of road layout and semantics is an important issue in modern Advanced Driving Assistance Systems (ADAS) and autonomous driving systems. In particular, trajectory planning algorithms need a road representation to operate on: this representation has to be spatial, as the system needs to know exactly on which areas it is safe to drive, so that they can safely plan fine maneuvers. Since typical trajectories are computed for timespans in the order of seconds, the spatial detection range needed for the road representation to achieve a stable and smooth trajectory is in the tenths to hundreds of meters. Direct detection, i.e. the usage of sensors that detect road area by direct observation (e.g. cameras or lasers), is often not sufficient to achieve this range, especially in inner-city, due to occlusions caused by various obstacles (e.g. buildings and high traffic) as well as hardware limitations. State-of-the-art systems cope with this problem by employing annotated road maps to complement direct detection. However, maps are expensive to make and not available on every road. Furthermore, ego-localization is a key issue in their usage. This thesis presents a novel approach that creates a spatial road representation derived from both direct and indirect road detection, i.e. the detection and interpretation of other cues for the purpose of inferring the road area layout. Direct detection on monocular images is provided by RTDS, a feature-based detection system that provides road terrain confidence. Indirect detection is based on the interpretation of the other vehicles' behavior. Since our main assumption is that vehicles move on road area, we estimate their past and future movements to infer the road layout where we cannot see it directly. The estimation is carried out using a function that models the probability for each vehicle to traverse each patch of the representation, taking into account position, direction and speed of the vehicle, as well as the possibility of small past and future maneuvers. The behavior of each vehicle is used not only to infer the area where road is, but also to infer where there is not. In fact, observing a vehicle steering away from an area it was predicted to go can be interpreted as evidence that said area is not road. The road confidences provided by RTDS and behavior interpretation are blended together by means of a visibility function that gives different weights to the two sources, according to the position of the patch in the field of view and possible occlusions that would prevent the camera to see the patch, thereby leading to unreliable results from RTDS. The addition of indirect detection improves the spatial range of the representation. It also exploits the scenarios of high traffic that are the most challenging ones for direct detection systems, and allows for the inclusion of additional semantics, such as lanes and driving directions. Geometrical considerations are applied to the road layout, obtaining a distributed measure of road width and orientation. These values are used to segment the road, and each segment is then divided into multiple lanes based on its width and the average width of a lane. Finally, a driving direction is assigned to each lane by observing the behavior of the other vehicles on it. The road representation is evaluated by comparison with a ground truth obtained from manually annotated real world images. As in most cases the entirety of road area cannot be seen in a single image (a problem that human users share with direct detection systems), every road is annotated in multiple different images, and the road portions observed are converted into BEV and fused together using GPS to form a comprehensive view of said road. This ground truth is then compared patch-wise to the representation obtained by our system, showing a clear improvement with respect to the representation obtained by RTDS alone. In order to demonstrate the advantages of our approach in concrete applications, we set up a system that couples our road representation with a basic trajectory planner. The system reads real-world data, recorded by a mobile platform. The representation is computed at each frame of the stream. The trajectory planner receives the current state of the ego-car (position, direction and speed) and the location of a target area (from a navigational map), and finds the path that leads to the target area with minimum cost. We show that indirect road detection complements direct detection in a way that leads to a substantial increase in spatial detection range and quality of the internal road representation, thereby improving the smoothness of trajectories that planners can compute, as well as their robustness over time, since the road layout in the representation does not dramatically change only when a new road is visible. This result can help autonomous driving systems to achieve a more human-like behavior, as their improved road awareness allows them to plan ahead, including areas they do not see yet, just as humans normally do

Human-like motorway lane change trajectory planning for autonomous vehicles.

The human lifestyle can be foreseen to have a tremendous change once the automation of transportation has been fully realised. The majority of current researches merely focus on improving the efficiency performance of autonomous vehicles(e.g. the energy management system, the handling, etc.)instead of putting the human acceptance and preference into consideration, leaving the knowledge gap of achieving the personalised automation. The primary objective of this research is to develop a novel human-like trajectory planning algorithm that is able to mimic the performance of human drivers and generate a feasible trajectory for an autonomous vehicle to complete a motorway lane change, which is the most representative and commonest manoeuvre on the motorway. This thesis can be divided into four main sections. Starting with the part of literature review, which summarises the existing techniques and the associated knowledges that can be taken the advantage of; including the trajectory planning, the driving styles, the lane change manoeuvre and the Model Predictive Control (MPC). An appropriate-designed experiment is then introduced and implemented, with the purpose of constructing a precise and reliable human driving database. This database contains 551 lane changes on the motorway from 12 different male drivers. Through applying data statistics methods, the human characteristics can be mined from the experimental data, showing that the vehicle velocity , the hand steering wheel angle , the longitudinal acceleration a, the rate of hand steering and the rate of longitudinal accelerating are the essential features for the motorway lane change manoeuvre. An off-line constraint table for the three nominated driving styles can be therefore constructed based on these features. Finally, the obtained human information is then fused with the traditional MPC planning technique so as to achieve the proposed human-like trajectory planning algorithm. The main contribution of this study is proposing a novel approach of combining the real human driving data and the traditional planning technique(i.e. MPC) to achieve human-like lane change trajectory planning for autonomous vehicles. An integrated human driving database which contains both the video footages and the vehicle-dynamic-based signals from 12 different participants is built. Moreover, the draft marginal values of the essential parameters for the driving styles while performing a right lane change on the motorway are also presented. Both the collected driving database and the driving styles’ constraint table can be seen as distinctive achievements, providing resourceful materials for future researches.PhD in Transport System

Road geometry identification with mobile mapping techniques

Durante il mio dottorato mi sono occupato di Tecniche e Tecnologie innovative per la ricostruzione della geometria dei tracciati stradali esistenti, quali ad esempio Mobile Mapping, analisi immagini e dati GIS; a fronte degli elevatissimi costi oggi richiesti per l’utilizzo di veicoli strumentati già reperibili in commercio per il raggiungimento di tali scopi, il valore aggiunto del lavoro di dottorato riguarda l’uso di strumenti a basso costo che comportano un rilevante lavoro di analisi, trattamento e correzione del dato che risente in maniera decisiva della medio/bassa qualità della strumentazione in uso. L’obiettivo della ricerca è consistito nella realizzazione di un algoritmo di riconoscimento (in ambiente MATLAB) che sia in grado di restituire la geometria as-built di una strada esistente. Parte del lavoro è stata svolta nell’analisi e nell’estrazione delle curvature locali con approcci differenti (successive circonferenze locali, funzioni polinomiali di fitting locale di vario grado e con ampiezza di analisi variabile), nonché sullo studio degli angoli di deviazione locali. Usando questi parametri, nel resto del lavoro, si è prima ricercata una metodologia d’identificazione dei diversi elementi che compongono la geometria stradale, e poi si è lavorato su procedure di fitting con svariate tecniche (minimi quadrati, metodi robusti e altri algoritmi) cercando di estrarre informazioni di carattere geometrico, quali raggi di curvatura e relativi centri, lunghezza e orientamento dei rettifili, fattori di scala delle curve di transizione

Authority-Sharing Control of Assistive Robotic Walkers

A recognized consequence of population aging is a reduced level of mobility, which undermines the life quality of several senior citizens. A promising solution is represented by assisitive robotic walkers, combining the benefits of standard walkers (improved stability and physical support) with sensing and computing ability to guarantee cognitive support. In this context, classical robot control strategies designed for fully autonomous systems (such as fully autonomous vehicles, where the user is excluded from the loop) are clearly not suitable, since the user’s residual abilities must be exploited and practiced. Conversely, to guarantee safety even in the presence of user’s cognitive deficits, the responsibility of controlling the vehicle motion cannot be entirely left to the assisted person. The authority-sharing paradigm, where the control authority, i.e., the capability of controlling the vehicle motion, is shared between the human user and the control system, is a promising solution to this problem. This research develops control strategies for assistive robotic walkers based on authority-sharing: this way, we ensure that the walker provides the user only the help he/she needs for safe navigation. For instance, if the user requires just physical support to reach the restrooms, the robot acts as a standard rollator; however, if the user’s cognitive abilities are limited (e.g., the user does not remember where the restrooms are, or he/she does not recognize obstacles on the path), the robot also drives the user towards the proper corridors, by planning and following a safe path to the restrooms. The authority is allocated on the basis of an error metric, quantifying the distance between the current vehicle heading and the desired movement direction to perform the task. If the user is safely performing the task, he/she is endowed with control authority, so that his/her residual abilities are exploited. Conversely, if the user is not capable of safely solving the task (for instance, he/is going to collide with an obstacle), the robot intervenes by partially or totally taking the control authority to help the user and ensure his/her safety (for instance, avoiding the collision). We provide detailed control design and theoretical and simulative analyses of the proposed strategies. Moreover, extensive experimental validation shows that authority-sharing is a successful approach to guide a senior citizen, providing both comfort and safety. The most promising solutions include the use of haptic systems to suggest the user a proper behavior, and the modification of the perceived physical interaction of the user with the robot to gradually share the control authority using a variable stiffness vehicle handling

Lane-Precise Localization with Production Vehicle Sensors and Application to Augmented Reality Navigation

This works describes an approach to lane-precise localization on current digital maps. A particle filter fuses data from production vehicle sensors, such as GPS, radar, and camera. Performance evaluations on more than 200 km of data show that the proposed algorithm can reliably determine the current lane. Furthermore, a possible architecture for an intuitive route guidance system based on Augmented Reality is proposed together with a lane-change recommendation for unclear situations

Actuators for Intelligent Electric Vehicles

This book details the advanced actuators for IEVs and the control algorithm design. In the actuator design, the configuration four-wheel independent drive/steering electric vehicles is reviewed. An in-wheel two-speed AMT with selectable one-way clutch is designed for IEV. Considering uncertainties, the optimization design for the planetary gear train of IEV is conducted. An electric power steering system is designed for IEV. In addition, advanced control algorithms are proposed in favour of active safety improvement. A supervision mechanism is applied to the segment drift control of autonomous driving. Double super-resolution network is used to design the intelligent driving algorithm. Torque distribution control technology and four-wheel steering technology are utilized for path tracking and adaptive cruise control. To advance the control accuracy, advanced estimation algorithms are studied in this book. The tyre-road peak friction coefficient under full slip rate range is identified based on the normalized tyre model. The pressure of the electro-hydraulic brake system is estimated based on signal fusion. Besides, a multi-semantic driver behaviour recognition model of autonomous vehicles is designed using confidence fusion mechanism. Moreover, a mono-vision based lateral localization system of low-cost autonomous vehicles is proposed with deep learning curb detection. To sum up, the discussed advanced actuators, control and estimation algorithms are beneficial to the active safety improvement of IEVs