Search-based Test Generation for Automated Driving Systems: From Perception to Control Logic

abstract: Automated driving systems are in an intensive research and development stage, and the companies developing these systems are targeting to deploy them on public roads in a very near future. Guaranteeing safe operation of these systems is crucial as they are planned to carry passengers and share the road with other vehicles and pedestrians. Yet, there is no agreed-upon approach on how and in what detail those systems should be tested. Different organizations have different testing approaches, and one common approach is to combine simulation-based testing with real-world driving. One of the expectations from fully-automated vehicles is never to cause an accident. However, an automated vehicle may not be able to avoid all collisions, e.g., the collisions caused by other road occupants. Hence, it is important for the system designers to understand the boundary case scenarios where an autonomous vehicle can no longer avoid a collision. Besides safety, there are other expectations from automated vehicles such as comfortable driving and minimal fuel consumption. All safety and functional expectations from an automated driving system should be captured with a set of system requirements. It is challenging to create requirements that are unambiguous and usable for the design, testing, and evaluation of automated driving systems. Another challenge is to define useful metrics for assessing the testing quality because in general, it is impossible to test every possible scenario. The goal of this dissertation is to formalize the theory for testing automated vehicles. Various methods for automatic test generation for automated-driving systems in simulation environments are presented and compared. The contributions presented in this dissertation include (i) new metrics that can be used to discover the boundary cases between safe and unsafe driving conditions, (ii) a new approach that combines combinatorial testing and optimization-guided test generation methods, (iii) approaches that utilize global optimization methods and random exploration to generate critical vehicle and pedestrian trajectories for testing purposes, (iv) a publicly-available simulation-based automated vehicle testing framework that enables application of the existing testing approaches in the literature, including the new approaches presented in this dissertation.Dissertation/ThesisDoctoral Dissertation Computer Engineering 201

An augmented reality interface for multi-robot tele-operation and control

This thesis presents a seamlessly controlled human multi-robot system comprised of ground and aerial robots of semi-autonomous nature for source localization tasks. The system combines augmented reality interfaces capabilities with human supervisor\u27s ability to control multiple robots. It used advanced path planning algorithms to ensure obstacles are avoided and that the operators are free for higher-level tasks. A sensor data fused AR view is displayed which helped the users pin point source information or help the operator with the goals of the mission. The paper studies a preliminary Human Factors evaluation of this system in which several interface conditions are tested for source detection tasks

Visibility in underwater robotics: Benchmarking and single image dehazing

Dealing with underwater visibility is one of the most important challenges in autonomous underwater robotics. The light transmission in the water medium degrades images making the interpretation of the scene difficult and consequently compromising the whole intervention. This thesis contributes by analysing the impact of the underwater image degradation in commonly used vision algorithms through benchmarking. An online framework for underwater research that makes possible to analyse results under different conditions is presented. Finally, motivated by the results of experimentation with the developed framework, a deep learning solution is proposed capable of dehazing a degraded image in real time restoring the original colors of the image.Una de las dificultades más grandes de la robótica autónoma submarina es lidiar con la falta de visibilidad en imágenes submarinas. La transmisión de la luz en el agua degrada las imágenes dificultando el reconocimiento de objetos y en consecuencia la intervención. Ésta tesis se centra en el análisis del impacto de la degradación de las imágenes submarinas en algoritmos de visión a través de benchmarking, desarrollando un entorno de trabajo en la nube que permite analizar los resultados bajo diferentes condiciones. Teniendo en cuenta los resultados obtenidos con este entorno, se proponen métodos basados en técnicas de aprendizaje profundo para mitigar el impacto de la degradación de las imágenes en tiempo real introduciendo un paso previo que permita recuperar los colores originales

Robots learn to behave: improving human-robot collaboration in flexible manufacturing applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

Exploring Natural User Abstractions For Shared Perceptual Manipulator Task Modeling & Recovery

State-of-the-art domestic robot assistants are essentially autonomous mobile manipulators capable of exerting human-scale precision grasps. To maximize utility and economy, non-technical end-users would need to be nearly as efficient as trained roboticists in control and collaboration of manipulation task behaviors. However, it remains a significant challenge given that many WIMP-style tools require superficial proficiency in robotics, 3D graphics, and computer science for rapid task modeling and recovery. But research on robot-centric collaboration has garnered momentum in recent years; robots are now planning in partially observable environments that maintain geometries and semantic maps, presenting opportunities for non-experts to cooperatively control task behavior with autonomous-planning agents exploiting the knowledge. However, as autonomous systems are not immune to errors under perceptual difficulty, a human-in-the-loop is needed to bias autonomous-planning towards recovery conditions that resume the task and avoid similar errors. In this work, we explore interactive techniques allowing non-technical users to model task behaviors and perceive cooperatively with a service robot under robot-centric collaboration. We evaluate stylus and touch modalities that users can intuitively and effectively convey natural abstractions of high-level tasks, semantic revisions, and geometries about the world. Experiments are conducted with \u27pick-and-place\u27 tasks in an ideal \u27Blocks World\u27 environment using a Kinova JACO six degree-of-freedom manipulator. Possibilities for the architecture and interface are demonstrated with the following features; (1) Semantic \u27Object\u27 and \u27Location\u27 grounding that describe function and ambiguous geometries (2) Task specification with an unordered list of goal predicates, and (3) Guiding task recovery with implied scene geometries and trajectory via symmetry cues and configuration space abstraction. Empirical results from four user studies show our interface was much preferred than the control condition, demonstrating high learnability and ease-of-use that enable our non-technical participants to model complex tasks, provide effective recovery assistance, and teleoperative control

Safe navigation and human-robot interaction in assistant robotic applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

Learning Motion Skills for a Humanoid Robot

This thesis investigates the learning of motion skills for humanoid robots. As groundwork, a humanoid robot with integrated fall management was developed as an experimental platform. Then, two different approaches for creating motion skills were investigated. First, one that is based on Cartesian quintic splines with optimized parameters. Second, a reinforcement learning-based approach that utilizes the first approach as a reference motion to guide the learning. Both approaches were tested on the developed robot and on further simulated robots to show their generalization. A special focus was set on the locomotion skill, but a standing-up and kick skill are also discussed. Diese Dissertation beschäftigt sich mit dem Lernen von Bewegungsfähigkeiten für humanoide Roboter. Als Grundlage wurde zunächst ein humanoider Roboter mit integriertem Fall Management entwickelt, welcher als Experimentalplatform dient. Dann wurden zwei verschiedene Ansätze für die Erstellung von Bewegungsfähigkeiten untersucht. Zu erst einer der kartesische quintische Splines mit optimierten Parametern nutzt. Danach wurde ein Ansatz basierend auf bestärkendem Lernen untersucht, welcher den ersten Ansatz als Referenzbewegung benutzt. Beide Ansätze wurden sowohl auf der entwickelten Roboterplatform, als auch auf weiteren simulierten Robotern getestet um die Generalisierbarkeit zu zeigen. Ein besonderer Fokus wurde auf die Fähigkeit des Gehens gelegt, aber auch Aufsteh- und Schussfähigkeiten werden diskutiert

Algorithms for Modular Self-reconfigurable Robots: Decision Making, Planning, and Learning

Modular self-reconfigurable robots (MSRs) are composed of multiple robotic modules which can change their connections with each other to take different shapes, commonly known as configurations. Forming different configurations helps the MSR to accomplish different types of tasks in different environments. In this dissertation, we study three different problems in MSRs: partitioning of modules, configuration formation planning and locomotion learning, and we propose algorithmic solutions to solve these problems. Partitioning of modules is a decision-making problem for MSRs where each module decides which partition or team of modules it should be in. To find the best set of partitions is a NP-complete problem. We propose game theory based both centralized and distributed solutions to solve this problem. Once the modules know which set of modules they should team-up with, they self-aggregate to form a specific shaped configuration, known as the configuration formation planning problem. Modules can be either singletons or connected in smaller configurations from which they need to form the target configuration. The configuration formation problem is difficult as multiple modules may select the same location in the target configuration to move to which might result in occlusion and consequently failure of the configuration formation process. On the other hand, if the modules are already in connected configurations in the beginning, then it would be beneficial to preserve those initial configurations for placing them into the target configuration as disconnections and re-connections are costly operations. We propose solutions based on an auction-like algorithm and (sub) graph-isomorphism technique to solve the configuration formation problem. Once the configuration is built, the MSR needs to move towards its goal location as a whole configuration for completing its task. If the configuration’s shape and size is not known a priori, then planning its locomotion is a difficult task as it needs to learn the locomotion pattern in dynamic time – the problem is known as adaptive locomotion learning. We have proposed reinforcement learning based fault-tolerant solutions for locomotion learning by MSRs

Ambiente de simulação para agentes em futebol robótico

Mestrado em Engenharia de Computadores e TelemáticaO teste de algoritmos na área da robótica pode ser uma tarefa difícil, especialmente se o teste envolver múltipos robots. Neste contexto o uso de um simulador torna-se uma ferramenta importante no teste de algoritmos pois permite ultrapassar algumas limitações e oferece várias vantagens. CAMBADA é a equipa de futebol robótico da liga de tamanho médio da Universidade de Aveiro, Portugal. A equipa está familiarizada com as limitações do uso de robots reais para o teste de algoritmos. Devido a isso o simulador criado pela equipa Brainstormers Tribots foi adaptado para prover um ambiente de simulação ao software CAMBADA e estava em uso aquando do início desta dissertação. O simulador oferecia pouca flexibilidade na modelação dos robots que resultava em comportamentos imprecisos, oferecia também reduzida interacção com a simulação. O objectivo desta dissertação é criar um ambiente de simulação para agentes em futebol robótico com a intenção de melhorar o ambiente de simulação da equipa CAMBADA. O simulador deve ser capaz de simular dinâmica de objectos a três dimensões, sensores e actuadores ao mesmo tempo que oferece visualização do mundo e a possibilidade de interagir com a simulação. Da pesquisa realizada sobre simuladores robóticos o simulador Gazebo respeitava os nossos requisitos e foi escolhido para código base do nosso simulador. Para criar um ambiente simulado adequado à equipa CAMBADA alguns componentes do Gazebo foram alterados e novos sensores e actuadores virtuais foram desenvolvidos. Vários componentes do software CAMBADA tiveram que sofrer alterações de modo a suportar um ambiente simulado. O robot virtual foi modelado de modo a assemelhar-se com o robot real com o objectivo de obter comportamentos mais precisos. O simulador desenvolvido substituiu a solução anteriormente criada pela equipa CAMBADA e foi usado nos testes de preparação para a participação da equipa no RoboCup 2010 em Singapura onde deu o seu contributo na obtenção do terceiro lugar.In the field of robotics, testing algorithms with the real robots can be a di cult task, specially if the test involves more than one robot. In this context a simulator is an important tool for testing algorithms because it helps overcome some limitation and o ers several advantages. CAMBADA is the RoboCup MSL soccer team of the University of Aveiro, Portugal. The team is familiar with the limitations of using the real robots for testing algorithms. Therefore, a simulator created by the Brainstormers Tribots team was adapted to provide a simulated environment for their software and was used for testing at the time of the beginning of this thesis. The simulator offered low flexibility on the modeling of the robots from which resulted inaccurate behaviors, it also o ered reduced interaction with the simulation. The purpose of this thesis is to create a simulation environment for robotic soccer agents with the intention of improving the simulated environment for the CAMBADA team. The simulation must provide three-dimensional dynamics of objects, be capable of simulating sensors and actuators, allow the visualization of the simulation and provide interaction with the simulation. From the conducted survey about robotic simulators, the simulator Gazebo complied with our requirements and was chosen to provide the code base for our simulator. To create an adequate simulation environment for the CAMBADA team some components of Gazebo were modi ed and new sensors and actuator were developed. Several components of the CAMBADA software had to be modified to support the simulated environment. The virtual robot was modeled to resemble the real robot to provide more accurate behaviors. The developed simulator substituted the previous solution created by CAMBADA team and was used in the preparation tests for the participation in the RoboCup 2010 in Singapore where it contributed to obtain of the third-place

An approach to simulation of autonomous vehicles

Tese de mestrado integrado. Engenharia Electrotécnica e de Computadores (Major de Telecomunicações). Faculdade de Engenharia. Universidade do Porto. 200