From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Driving

Simulation is an integral part in the process of developing autonomous vehicles and advantageous for training, validation, and verification of driving functions. Even though simulations come with a series of benefits compared to real-world experiments, various challenges still prevent virtual testing from entirely replacing physical test-drives. Our work provides an overview of these challenges with regard to different aspects and types of simulation and subsumes current trends to overcome them. We cover aspects around perception-, behavior- and content-realism as well as general hurdles in the domain of simulation. Among others, we observe a trend of data-driven, generative approaches and high-fidelity data synthesis to increasingly replace model-based simulation.Comment: Ferdinand M\"utsch, Helen Gremmelmaier, and Nicolas Becker contributed equally. Accepted for publication at CVPR 2023 VCAD worksho

Vision-Based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

Scene understanding plays a crucial role in autonomous driving by utilizing sensory data for contextual information extraction and decision making. Beyond modeling advances, the enabler for vehicles to become aware of their surroundings is the availability of visual sensory data, which expand the vehicular perception and realizes vehicular contextual awareness in real-world environments. Research directions for scene understanding pursued by related studies include person/vehicle detection and segmentation, their transition analysis, lane change, and turns detection, among many others Unfortunately, these tasks seem insufficient to completely develop fully-autonomous vehicles i.e. achieving level-5 autonomy, travelling just like human-controlled cars. This latter statement is among the conclusions drawn from this review paper: scene understanding for autonomous driving cars using vision sensors still requires significant improvements. With this motivation, this survey defines, analyzes, and reviews the current achievements of the scene understanding research area that mostly rely on computationally complex deep learning models. Furthermore, it covers the generic scene understanding pipeline, investigates the performance reported by the state-of-the-art, informs about the time complexity analysis of avant garde modeling choices, and highlights major triumphs and noted limitations encountered by current research efforts. The survey also includes a comprehensive discussion on the available datasets, and the challenges that, even if lately confronted by researchers, still remain open to date. Finally, our work outlines future research directions to welcome researchers and practitioners to this exciting domain.This work was supported by the European Commission through European Union (EU) and Japan for Artificial Intelligence (AI) under Grant 957339

A Machine Learning Approach for Predicting Docking-Based Structures

A indústria marítima encontra-se ainda numa fase inicial relativamente à automatização de processos e integração da Inteligência Artificial (AI). O desenvolvimento de Veículos Autónomos de Superfície (ASVs) insere-se neste âmbito, trazendo a possibilidade de reduzir drasticamente a intervenção humana nesta indústria e as consequências inerentes ao erro humano. A investigação no contexto da operação deste tipo de veículos tem crescido recentemente, apesar de a sua utilização em larga escala ainda encontrar enúmeras limitações, tais como a necessidade de assegurar a segurança do veículo, intervenientes e infra-estruturas, e de desenvolver a tecnologia apropriada. O processo de atracagem encontra-se entre as operações mais complexas e exigentes para um veículo marítimo. De facto, a maioria dos acidentes marítimos ocorre em portos. Tendo em conta esta tendência, assim como a diminuta investigação científica acerca do tema que se verifica atualmente, o processo de atracagem de um veículo autónomo mostra-se como uma problemática interessante e promissora. Neste sentido, esta dissertação pretende contribuir para o processo de tornar um veículo marítimo capaz de atracar autonomamente, através do desenvolvimento de uma ferramenta de perceção necessária para esta manobra. O trabalho realizado consiste numa rede de Deep Learning (DL) capaz de detetar a presença de uma doca no meio circundante do veículo bem como de classificar a estrutura identificada. Considerando que não existe muita informação referente a este contexto, tornou-se necessário proceder a uma recolha de dados no simulador 3D Gazebo, durante a qual cinco modelos de diferentes docas foram colocados num ambiente marítimo simulado e foi capturada informação de vários sensores, como um LiDAR e um par de câmaras stereo. Os dados recolhidos foram usados para o treino do classificador, que é composto por duas redes em cascata: uma para detetar a existência de uma doca na vizinhança do veículo e outra para classificar o tipo da estrutura. Esta dissertação também propõe um algorithmo de análise da ocupação da doca, a partir de uma abordagem baseada em template matching. O trabalho desenvolvido foi testado nos dados recolhidos, assim como numa configuração mais dinâmica no simulador mencionado acima, tendo em consideração diferentes condições de ruído para melhor replicar um ambiente marítimo autêntico. O modelo de deteção obteve uma precisão de 96.44% em condições ótimas e uma precisão média de 90.43% na análise do efeito de ruído, com um desvio máximo de 2.8%. O modelo de classificação do tipo de doca obteve uma precisão máxima de 86.70% e média de 80.90% nos testes de ruído. O algoritmo de análise da ocupação permite determinar o número de vagas da estrutura assim como as respetivas coordenadas. Este trabalho foi desenvolvido com uma abordagem que permite facilmente a adaptação a outros tipos de docas, diferenciando-se, assim, de outros projetos da área.The maritime industry has only recently taken its first steps towards automation and Artificial Intelligence (AI). The development of Autonomous Surface Vehicles (ASVs) falls under this scope, uncovering the possibility of greatly reducing the role of human intervention in this industry and the inherent consequences of human error. Research concerning the operation of this type of vehicles has seen an upward trend in recent years, although its full-scale application still encounters various limitations, such as the need to ensure safety and develop the required technology. The docking and undocking processes are among the most challenging tasks for a maritime vehicle. In fact, most maritime accidents occur in sea ports and harbours. Such evidence, along with the scarceness of scientific endeavours in this topic, make the docking approach of an ASV an alluring matter to delve into. With this intention in mind, this dissertation aims to take one step further towards enabling a vessel to dock autonomously, by providing a perception tool necessary for this maneuver. The developed work comprises a Deep Learning (DL) network able to detect the existence of a docking platform in the vehicle's surrounding environment and classify the type of the structure. As data concerning this context is not widely available, it was necessary to conduct a data acquisition process in the 3D simulator Gazebo, in which models of five different types of docks were placed in a simulated maritime environment and data from several sensors, such as a LiDAR and a stereo camera, was captured. The gathered data was used for the training of the cascaded classifier, which was composed of two networks: a first one to ascertain whether there is a dock in the vicinity of the vehicle and a second one to categorize the structure. This dissertation also proposes a mechanism to perform an occupation analysis of the dock, based on a template matching approach. The developed work was tested on the gathered data and in a dynamic setup in the aforementioned simulator, taking into account different noise conditions to better replicate an authentic maritime environment. The detection model achieved an accuracy of 96.44% in optimal conditions and an average of 90.43% considering light to very severe noise conditions, with a deviation of 2.8% for the worst case. The categorizing model obtained a maximum accuracy of 86.70% and an average accuracy of 80.90% for noise tests. The occupation analysis algorithm is able to return the number and coordinates of the vacant spots of the dock. Both the cascade classifier and the occupation tool were developed through an approach that allows an easy adaptation to other types of docks, thus differentiating it from other works on the topic

Vision-based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

Scene understanding plays a crucial role in autonomous driving by utilizing sensory data for contextual information extraction and decision making. Beyond modeling advances, the enabler for vehicles to become aware of their surroundings is the availability of visual sensory data, which expand the vehicular perception and realizes vehicular contextual awareness in real-world environments. Research directions for scene understanding pursued by related studies include person/vehicle detection and segmentation, their transition analysis, lane change, and turns detection, among many others Unfortunately, these tasks seem insufficient to completely develop fully-autonomous vehicles i.e. achieving level-5 autonomy, travelling just like human-controlled cars. This latter statement is among the conclusions drawn from this review paper: scene understanding for autonomous driving cars using vision sensors still requires significant improvements. With this motivation, this survey defines, analyzes, and reviews the current achievements of the scene understanding research area that mostly rely on computationally complex deep learning models. Furthermore, it covers the generic scene understanding pipeline, investigates the performance reported by the state-of-the-art, informs about the time complexity analysis of avant garde modeling choices, and highlights major triumphs and noted limitations encountered by current research efforts. The survey also includes a comprehensive discussion on the available datasets, and the challenges that, even if lately confronted by researchers, still remain open to date. Finally, our work outlines future research directions to welcome researchers and practitioners to this exciting domain.This work was supported by the European Commission through European Union (EU) and Japan for Artificial Intelligence (AI) under Grant 957339

Protective Behavior Detection in Chronic Pain Rehabilitation: From Data Preprocessing to Learning Model

Chronic pain (CP) rehabilitation extends beyond physiotherapist-directed clinical sessions and primarily functions in people's everyday lives. Unfortunately, self-directed rehabilitation is difficult because patients need to deal with both their pain and the mental barriers that pain imposes on routine functional activities. Physiotherapists adjust patients' exercise plans and advice in clinical sessions based on the amount of protective behavior (i.e., a sign of anxiety about movement) displayed by the patient. The goal of such modifications is to assist patients in overcoming their fears and maintaining physical functioning. Unfortunately, physiotherapists' support is absent during self-directed rehabilitation or also called self-management that people conduct in their daily life. To be effective, technology for chronic-pain self-management should be able to detect protective behavior to facilitate personalized support. Thereon, this thesis addresses the key challenges of ubiquitous automatic protective behavior detection (PBD). Our investigation takes advantage of an available dataset (EmoPain) containing movement and muscle activity data of healthy people and people with CP engaged in typical everyday activities. To begin, we examine the data augmentation methods and segmentation parameters using various vanilla neural networks in order to enable activity-independent PBD within pre-segmented activity instances. Second, by incorporating temporal and bodily attention mechanisms, we improve PBD performance and support theoretical/clinical understanding of protective behavior that the attention of a person with CP shifts between body parts perceived as risky during feared movements. Third, we use human activity recognition (HAR) to improve continuous PBD in data of various activity types. The approaches proposed above are validated against the ground truth established by majority voting from expert annotators. Unfortunately, using such majority-voted ground truth causes information loss, whereas direct learning from all annotators is vulnerable to noise from disagreements. As the final study, we improve the learning from multiple annotators by leveraging the agreement information for regularization

Segmentació Semàntica a la Representació Latent

This project proposes a method to merge image compression and semantic segmentation, in a single stage, for the foreground/background segmentation approach. This binary segmentation is based on the case of person segmentation, whereby the foreground corresponds to each person in the image and the background is everything else. The proposed method is analysed and compared with an end-to-end compression model followed by a semantic segmentation stage, as well as with the results obtained from uncompressed image segmentation. The CompressAI cheng2020-anchor model and the Pyramid Scene Parsing Network (PSPNet), implemented through the Semseg repository, have been used to develop this approach. The results obtained by the proposed merging method outperform those obtained by the end-to-end compression model followed by the semantic segmentation stage for low bitrates.Este proyecto propone un método para fusionar la compresión de imágenes y la segmentación semántica, en una sola etapa, para el enfoque de segmentación primer plano/fondo. Esta segmentación binaria se basa en el caso de la segmentación de personas, según la cual el primer plano corresponde a cada persona de la imagen y el fondo es todo lo demás. El método propuesto se analiza y compara con un modelo de compresión de extremo a extremo seguido de una etapa de segmentación semántica, así como con los resultados obtenidos de la segmentación de imágenes sin comprimir. Para desarrollar este método se ha utilizado el modelo cheng2020-anchor de CompressAI y la Pyramid Scene Parsing Network (PSPNet), implementada a través del repositorio Semseg. Los resultados obtenidos por el método de fusión propuesto superan a los obtenidos por el modelo de compresión de extremo a extremo seguido de la etapa de segmentación semántica para tasas de bits bajas.Aquest projecte proposa un mètode per a fusionar la compressió d'imatges i la segmentació semàntica, en una sola etapa, per a l'enfocament de segmentació primer pla/fons. Aquesta segmentació binària es basa en el cas de la segmentació de persones, segons la qual el primer pla correspon a cada persona de la imatge i el fons és tota la resta. El mètode proposat s'analitza i compara amb un model de compressió d'extrem a extrem seguit d'una etapa de segmentació semàntica, així com amb els resultats obtinguts de la segmentació d'imatges sense comprimir. Per a desenvolupar aquest mètode s'ha utilitzat el model cheng2020-*anchor de CompressAI i la Pyramid Scene Parsing Network (PSPNet), implementada a través del repositori Semseg. Els resultats obtinguts pel mètode de fusió proposat superen als obtinguts pel model de compressió d'extrem a extrem seguit de l'etapa de segmentació semàntica per a taxes de bits baixes

The Future of Humanoid Robots

This book provides state of the art scientific and engineering research findings and developments in the field of humanoid robotics and its applications. It is expected that humanoids will change the way we interact with machines, and will have the ability to blend perfectly into an environment already designed for humans. The book contains chapters that aim to discover the future abilities of humanoid robots by presenting a variety of integrated research in various scientific and engineering fields, such as locomotion, perception, adaptive behavior, human-robot interaction, neuroscience and machine learning. The book is designed to be accessible and practical, with an emphasis on useful information to those working in the fields of robotics, cognitive science, artificial intelligence, computational methods and other fields of science directly or indirectly related to the development and usage of future humanoid robots. The editor of the book has extensive R&D experience, patents, and publications in the area of humanoid robotics, and his experience is reflected in editing the content of the book

Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries

This two-volume set LNCS 12962 and 12963 constitutes the thoroughly refereed proceedings of the 7th International MICCAI Brainlesion Workshop, BrainLes 2021, as well as the RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge, the Federated Tumor Segmentation (FeTS) Challenge, the Cross-Modality Domain Adaptation (CrossMoDA) Challenge, and the challenge on Quantification of Uncertainties in Biomedical Image Quantification (QUBIQ). These were held jointly at the 23rd Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2020, in September 2021. The 91 revised papers presented in these volumes were selected form 151 submissions. Due to COVID-19 pandemic the conference was held virtually. This is an open access book