Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

This paper was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the work, We propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website based on LFRL

Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learning

We present an approach for mobile robots to learn to navigate in dynamic environments with pedestrians via raw depth inputs, in a socially compliant manner. To achieve this, we adopt a generative adversarial imitation learning (GAIL) strategy, which improves upon a pre-trained behavior cloning policy. Our approach overcomes the disadvantages of previous methods, as they heavily depend on the full knowledge of the location and velocity information of nearby pedestrians, which not only requires specific sensors, but also the extraction of such state information from raw sensory input could consume much computation time. In this paper, our proposed GAIL-based model performs directly on raw depth inputs and plans in real-time. Experiments show that our GAIL-based approach greatly improves the safety and efficiency of the behavior of mobile robots from pure behavior cloning. The real-world deployment also shows that our method is capable of guiding autonomous vehicles to navigate in a socially compliant manner directly through raw depth inputs. In addition, we release a simulation plugin for modeling pedestrian behaviors based on the social force model.Comment: ICRA 2018 camera-ready version. 7 pages, video link: https://www.youtube.com/watch?v=0hw0GD3lkA

Informative Path Planning for Active Field Mapping under Localization Uncertainty

Information gathering algorithms play a key role in unlocking the potential of robots for efficient data collection in a wide range of applications. However, most existing strategies neglect the fundamental problem of the robot pose uncertainty, which is an implicit requirement for creating robust, high-quality maps. To address this issue, we introduce an informative planning framework for active mapping that explicitly accounts for the pose uncertainty in both the mapping and planning tasks. Our strategy exploits a Gaussian Process (GP) model to capture a target environmental field given the uncertainty on its inputs. For planning, we formulate a new utility function that couples the localization and field mapping objectives in GP-based mapping scenarios in a principled way, without relying on any manually tuned parameters. Extensive simulations show that our approach outperforms existing strategies, with reductions in mean pose uncertainty and map error. We also present a proof of concept in an indoor temperature mapping scenario.Comment: 8 pages, 7 figures, submission (revised) to Robotics & Automation Letters (and IEEE International Conference on Robotics and Automation

Integrating Remote-Controlled Robotics with Standalone 5G Networks in Industry 4.0

Denne masteroppgaven undersøker integrasjonen av fjernstyrt robotikk med standalone 5G-nettverk i en Industry 4.0-kontekst. Hovedmålene er å kartlegge og få innsikt om frittstående 5G-nettverk i en industriell lab, og effektivisere prosessen med å sette opp et proof-of-concept produksjonsmiljø som inkluderer bruk av automatisert robotikk. Studien innebærer å teste et privat standalone 5G-nettverk i NTNUs Manulab og analysere nettverksytelsesmålinger som forsinkelse, hastighet og signaldekning. Forskningen identifiserte flere utfordringer, inkludert nettverksvariabilitet, utstyrsforsinkelser og fravær av 5G core slicing, noe som påvirket muligheten til å oppnå lav forsinkelse og høy pålitelighet. Til tross for disse problemene ga testene verdifulle data om nettverksytelse og signaldekning i industrielle omgivelser. Funnene indikerer at 5G-teknologien som er testet foreløpig ikke er egnet for industrielle omgivelser. Imidlertid kan det bli egnet hvis nettverket bruker nettverksslicing og mmWave-teknologi.This master's thesis investigates the integration of remote-controlled robotics with standalone 5G networks in an Industry 4.0 context. The main objectives are to map and capture key insights about the standalone 5G network in an industrial lab and to update and streamline the process of setting up a proof-of-concept industrial production environment using automated robotics. The study involved testing a private standalone 5G network in NTNU's Manulab and analyzing network performance metrics such as latency, speed, and signal coverage. The research identifies several challenges, including network variability, equipment delays, and the absence of 5G core slicing, which affected the ability to achieve low latency and high reliability. Despite these issues, the tests provided valuable data on network performance and signal coverage in an industrial setting. The findings indicate that the 5G technology tested is currently not suitable for industrial settings. However, it could become suitable if the network utilizes network slicing and mmWave technology