Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

This paper was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the work, We propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website based on LFRL

Improving Makespan in Dynamic Task Scheculing for Cloud Robotic Systems with Time Window Constraints

A scheduling method in a robotic network cloud system with minimal makespan is beneficial as the system can complete all the tasks assigned to it in the fastest way. Robotic network cloud systems can be translated into graphs where nodes represent hardware with independent computing power and edges represent data transmissions between nodes. Time-window constraints on tasks are a natural way to order tasks. The makespan is the maximum amount of time between when a node starts executing its first scheduled task and when all nodes have completed their last scheduled task. Load balancing allocation and scheduling ensures that the time between when the first node completes its scheduled tasks and when all other nodes complete their scheduled tasks is as short as possible. We propose a new load balancing algorithm for task allocation and scheduling with minimal makespan. We theoretically prove the correctness of the proposed algorithm and present simulations illustrating the obtained results.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

A plug and play transparent communication layer for cloud robotics architectures

The cloud robotics paradigm aims at enhancing the abilities of robots by using cloud services, but it still poses several challenges in the research community. Most of the current literature focuses on how to enrich specific robotic capabilities, overlooking how to effectively establish communication between the two fields. Our work proposes a “plug-and-play” solution to bridge the communication gap between cloud and robotic applications. The proposed solution is designed based on the mature WebSocket technology and it can be extended to any ROS-based robotic platform. The main contributions of this work are the definition of a reliable autoconnection/autoconfiguration mechanism as well as to outline a scalable communication layer that allows the effective control of multiple robots from multiple users. The “plug-and-play” solution was evaluated in both simulated and real scenarios. In the first case, the presence of users and robots was simulated with Robot Operating System (ROS) nodes running on five machines. In the real scenario, three non-expert users teleoperated, simultaneously, three remote robots by using the proposed communication layer with different networking protocols. Results confirmed the reliability at different levels: at startup (success_rate = 100%); during high-rate communications (message_lost = 0%); in performing open-loop spiral trajectories with enhancement, with respect to similar works; and in the quality of simultaneous teleoperations

Networked Heterogeneous Systems in a ROS-Enabled Cloud Environment

It is important in the development of cloud robotics that the challenges presented by transferring computational loads to networked resources are properly addressed. The challenges include network latency, data integrity, security, and privacy. The objective of the present work is to investigate the issues of latency and data integrity in a representative cloud robotics environment. The present work involves setting up a cloud robotics network in an open-source Robot Operating System (ROS) framework and carrying out investigations on the levels of latency and reduction in data integrity as utilization of the network increases. In this study, a virtual datacenter has been set up to provide the foundation on which to build software systems to provide cloud services. Robot Operating System (ROS) framework has been used to facilitate communication among heterogeneous systems in the network. Three types of robots, including the Parrot AR.Drone2.0, the Kobuki Turtlebot 2, and the LEGO EV3 have been implemented in the system. The system has been tested for baseline connectivity and under low- and high-bandwidth conditions to determine the latency and data integrity of the network connections. Additionally, a heterogeneous system consisting of sensor feedback from the AR.Drone2.0 and motor control of the Turtlebot 2 has been built to examine the connection between the devices themselves. Through this study, it has been demonstrated that under low-bandwidth conditions, the network performs reasonably well in the areas of latency and data integrity. However, for high-bandwidth conditions involving image transmission, the network performance deteriorates considerably, both in terms of latency and data integrity. One possible reason is the wireless router used in the current setup. It is also recommended that, especially under high-bandwidth loads, it is necessary for networked systems to perform some portion of their computations on-board and high-bandwidth wireless connectivity to the cloud is facilitated. Ongoing research and future directions are also outlined