467 research outputs found
Adaptive and learning-based formation control of swarm robots
Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation
Sustainable Task Offloading in UAV Networks via Multi-Agent Reinforcement Learning
The recent growth of IoT devices, along with edge computing, has revealed many opportunities for novel applications. Among them, Unmanned Aerial Vehicles (UAVs), which are deployed for surveillance and environmental monitoring, are attracting increasing attention. In this context, typical solutions must deal with events that may change the state of the network, providing a service that continuously maintains a high level of performance. In this paper, we address this problem by proposing a distributed architecture that leverages a Multi-Agent Reinforcement Learning (MARL) technique to dynamically offload tasks from UAVs to the edge cloud. Nodes of the system co-operate to jointly minimize the overall latency perceived by the user and the energy usage on UAVs by continuously learning from the environment the best action, which entails the decision of offloading and, in this case, the best transmission technology, i.e., Wi-Fi or cellular. Results validate our distributed architecture and show the effectiveness of the approach in reaching the above targets
Multi-Robot Systems: Challenges, Trends and Applications
This book is a printed edition of the Special Issue entitled “Multi-Robot Systems: Challenges, Trends, and Applications” that was published in Applied Sciences. This Special Issue collected seventeen high-quality papers that discuss the main challenges of multi-robot systems, present the trends to address these issues, and report various relevant applications. Some of the topics addressed by these papers are robot swarms, mission planning, robot teaming, machine learning, immersive technologies, search and rescue, and social robotics
Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control
Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs)
in monitoring and remote area surveillance applications has become widespread
thanks to the price reduction and the increased capabilities of drones. The
drones in the swarm need to cooperatively explore an unknown area, in order to
identify and monitor interesting targets, while minimizing their movements. In
this work, we propose a distributed Reinforcement Learning (RL) approach that
scales to larger swarms without modifications. The proposed framework relies on
the possibility for the UAVs to exchange some information through a
communication channel, in order to achieve context-awareness and implicitly
coordinate the swarm's actions. Our experiments show that the proposed method
can yield effective strategies, which are robust to communication channel
impairments, and that can easily deal with non-uniform distributions of targets
and obstacles. Moreover, when agents are trained in a specific scenario, they
can adapt to a new one with minimal additional training. We also show that our
approach achieves better performance compared to a computationally intensive
look-ahead heuristic.Comment: Preprint of the paper published in IEEE Transactions on Cognitive
Communications and Networking ( Early Access
Machine Learning for Unmanned Aerial System (UAS) Networking
Fueled by the advancement of 5G new radio (5G NR), rapid development has occurred in many fields. Compared with the conventional approaches, beamforming and network slicing enable 5G NR to have ten times decrease in latency, connection density, and experienced throughput than 4G long term evolution (4G LTE). These advantages pave the way for the evolution of Cyber-physical Systems (CPS) on a large scale. The reduction of consumption, the advancement of control engineering, and the simplification of Unmanned Aircraft System (UAS) enable the UAS networking deployment on a large scale to become feasible. The UAS networking can finish multiple complex missions simultaneously. However, the limitations of the conventional approaches are still a big challenge to make a trade-off between the massive management and efficient networking on a large scale.
With 5G NR and machine learning, in this dissertation, my contributions can be summarized as the following: I proposed a novel Optimized Ad-hoc On-demand Distance Vector (OAODV) routing protocol to improve the throughput of Intra UAS networking. The novel routing protocol can reduce the system overhead and be efficient. To improve the security, I proposed a blockchain scheme to mitigate the malicious basestations for cellular connected UAS networking and a proof-of-traffic (PoT) to improve the efficiency of blockchain for UAS networking on a large scale. Inspired by the biological cell paradigm, I proposed the cell wall routing protocols for heterogeneous UAS networking. With 5G NR, the inter connections between UAS networking can strengthen the throughput and elasticity of UAS networking. With machine learning, the routing schedulings for intra- and inter- UAS networking can enhance the throughput of UAS networking on a large scale. The inter UAS networking can achieve the max-min throughput globally edge coloring. I leveraged the upper and lower bound to accelerate the optimization of edge coloring.
This dissertation paves a way regarding UAS networking in the integration of CPS and machine learning. The UAS networking can achieve outstanding performance in a decentralized architecture. Concurrently, this dissertation gives insights into UAS networking on a large scale. These are fundamental to integrating UAS and National Aerial System (NAS), critical to aviation in the operated and unmanned fields. The dissertation provides novel approaches for the promotion of UAS networking on a large scale. The proposed approaches extend the state-of-the-art of UAS networking in a decentralized architecture. All the alterations can contribute to the establishment of UAS networking with CPS
A survey on intelligent computation offloading and pricing strategy in UAV-Enabled MEC network: Challenges and research directions
The lack of resource constraints for edge servers makes it difficult to simultaneously perform a large number of Mobile Devices’ (MDs) requests. The Mobile Network Operator (MNO) must then select how to delegate MD queries to its Mobile Edge Computing (MEC) server in order to maximize the overall benefit of admitted requests with varying latency needs. Unmanned Aerial Vehicles (UAVs) and Artificial Intelligent (AI) can increase MNO performance because of their flexibility in deployment, high mobility of UAV, and efficiency of AI algorithms. There is a trade-off between the cost incurred by the MD and the profit received by the MNO. Intelligent computing offloading to UAV-enabled MEC, on the other hand, is a promising way to bridge the gap between MDs' limited processing resources, as well as the intelligent algorithms that are utilized for computation offloading in the UAV-MEC network and the high computing demands of upcoming applications. This study looks at some of the research on the benefits of computation offloading process in the UAV-MEC network, as well as the intelligent models that are utilized for computation offloading in the UAV-MEC network. In addition, this article examines several intelligent pricing techniques in different structures in the UAV-MEC network. Finally, this work highlights some important open research issues and future research directions of Artificial Intelligent (AI) in computation offloading and applying intelligent pricing strategies in the UAV-MEC network
Towards Autonomous Computer Networks in Support of Critical Systems
L'abstract è presente nell'allegato / the abstract is in the attachmen
A Comprehensive Overview on 5G-and-Beyond Networks with UAVs: From Communications to Sensing and Intelligence
Due to the advancements in cellular technologies and the dense deployment of
cellular infrastructure, integrating unmanned aerial vehicles (UAVs) into the
fifth-generation (5G) and beyond cellular networks is a promising solution to
achieve safe UAV operation as well as enabling diversified applications with
mission-specific payload data delivery. In particular, 5G networks need to
support three typical usage scenarios, namely, enhanced mobile broadband
(eMBB), ultra-reliable low-latency communications (URLLC), and massive
machine-type communications (mMTC). On the one hand, UAVs can be leveraged as
cost-effective aerial platforms to provide ground users with enhanced
communication services by exploiting their high cruising altitude and
controllable maneuverability in three-dimensional (3D) space. On the other
hand, providing such communication services simultaneously for both UAV and
ground users poses new challenges due to the need for ubiquitous 3D signal
coverage as well as the strong air-ground network interference. Besides the
requirement of high-performance wireless communications, the ability to support
effective and efficient sensing as well as network intelligence is also
essential for 5G-and-beyond 3D heterogeneous wireless networks with coexisting
aerial and ground users. In this paper, we provide a comprehensive overview of
the latest research efforts on integrating UAVs into cellular networks, with an
emphasis on how to exploit advanced techniques (e.g., intelligent reflecting
surface, short packet transmission, energy harvesting, joint communication and
radar sensing, and edge intelligence) to meet the diversified service
requirements of next-generation wireless systems. Moreover, we highlight
important directions for further investigation in future work.Comment: Accepted by IEEE JSA
- …