Priority-based reserved spectrum allocation by multi-agent through reinforcement learning in cognitive radio network

Research in cognitive radio networks aims at maximized spectrum utilization by giving access to increased users with the help of dynamic spectrum allocation policy. The unknown and rapid dynamic nature of the radio environment makes the decision making and optimized resource allocation to be a challenging one. In order to support dynamic spectrum allocation, intelligence is needed to be incorporated in the cognitive system to study the environment parameters, internal state, and operating behaviour of the radio and based on which decisions need to be made for the allocation of under-utilized spectrum. A novel priority-based reserved allocation method with a multi-agent system is proposed for spectrum allocation. The multi-agent system performs the task of gathering environmental artefacts used for decision making to give the best of effort service in this adaptive communication

Dynamic Task-Allocation for Unmanned Aircraft Systems

This dissertation addresses improvements to a consensus based task allocation algorithms for improving the Quality of Service in multi-task and multi-agent environments. Research in the past has led to many centralized task allocation algorithms where a central computation unit is calculating the global optimum task allocation solution. The centralized algorithms are plagued by creating a single point of failure and the bandwidth needed for creating consistent and accurate situational awareness off all agents. This work will extend upon a widely researched decentralized task assignment algorithm based on the consensus principle. Although many extensions have led to improvements of the original algorithm, there is still much opportunity for improvement in providing sufficient and reliable task assignments in real-world dynamic conditions and changing environments. This research addresses practical changes made to the consensus based task allocation algorithms for improving the Quality of Service in multi-task and multi-agent environments

A Self-adaptive Agent-based System for Cloud Platforms

Cloud computing is a model for enabling on-demand network access to a shared pool of computing resources, that can be dynamically allocated and released with minimal effort. However, this task can be complex in highly dynamic environments with various resources to allocate for an increasing number of different users requirements. In this work, we propose a Cloud architecture based on a multi-agent system exhibiting a self-adaptive behavior to address the dynamic resource allocation. This self-adaptive system follows a MAPE-K approach to reason and act, according to QoS, Cloud service information, and propagated run-time information, to detect QoS degradation and make better resource allocation decisions. We validate our proposed Cloud architecture by simulation. Results show that it can properly allocate resources to reduce energy consumption, while satisfying the users demanded QoS

Partial Replanning for Decentralized Dynamic Task Allocation

In time-sensitive and dynamic missions, multi-UAV teams must respond quickly to new information and objectives. This paper presents a dynamic decentralized task allocation algorithm for allocating new tasks that appear online during the solving of the task allocation problem. Our algorithm extends the Consensus-Based Bundle Algorithm (CBBA), a decentralized task allocation algorithm, allowing for the fast allocation of new tasks without a full reallocation of existing tasks. CBBA with Partial Replanning (CBBA-PR) enables the team to trade-off between convergence time and increased coordination by resetting a portion of their previous allocation at every round of bidding on tasks. By resetting the last tasks allocated by each agent, we are able to ensure the convergence of the team to a conflict-free solution. CBBA-PR can be further improved by reducing the team size involved in the replanning, further reducing the communication burden of the team and runtime of CBBA-PR. Finally, we validate the faster convergence and improved solution quality of CBBA-PR in multi-UAV simulations.Comment: 11 pages, Accepted to AIAA GNC 201

Toward multi-target self-organizing pursuit in a partially observable Markov game

The multiple-target self-organizing pursuit (SOP) problem has wide applications and has been considered a challenging self-organization game for distributed systems, in which intelligent agents cooperatively pursue multiple dynamic targets with partial observations. This work proposes a framework for decentralized multi-agent systems to improve intelligent agents' search and pursuit capabilities. We model a self-organizing system as a partially observable Markov game (POMG) with the features of decentralization, partial observation, and noncommunication. The proposed distributed algorithm: fuzzy self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the three challenges in multi-target SOP: distributed self-organizing search (SOS), distributed task allocation, and distributed single-target pursuit. FSC2 includes a coordinated multi-agent deep reinforcement learning method that enables homogeneous agents to learn natural SOS patterns. Additionally, we propose a fuzzy-based distributed task allocation method, which locally decomposes multi-target SOP into several single-target pursuit problems. The cooperative coevolution principle is employed to coordinate distributed pursuers for each single-target pursuit problem. Therefore, the uncertainties of inherent partial observation and distributed decision-making in the POMG can be alleviated. The experimental results demonstrate that distributed noncommunicating multi-agent coordination with partial observations in all three subtasks are effective, and 2048 FSC2 agents can perform efficient multi-target SOP with almost 100% capture rates

Centralised Versus Market-Based Control Under Environment Uncertainty: Case of the Mobile Task Allocation Problem (MTAP)

This paper aims at comparing the centralised versus the market-based approach. This is done in the context of the mobile task allocation problem (MTAP) from the perspective of environmental uncertainty. MTAP is defined as an optimization problem for planning the assignment of service tasks to mobile workers. Environmental uncertainty is introduced through the injection of stochastic tasks and dynamic travel delays. A multi-agent simulator is employed to experiment the behaviour of each approach in reaction to different uncertainty levels. Preliminary results suggest a tentative conceptual model to evaluate the suitability of each approach to address MTAP in function of uncertainty. It is suggested that uncertainty’s effect on achieved performance is moderated by the timeliness of decision making, workers’ degree of local knowledge, and problem’s complexity and size

Multi-robot task allocation for safe planning under dynamic uncertainties

This paper considers the problem of multi-robot safe mission planning in uncertain dynamic environments. This problem arises in several applications including safety-critical exploration, surveillance, and emergency rescue missions. Computation of a multi-robot optimal control policy is challenging not only because of the complexity of incorporating dynamic uncertainties while planning, but also because of the exponential growth in problem size as a function of the number of robots. Leveraging recent works obtaining a tractable safety maximizing plan for a single robot, we propose a scalable two-stage framework to solve the problem at hand. Specifically, the problem is split into a low-level single-agent planning problem and a high-level task allocation problem. The low-level problem uses an efficient approximation of stochastic reachability for a Markov decision process to handle the dynamic uncertainty. The task allocation, on the other hand, is solved using polynomial-time forward and reverse greedy heuristics. The safety objective of our multi-robot safe planning problem allows an implementation of the greedy heuristics through a distributed auction-based approach. Moreover, by leveraging the properties of the safety objective function, we ensure provable performance bounds on the safety of the approximate solutions proposed by these two heuristics. Our result is illustrated through case studies