Agent-Based Distributed Resource Allocation in Continuous Dynamic Systems

Intelligent agents and multiagent systems reveal new strategies to design highly flexible automation systems. There are first promising industrial applications of multiagent systems for the control of manufacturing, logistics, traffic or multi-robot systems. One reason for the success of most of these applications is their nature as some form of a distributed resource allocation problem which can be addressed very well by multiagent systems. Resource allocation problems solved by agents can be further categorized into static or dynamic problems. In static problems, the allocations do not depend on time and many resource allocation problem of practical interest can be solved using these static considerations, even in discrete-event systems like manufacturing or logistic systems. However, problems especially in highly dynamic environments cannot be addressed by this pure static approach since the allocations, i.e. the decision variables, depend on time and previous states of the considered system. These problems are hardly considered in the relevant agent literature and if, most often only discrete-event systems are considered. This work focuses on agent-based distributed dynamic resource allocation problems especially in continuous production systems or other continuous systems. Based on the current states of the distributed dynamic system, continuous-time allocation trajectories must be computed in real-time. Designing multiagent systems for distributed resource allocation mainly comprises the design of the local capabilities of the single agents and the interaction mechanisms that makes them find the best or at least a feasible allocation without any central control. In this work, the agents are designed as two-level entities: while the low-level functions are responsible for the real-time allocation of the resources in the form of closed-loop feedback control, the high-level functionalities realize the deliberative capabilities such as long-term planning and negotiation of the resource allocations. Herein, the resource allocation problem is considered as a distributed optimization problem under certain constraints. The agents play the role of local optimizers which then have to coordinate their local solutions to an overall consistent solution. It is shown in this contribution that the described approach can be interpreted as a market-based allocation scheme based on balancing of supply and demand of the resources using a virtual price. However, the agents calculate and negotiate complete supply and demand trajectories using model-based predictions which also leads to the calculation of a price trajectory. This novel approach does not only consider the dynamic behaviour of the distributed system but also combines control tasks and resource allocation in a very consistent way. The approach is demonstrated using two practical applications: a heating system and an industrial sugar extraction process

Optimal Event-Driven Multi-Agent Persistent Monitoring of a Finite Set of Targets

We consider the problem of controlling the movement of multiple cooperating agents so as to minimize an uncertainty metric associated with a finite number of targets. In a one-dimensional mission space, we adopt an optimal control framework and show that the solution is reduced to a simpler parametric optimization problem: determining a sequence of locations where each agent may dwell for a finite amount of time and then switch direction. This amounts to a hybrid system which we analyze using Infinitesimal Perturbation Analysis (IPA) to obtain a complete on-line solution through an event-driven gradient-based algorithm which is also robust with respect to the uncertainty model used. The resulting controller depends on observing the events required to excite the gradient-based algorithm, which cannot be guaranteed. We solve this problem by proposing a new metric for the objective function which creates a potential field guaranteeing that gradient values are non-zero. This approach is compared to an alternative graph-based task scheduling algorithm for determining an optimal sequence of target visits. Simulation examples are included to demonstrate the proposed methods.Comment: 12 pages full version, IEEE Conference on Decision and Control, 201

Potential applications of simulation modelling techniques in healthcare: lessons learned from aerospace and military

The Aerospace and Military areas are to do with complex missions and situations. Modelling and Simulation (M&S) has been applied in many areas of defence ranging from space sciences, satellite engineering to multi-warfare (air warfare, undersea warfare), air & missile defence, acquisition, tactical military trainings & exercises, national security analysis and strategic decision making & planning, etc. The application of simulation modelling techniques in healthcare would improve the provision of healthcare services; however, their application has been much relatively feeble in the healthcare sector as compared to the defence sector. This paper presents results from a systematic literature survey on applications of modelling simulation techniques in the Aerospace & Military. The knowledge gained or lessons learned from the survey were finally used to analyze the potential applications of the simulation modelling techniques to the healthcare sector. Results show that in the defence sector, Distributed Simulation has now become a widely adopted technique. However, System Dynamics (SD) and Discrete Event Simulation (DSE) have also gained relative attention. From this survey it becomes clear that various simulation modelling techniques are useful for specific purposes and have potential applications in the healthcare sector

Conservative collision prediction and avoidance for stochastic trajectories in continuous time and space

Existing work in multi-agent collision prediction and avoidance typically assumes discrete-time trajectories with Gaussian uncertainty or that are completely deterministic. We propose an approach that allows detection of collisions even between continuous, stochastic trajectories with the only restriction that means and variances can be computed. To this end, we employ probabilistic bounds to derive criterion functions whose negative sign provably is indicative of probable collisions. For criterion functions that are Lipschitz, an algorithm is provided to rapidly find negative values or prove their absence. We propose an iterative policy-search approach that avoids prior discretisations and yields collision-free trajectories with adjustably high certainty. We test our method with both fixed-priority and auction-based protocols for coordinating the iterative planning process. Results are provided in collision-avoidance simulations of feedback controlled plants.Comment: This preprint is an extended version of a conference paper that is to appear in \textit{Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2014)

CompILE: Compositional Imitation Learning and Execution

We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data. CompILE uses a novel unsupervised, fully-differentiable sequence segmentation module to learn latent encodings of sequential data that can be re-composed and executed to perform new tasks. Once trained, our model generalizes to sequences of longer length and from environment instances not seen during training. We evaluate CompILE in a challenging 2D multi-task environment and a continuous control task, and show that it can find correct task boundaries and event encodings in an unsupervised manner. Latent codes and associated behavior policies discovered by CompILE can be used by a hierarchical agent, where the high-level policy selects actions in the latent code space, and the low-level, task-specific policies are simply the learned decoders. We found that our CompILE-based agent could learn given only sparse rewards, where agents without task-specific policies struggle.Comment: ICML (2019