Multi-Task Generative Adversarial Nets with Shared Memory for Cross-Domain Coordination Control

Generating sequential decision process from huge amounts of measured process data is a future research direction for collaborative factory automation, making full use of those online or offline process data to directly design flexible make decisions policy, and evaluate performance. The key challenges for the sequential decision process is to online generate sequential decision-making policy directly, and transferring knowledge across tasks domain. Most multi-task policy generating algorithms often suffer from insufficient generating cross-task sharing structure at discrete-time nonlinear systems with applications. This paper proposes the multi-task generative adversarial nets with shared memory for cross-domain coordination control, which can generate sequential decision policy directly from raw sensory input of all of tasks, and online evaluate performance of system actions in discrete-time nonlinear systems. Experiments have been undertaken using a professional flexible manufacturing testbed deployed within a smart factory of Weichai Power in China. Results on three groups of discrete-time nonlinear control tasks show that our proposed model can availably improve the performance of task with the help of other related tasks

Efficient Collaborative Multi-Agent Deep Reinforcement Learning for Large-Scale Fleet Management

Large-scale online ride-sharing platforms have substantially transformed our lives by reallocating transportation resources to alleviate traffic congestion and promote transportation efficiency. An efficient fleet management strategy not only can significantly improve the utilization of transportation resources but also increase the revenue and customer satisfaction. It is a challenging task to design an effective fleet management strategy that can adapt to an environment involving complex dynamics between demand and supply. Existing studies usually work on a simplified problem setting that can hardly capture the complicated stochastic demand-supply variations in high-dimensional space. In this paper we propose to tackle the large-scale fleet management problem using reinforcement learning, and propose a contextual multi-agent reinforcement learning framework including three concrete algorithms to achieve coordination among a large number of agents adaptive to different contexts. We show significant improvements of the proposed framework over state-of-the-art approaches through extensive empirical studies

Multi-Agent Actor-Critic with Generative Cooperative Policy Network

We propose an efficient multi-agent reinforcement learning approach to derive equilibrium strategies for multi-agents who are participating in a Markov game. Mainly, we are focused on obtaining decentralized policies for agents to maximize the performance of a collaborative task by all the agents, which is similar to solving a decentralized Markov decision process. We propose to use two different policy networks: (1) decentralized greedy policy network used to generate greedy action during training and execution period and (2) generative cooperative policy network (GCPN) used to generate action samples to make other agents improve their objectives during training period. We show that the samples generated by GCPN enable other agents to explore the policy space more effectively and favorably to reach a better policy in terms of achieving the collaborative tasks.Comment: 10 pages, total 9 figures including all sub-figure

MARL-FWC: Optimal Coordination of Freeway Traffic Control Measures

The objective of this article is to optimize the overall traffic flow on freeways using multiple ramp metering controls plus its complementary Dynamic Speed Limits (DSLs). An optimal freeway operation can be reached when minimizing the difference between the freeway density and the critical ratio for maximum traffic flow. In this article, a Multi-Agent Reinforcement Learning for Freeways Control (MARL-FWC) system for ramps metering and DSLs is proposed. MARL-FWC introduces a new microscopic framework at the network level based on collaborative Markov Decision Process modeling (Markov game) and an associated cooperative Q-learning algorithm. The technique incorporates payoff propagation (Max-Plus algorithm) under the coordination graphs framework, particularly suited for optimal control purposes. MARL-FWC provides three control designs: fully independent, fully distributed, and centralized; suited for different network architectures. MARL-FWC was extensively tested in order to assess the proposed model of the joint payoff, as well as the global payoff. Experiments are conducted with heavy traffic flow under the renowned VISSIM traffic simulator to evaluate MARL-FWC. The experimental results show a significant decrease in the total travel time and an increase in the average speed (when compared with the base case) while maintaining an optimal traffic flow

Training on multi-agent systems, social sciences, and integrated natural resource management : lessons from an Inter-University Project in Thailand

In this new century, there is an urgent need to integrate and organize knowledge into suitable frameworks to examine essential problems with the people involved in solving them. Recent advances in computer science, particularly distributed artificial intelligence and multi-agent systems (MAS), are creating a strong interest in using this new knowledge and technologies for various applications to better deal with the increasing complexity of our fast-changing world, particularly for studying interactions between societies and their environment. By emphasizing the importance of interactions and points of view, the MAS way of thinking can facilitate high-level interdisciplinary training and collaborative research among scientists working in ecology and social sciences to examine complex problems in the field of integrated natural resource management (INRM). This paper describes how a recent project based on a series of short courses in the field of MAS, social sciences, and INRM at three different universities in Thailand tried to transfer European expertise and research results to an Asian audience of graduate and postgraduate students and young researchers interested in innovative and action-research-oriented interdisciplinary approaches. The course structure, organization, and contents are described and assessed. The course participants are characterized and their opinions are used to evaluate the strengths and weaknesses of this very interdisciplinary training program. The first sustainable outputs and key preliminary lessons learned from this innovative collective learning experience are presented. In conclusion, the authors suggest ways to support the emergence of a regional network of "MAS for INRM" practitioners in Southeast Asia to build on the dynamics begun by this project and serve the need for such interdisciplinary training across Southeast Asia. (Résumé d'auteur

Heterogeneous Coexistence of Cognitive Radio Networks in TV White Space

Wireless standards (e.g., IEEE 802.11af and 802.22) have been developed for enabling opportunistic access in TV white space (TVWS) using cognitive radio (CR) technology. When heterogeneous CR networks that are based on different wireless standards operate in the same TVWS, coexistence issues can potentially cause major problems. Enabling collaborative coexistence via direct coordination between heterogeneous CR networks is very challenging, due to incompatible MAC/PHY designs of coexisting networks, requirement of an over-the-air common control channel for inter-network communications, and time synchronization across devices from different networks. Moreover, such a coexistence scheme would require competing networks or service providers to exchange sensitive control information that may raise conflict of interest issues and customer privacy concerns. In this paper, we present an architecture for enabling collaborative coexistence of heterogeneous CR networks over TVWS, called Symbiotic Heterogeneous coexistence ARchitecturE (SHARE). Define "indirect coordination" first before using it. Because coexistence cannot avoid coordination By mimicking the symbiotic relationships between heterogeneous organisms in a stable ecosystem, SHARE establishes an indirect coordination mechanism between heterogeneous CR networks via a mediator system, which avoids the drawbacks of direct coordination. SHARE includes two spectrum sharing algorithms whose designs were inspired by well-known models and theories from theoretical ecology, viz, the interspecific competition model and the ideal free distribution model

The FuturIcT Knowledge Accelerator: Unleashing the Power of Information for a Sustainable Future

With our knowledge of the universe, we have sent men to the moon. We know microscopic details of objects around us and within us. And yet we know relatively little about how our society works and how it reacts to changes brought upon it. Humankind is now facing serious crises for which we must develop new ways to tackle the global challenges of humanity in the 21st century. With connectivity between people rapidly increasing, we are now able to exploit information and communication technologies to achieve major breakthroughs that go beyond the step-wise improvements in other areas. The need of a socio-economic knowledge collider was first pointed out in the OECD Global Science Forum on Applications of Complexity Science for Public Policy in Erice from October 5 to 7, 2008. Since then, many scientists have called for a large-scale ICT-based research initiative on techno-socialeconomic- environmental issues, sometimes phrased as a Manhattan-, Apollo-, or CERN-like project to study the way our living planet works in a social dimension. Due to the connotations, we use the term knowledge accelerator, here.Comment: For related information see http://www.futurict.eu (The spelling error in Sec. 2.5 was removed: "exclusion" was replaced by "inclusion"

CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms

How to optimally dispatch orders to vehicles and how to tradeoff between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a large-scale parallel ranking problem and study the joint decision-making task of order dispatching and fleet management in online ride-hailing platforms. This task brings unique challenges in the following four aspects. First, to facilitate a huge number of vehicles to act and learn efficiently and robustly, we treat each region cell as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to achieve long-term benefits, we leverage the geographical hierarchy of the region grids to perform hierarchical reinforcement learning. Third, to deal with the heterogeneous and variant action space for joint order dispatching and fleet management, we design the action as the ranking weight vector to rank and select the specific order or the fleet management destination in a unified formulation. Fourth, to achieve the multi-scale ride-hailing platform, we conduct the decision-making process in a hierarchical way where a multi-head attention mechanism is utilized to incorporate the impacts of neighbor agents and capture the key agent in each scale. The whole novel framework is named as CoRide. Extensive experiments based on multiple cities real-world data as well as analytic synthetic data demonstrate that CoRide provides superior performance in terms of platform revenue and user experience in the task of city-wide hybrid order dispatching and fleet management over strong baselines.Comment: CIKM 201

Spiking Neural Networks for Early Prediction in Human Robot Collaboration

This paper introduces the Turn-Taking Spiking Neural Network (TTSNet), which is a cognitive model to perform early turn-taking prediction about human or agent's intentions. The TTSNet framework relies on implicit and explicit multimodal communication cues (physical, neurological and physiological) to be able to predict when the turn-taking event will occur in a robust and unambiguous fashion. To test the theories proposed, the TTSNet framework was implemented on an assistant robotic nurse, which predicts surgeon's turn-taking intentions and delivers surgical instruments accordingly. Experiments were conducted to evaluate TTSNet's performance in early turn-taking prediction. It was found to reach a F1 score of 0.683 given 10% of completed action, and a F1 score of 0.852 at 50% and 0.894 at 100% of the completed action. This performance outperformed multiple state-of-the-art algorithms, and surpassed human performance when limited partial observation is given (< 40%). Such early turn-taking prediction capability would allow robots to perform collaborative actions proactively, in order to facilitate collaboration and increase team efficiency.Comment: Under review for journa

An agent-based dynamic information network for supply chain management

One of the main research issues in supply chain management is to improve the global efficiency of supply chains. However, the improvement efforts often fail because supply chains are complex, are subject to frequent changes, and collaboration and information sharing in the supply chains are often infeasible. This paper presents a practical collaboration framework for supply chain management wherein multi-agent systems form dynamic information networks and coordinate their production and order planning according to synchronized estimation of market demands. In the framework, agents employ an iterative relaxation contract net protocol to find the most desirable suppliers by using data envelopment analysis. Furthermore, the chain of buyers and suppliers, from the end markets to raw material suppliers, form dynamic information networks for synchronized planning. This paper presents an agent-based dynamic information network for supply chain management and discusses the associated pros and cons