Multi-Agent Modelling of Industrial Cyber-Physical Systems for IEC 61499 Based Distributed Intelligent Automation

Traditional industrial automation systems developed under IEC 61131-3 in centralized architectures are statically programmed with determined procedures to perform predefined tasks in structured environments. Major challenges are that these systems designed under traditional engineering techniques and running on legacy automation platforms are unable to automatically discover alternative solutions, flexibly coordinate reconfigurable modules, and actively deploy corresponding functions, to quickly respond to frequent changes and intelligently adapt to evolving requirements in dynamic environments. The core objective of this research is to explore the design of multi-layer automation architectures to enable real-time adaptation at the device level and run-time intelligence throughout the whole system under a well-integrated modelling framework. Central to this goal is the research on the integration of multi-agent modelling and IEC 61499 function block modelling to form a new automation infrastructure for industrial cyber-physical systems. Multi-agent modelling uses autonomous and cooperative agents to achieve run-time intelligence in system design and module reconfiguration. IEC 61499 function block modelling applies object-oriented and event-driven function blocks to realize real-time adaption of automation logic and control algorithms. In this thesis, the design focuses on a two-layer self-manageable architecture modelling: a) the high-level cyber module designed as multi-agent computing model consisting of Monitoring Agent, Analysis Agent, Self-Learning Agent, Planning Agent, Execution Agent, and Knowledge Agent; and b) the low-level physical module designed as agent-embedded IEC 61499 function block model with Self-Manageable Service Execution Agent, Self-Configuration Agent, Self-Healing Agent, Self-Optimization Agent, and Self-Protection Agent. The design results in a new computing module for high-level multi-agent based automation architectures and a new design pattern for low-level function block modelled control solutions. The architecture modelling framework is demonstrated through various tests on the multi-agent simulation model developed in the agent modelling environment NetLogo and the experimental testbed designed on the Jetson Nano and Raspberry Pi platforms. The performance evaluation of regular execution time and adaptation time in two typical conditions for systems designed under three different architectures are also analyzed. The results demonstrate the ability of the proposed architecture to respond to major challenges in Industry 4.0

Architecture and negotiation protocols for a smart parking system

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáSmart City uses emerging technologies to improve citizens’ quality of life. A branch of this topic is the Smart Parking, where the parking system implements intelligent mechanisms to simplify to the searching of parking spots and consequently decrease the traffic of cars. This work proposes an architecture using Multi-Agent System (MAS), enhanced with some holonic systems principles, that is capable to be applied to different range of parking systems, e.g., considering trucks, cars, or bicycles. Being a distributed architecture, a special attention is devoted to study the negotiation protocols that will regulate the behavior of autonomous and cooperative actors in the system, namely drivers and parking spots, during allocation process of parking spots to drivers. For this purpose, the Contract Net Protocol (CNP), English Auction, Dutch Auction and Faratin Auction were the tested, being the CNP the selected protocol for this problem. Also addressing the distributed nature of the system, some efforts were focused on the security of the messages exchanged between the agents was proposed using Secure Socket Layer (SSL). The proposed multi-agent systems architecture was implemented using JADE (Java Agent DEvelopment Framework), which is a FIPA-compliant agent development framework that simplifies the development of agent-based applications. The exchange of messages follows the FIPA-ACL protocol using the CNP protocol for the negotiation. The communication between the agents and the User Interface is performed through the use of Message Queuing Telemetry Transport (MQTT) protocol

Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved significant success in single-agent decision making tasks. However, it is challenging to extend Q-learning to large-scale multi-agent scenarios, due to the explosion of action space resulting from the complex dynamics between the environment and the agents. In this paper, we propose to make the computation of multi-agent Q-learning tractable by treating the Q-function (w.r.t. state and joint-action) as a high-order high-dimensional tensor and then approximate it with factorized pairwise interactions. Furthermore, we utilize a composite deep neural network architecture for computing the factorized Q-function, share the model parameters among all the agents within the same group, and estimate the agents' optimal joint actions through a coordinate descent type algorithm. All these simplifications greatly reduce the model complexity and accelerate the learning process. Extensive experiments on two different multi-agent problems demonstrate the performance gain of our proposed approach in comparison with strong baselines, particularly when there are a large number of agents.Comment: 7 pages, 5 figures, DAI 201

Decentralization of Multiagent Policies by Learning What to Communicate

Effective communication is required for teams of robots to solve sophisticated collaborative tasks. In practice it is typical for both the encoding and semantics of communication to be manually defined by an expert; this is true regardless of whether the behaviors themselves are bespoke, optimization based, or learned. We present an agent architecture and training methodology using neural networks to learn task-oriented communication semantics based on the example of a communication-unaware expert policy. A perimeter defense game illustrates the system's ability to handle dynamically changing numbers of agents and its graceful degradation in performance as communication constraints are tightened or the expert's observability assumptions are broken.Comment: 7 page