Multi-Agent Systems

A multi-agent system (MAS) is a system composed of multiple interacting intelligent agents. Multi-agent systems can be used to solve problems which are difficult or impossible for an individual agent or monolithic system to solve. Agent systems are open and extensible systems that allow for the deployment of autonomous and proactive software components. Multi-agent systems have been brought up and used in several application domains

An Investigation into Factors Affecting the Chilled Food Industry

With the advent of Industry 4.0, many new approaches towards process monitoring, benchmarking and traceability are becoming available, and these techniques have the potential to radically transform the agri-food sector. In particular, the chilled food supply chain (CFSC) contains a number of unique challenges by virtue of it being thought of as a temperature controlled supply chain. Therefore, once the key issues affecting the CFSC have been identified, algorithms can be proposed, which would allow realistic thresholds to be established for managing these problems on the micro, meso and macro scales. Hence, a study is required into factors affecting the CFSC within the scope of Industry 4.0. The study itself has been broken down into four main topics: identifying the key issues within the CFSC; implementing a philosophy of continuous improvement within the CFSC; identifying uncertainty within the CFSC; improving and measuring the performance of the supply chain. However, as a consequence of this study two further topics were added: a discussion of some of the issues surrounding information sharing between retailers and suppliers; some of the wider issues affecting food losses and wastage (FLW) on the micro, meso and macro scales. A hybrid algorithm is developed, which incorporates the analytic hierarchical process (AHP) for qualitative issues and data envelopment analysis (DEA) for quantitative issues. The hybrid algorithm itself is a development of the internal auditing algorithm proposed by Sueyoshi et al (2009), which in turn was developed following corporate scandals such as Tyco, Enron, and WorldCom, which have led to a decline in public trust. However, the advantage of the proposed solution is that all of the key issues within the CFSC identified can be managed from a single computer terminal, whilst the risk of food contamination such as the 2013 horsemeat scandal can be avoided via improved traceability

Journal of Telecommunications and Information Technology

kwartalni

Many-agent Reinforcement Learning

Multi-agent reinforcement learning (RL) solves the problem of how each agent should behave optimally in a stochastic environment in which multiple agents are learning simultaneously. It is an interdisciplinary domain with a long history that lies in the joint area of psychology, control theory, game theory, reinforcement learning, and deep learning. Following the remarkable success of the AlphaGO series in single-agent RL, 2019 was a booming year that witnessed significant advances in multi-agent RL techniques; impressive breakthroughs have been made on developing AIs that outperform humans on many challenging tasks, especially multi-player video games. Nonetheless, one of the key challenges of multi-agent RL techniques is the scalability; it is still non-trivial to design efficient learning algorithms that can solve tasks including far more than two agents ($N \gg 2$), which I name by \emph{many-agent reinforcement learning} (MARL\footnote{I use the world of ``MARL" to denote multi-agent reinforcement learning with a particular focus on the cases of many agents; otherwise, it is denoted as ``Multi-Agent RL" by default.}) problems. In this thesis, I contribute to tackling MARL problems from four aspects. Firstly, I offer a self-contained overview of multi-agent RL techniques from a game-theoretical perspective. This overview fills the research gap that most of the existing work either fails to cover the recent advances since 2010 or does not pay adequate attention to game theory, which I believe is the cornerstone to solving many-agent learning problems. Secondly, I develop a tractable policy evaluation algorithm -- $\alpha^\alpha$-Rank -- in many-agent systems. The critical advantage of $\alpha^\alpha$-Rank is that it can compute the solution concept of $\alpha$-Rank tractably in multi-player general-sum games with no need to store the entire pay-off matrix. This is in contrast to classic solution concepts such as Nash equilibrium which is known to be $PPAD$-hard in even two-player cases. $\alpha^\alpha$-Rank allows us, for the first time, to practically conduct large-scale multi-agent evaluations. Thirdly, I introduce a scalable policy learning algorithm -- mean-field MARL -- in many-agent systems. The mean-field MARL method takes advantage of the mean-field approximation from physics, and it is the first provably convergent algorithm that tries to break the curse of dimensionality for MARL tasks. With the proposed algorithm, I report the first result of solving the Ising model and multi-agent battle games through a MARL approach. Fourthly, I investigate the many-agent learning problem in open-ended meta-games (i.e., the game of a game in the policy space). Specifically, I focus on modelling the behavioural diversity in meta-games, and developing algorithms that guarantee to enlarge diversity during training. The proposed metric based on determinantal point processes serves as the first mathematically rigorous definition for diversity. Importantly, the diversity-aware learning algorithms beat the existing state-of-the-art game solvers in terms of exploitability by a large margin. On top of the algorithmic developments, I also contribute two real-world applications of MARL techniques. Specifically, I demonstrate the great potential of applying MARL to study the emergent population dynamics in nature, and model diverse and realistic interactions in autonomous driving. Both applications embody the prospect that MARL techniques could achieve huge impacts in the real physical world, outside of purely video games

State of the art of cyber-physical systems security: An automatic control perspective

Cyber-physical systems are integrations of computation, networking, and physical processes. Due to the tight cyber-physical coupling and to the potentially disrupting consequences of failures, security here is one of the primary concerns. Our systematic mapping study sheds light on how security is actually addressed when dealing with cyber-physical systems from an automatic control perspective. The provided map of 138 selected studies is defined empirically and is based on, for instance, application fields, various system components, related algorithms and models, attacks characteristics and defense strategies. It presents a powerful comparison framework for existing and future research on this hot topic, important for both industry and academia

PLATFORM-DRIVEN CROWDSOURCED MANUFACTURING FOR MANUFACTURING AS A SERVICE

Platform-driven crowdsourced manufacturing is an emerging manufacturing paradigm to instantiate the adoption of the open business model in the context of achieving Manufacturing-as-a-Service (MaaS). It has attracted attention from both industries and academia as a powerful way of searching for manufacturing solutions extensively in a smart manufacturing era. In this regard, this work examines the origination and evolution of the open business model and highlights the trends towards platform-driven crowdsourced manufacturing as a solution for MaaS. Platform-driven crowdsourced manufacturing has a full function of value capturing, creation, and delivery approach, which is fulfilled by the cooperation among manufacturers, open innovators, and platforms. The platform-driven crowdsourced manufacturing workflow is proposed to organize these three decision agents by specifying the domains and interactions, following a functional, behavioral, and structural mapping model. A MaaS reference model is proposed to outline the critical functions and inter-relationships. A series of quantitative, qualitative, and computational solutions are developed for fulfilling the outlined functions. The case studies demonstrate the proposed methodologies and can pace the way towards a service-oriented product fulfillment process. This dissertation initially proposes a manufacturing theory and decision models by integrating manufacturer crowds through a cyber platform. This dissertation reveals the elementary conceptual framework based on stakeholder analysis, including dichotomy analysis of industrial applicability, decision agent identification, workflow, and holistic framework of platform-driven crowdsourced manufacturing. Three stakeholders require three essential service fields, and their cooperation requires an information service system as a kernel. These essential functions include contracting evaluation services for open innovators, manufacturers' task execution services, and platforms' management services. This research tackles these research challenges to provide a technology implementation roadmap and transition guidebook for industries towards crowdsourcing.Ph.D