Agent and cyber-physical system based self-organizing and self-adaptive intelligent shopfloor

The increasing demand of customized production results in huge challenges to the traditional manufacturing systems. In order to allocate resources timely according to the production requirements and to reduce disturbances, a framework for the future intelligent shopfloor is proposed in this paper. The framework consists of three primary models, namely the model of smart machine agent, the self-organizing model, and the self-adaptive model. A cyber-physical system for manufacturing shopfloor based on the multiagent technology is developed to realize the above-mentioned function models. Gray relational analysis and the hierarchy conflict resolution methods were applied to achieve the self-organizing and self-adaptive capabilities, thereby improving the reconfigurability and responsiveness of the shopfloor. A prototype system is developed, which has the adequate flexibility and robustness to configure resources and to deal with disturbances effectively. This research provides a feasible method for designing an autonomous factory with exception-handling capabilities

Budgeted Reinforcement Learning in Continuous State Space

A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints. It relies on a notion of risk implemented in the shape of a cost signal constrained to lie below an - adjustable - threshold. So far, BMDPs could only be solved in the case of finite state spaces with known dynamics. This work extends the state-of-the-art to continuous spaces environments and unknown dynamics. We show that the solution to a BMDP is a fixed point of a novel Budgeted Bellman Optimality operator. This observation allows us to introduce natural extensions of Deep Reinforcement Learning algorithms to address large-scale BMDPs. We validate our approach on two simulated applications: spoken dialogue and autonomous driving.Comment: N. Carrara and E. Leurent have equally contribute

An Agent-Based Approach to Self-Organized Production

The chapter describes the modeling of a material handling system with the production of individual units in a scheduled order. The units represent the agents in the model and are transported in the system which is abstracted as a directed graph. Since the hindrances of units on their path to the destination can lead to inefficiencies in the production, the blockages of units are to be reduced. Therefore, the units operate in the system by means of local interactions in the conveying elements and indirect interactions based on a measure of possible hindrances. If most of the units behave cooperatively ("socially"), the blockings in the system are reduced. A simulation based on the model shows the collective behavior of the units in the system. The transport processes in the simulation can be compared with the processes in a real plant, which gives conclusions about the consequencies for the production based on the superordinate planning.Comment: For related work see http://www.soms.ethz.c