Enhancing Dynamic Production Scheduling And Resource Allocation Through Adaptive Control Systems With Deep Reinforcement Learning

Traditional production scheduling and resource allocation methods often struggle to adapt to changing conditions in manufacturing environments. To address this challenge, this study leverages an adaptive control system integrated with a Deep Deterministic Policy Gradient (DDPG) alongside a particle swarm optimization algorithm to enable real-time production scheduling and allocation of resources. The system continuously learns from generated production data and adjusts production schedules with resource allocations based on evolving conditions such as demand fluctuations and resource availability. By harnessing the capabilities of Deep Reinforcement learning, the proposed approach of applying the DDPG algorithm to simulate the environment improves production efficiency, minimizes delays, and optimizes resource utilization. Through conducted experiments, the effectiveness of the DDPG-Particle Swarm Optimization technique (DRPO) was demonstrated in enhancing dynamic production scheduling and resource allocation in simulated manufacturing settings. This study presents a significant step towards intelligent, self-improving production control systems that can navigate complex and dynamic manufacturing environments

Self-Learning Cloud Controllers: Fuzzy Q-Learning for Knowledge Evolution

Cloud controllers aim at responding to application demands by automatically scaling the compute resources at runtime to meet performance guarantees and minimize resource costs. Existing cloud controllers often resort to scaling strategies that are codified as a set of adaptation rules. However, for a cloud provider, applications running on top of the cloud infrastructure are more or less black-boxes, making it difficult at design time to define optimal or pre-emptive adaptation rules. Thus, the burden of taking adaptation decisions often is delegated to the cloud application. Yet, in most cases, application developers in turn have limited knowledge of the cloud infrastructure. In this paper, we propose learning adaptation rules during runtime. To this end, we introduce FQL4KE, a self-learning fuzzy cloud controller. In particular, FQL4KE learns and modifies fuzzy rules at runtime. The benefit is that for designing cloud controllers, we do not have to rely solely on precise design-time knowledge, which may be difficult to acquire. FQL4KE empowers users to specify cloud controllers by simply adjusting weights representing priorities in system goals instead of specifying complex adaptation rules. The applicability of FQL4KE has been experimentally assessed as part of the cloud application framework ElasticBench. The experimental results indicate that FQL4KE outperforms our previously developed fuzzy controller without learning mechanisms and the native Azure auto-scaling

Enhancing the Supply Chain Performance by Integrating Simulated and Physical Agents into Organizational Information Systems

As the business environment gets more complicated, organizations must be able to respond to the business changes and adjust themselves quickly to gain their competitive advantages. This study proposes an integrated agent system, called SPA, which coordinates simulated and physical agents to provide an efficient way for organizations to meet the challenges in managing supply chains. In the integrated framework, physical agents coordinate with inter-organizations\' physical agents to form workable business processes and detect the variations occurring in the outside world, whereas simulated agents model and analyze the what-if scenarios to support physical agents in making decisions. This study uses a supply chain that produces digital still cameras as an example to demonstrate how the SPA works. In this example, individual information systems of the involved companies equip with the SPA and the entire supply chain is modeled as a hierarchical object oriented Petri nets. The SPA here applies the modified AGNES data clustering technique and the moving average approach to help each firm generalize customers\' past demand patterns and forecast their future demands. The amplitude of forecasting errors caused by bullwhip effects is used as a metric to evaluate the degree that the SPA affects the supply chain performance. The experimental results show that the SPA benefits the entire supply chain by reducing the bullwhip effects and forecasting errors in a dynamic environment.Supply Chain Performance Enhancement; Bullwhip Effects; Simulated Agents; Physical Agents; Dynamic Customer Demand Pattern Discovery

RAMARL: Robustness Analysis with Multi-Agent Reinforcement Learning - Robust Reasoning in Autonomous Cyber-Physical Systems

A key driver to offering smart services is an infrastructure of Cyber-Physical systems (CPS)s. By definition, CPSs are intertwined physical and computational components that integrate physical behaviour with computation. The reason is to autonomously execute a task or a set of tasks providing a service or a list of end-users services. In real-life applications, CPSs operate in dynamically changing surroundings characterized by unexpected or unpredictable situations. Such operations involve complex interactions between multiple intelligent agents in a highly non-stationary environment. For safety reasons, a CPS should withstand a certain amount of disruption and exert the operations in a stable and robust manner when performing complex tasks. Recent advances in reinforcement learning have proven suitable for enabling multi-agents to robustly adapt to their environment, yet they often depend on a massive amount of training data and experiences. In these cases, robustness analysis outlines necessary components and specifications in a framework, ensuring reliable and stable behaviour while considering the dynamicity of the environment. This paper presents a combination of multi-agent reinforcement learning with robustness analysis shaping a cyber-physical system infrastructure that reasons robustly in a dynamically changing environment. The combination strengthens the reinforcement learning, increasing the reliability and flexibility of the system by applying robustness analysis. Robustness analysis identifies vulnerability issues when the system interacts within a dynamically changing environment. Based on this identification, when incorporated into the system, robustness analysis suggests robust solutions and actions rather than optimal ones provided by reinforcement learning alone. Results from the combination show that this infrastructure can enable reliable operations with the flexibility to adapt to the changing environment dynamics.publishedVersio

Learning Team-Based Navigation: A Review of Deep Reinforcement Learning Techniques for Multi-Agent Pathfinding

Multi-agent pathfinding (MAPF) is a critical field in many large-scale robotic applications, often being the fundamental step in multi-agent systems. The increasing complexity of MAPF in complex and crowded environments, however, critically diminishes the effectiveness of existing solutions. In contrast to other studies that have either presented a general overview of the recent advancements in MAPF or extensively reviewed Deep Reinforcement Learning (DRL) within multi-agent system settings independently, our work presented in this review paper focuses on highlighting the integration of DRL-based approaches in MAPF. Moreover, we aim to bridge the current gap in evaluating MAPF solutions by addressing the lack of unified evaluation metrics and providing comprehensive clarification on these metrics. Finally, our paper discusses the potential of model-based DRL as a promising future direction and provides its required foundational understanding to address current challenges in MAPF. Our objective is to assist readers in gaining insight into the current research direction, providing unified metrics for comparing different MAPF algorithms and expanding their knowledge of model-based DRL to address the existing challenges in MAPF.Comment: 36 pages, 10 figures, published in Artif Intell Rev 57, 41 (2024

A Survey and Analysis of Multi-Robot Coordination

International audienceIn the field of mobile robotics, the study of multi-robot systems (MRSs) has grown significantly in size and importance in recent years. Having made great progress in the development of the basic problems concerning single-robot control, many researchers shifted their focus to the study of multi-robot coordination. This paper presents a systematic survey and analysis of the existing literature on coordination, especially in multiple mobile robot systems (MMRSs). A series of related problems have been reviewed, which include a communication mechanism, a planning strategy and a decision-making structure. A brief conclusion and further research perspectives are given at the end of the paper

Algoritmo bioinspirado a redes de robots para la asistencia en operaciones de busqueda y rescate

ilustraciones, diagramas, fotografíasThis thesis proposes a bio-inspired algorithm for robot networks assisting in the operations of search and rescue scenarios. We consider ants as social animals to study and abstract beha- viors that can be useful in the framework of search and rescue using robots. We consider three main topics to address when using robots to assist rescuers. First, the exploration and mapping of the disaster zones. For this, we consider the mecha- nisms and interactions of ants to explore their environment, look for food, avoid predators, and explore better places to establish a nest. Then, we deploy robots to explore the en- vironment and discourage robots from entering regions other robots have explored using pheromones as markers for the robots. We also abstract the randomness ants use to explore and implement a Q-learning algorithm that allows robots to explore unvisited regions. Second, the navigation and victim detection. Once the environment has been explored, we vi use Reynolds rules to allow the navigation of robots to create cohesion, attraction to target goals, and repulsion to obstacles and inter-agent collisions. Then, we use a neural network to determine whether what robots are detecting is a victim. Lastly, we use a consensus-like approach to classify victims or no victims based on distributed information. Lastly, ants have been famous for carrying loads that surpass their size and payload capacity by cooperating. We consider quadrotors to carry loads cooperatively that can be medical supplies or victims in search and rescue (Texto tomado de la fuente)Esta tesis propone un algoritmo bioinspirado para redes de robots que asisten en las operaciones de escenarios de busqueda y rescate. Consideramos a las hormigas como animales sociales para estudiar y abstraer comportamientos que pueden ser utiles en el marco de la busqueda y rescate mediante robots. Consideramos tres temas principales para abordar cuando se utilizan robots para ayudar a los rescatistas. Primero, la exploracion y mapeo de las zonas de desastre. Para esto, consideramos los mecanismos e interacciones de las hormigas para explorar su entorno, buscar comida, evitar depredadores y explorar mejores lugares para establecer un nido. Luego, desplegamos robots para explorar el entorno y disuadimos a los robots de ingresar a regiones que otros robots han explorado usando feromonas como marcadores para los robots. Tambien abstraemos la aleatoriedad que usan las hormigas para explorar e implementar un algoritmo Q-learning que permite a los robots explorar regiones no visitadas. En segundo lugar, la navegacion y deteccion de vıctimas. Una vez que se ha explorado el entorno, usamos las reglas de Reynolds para permitir que la navegacion de los robots cree cohesion, atraccion hacia los objetivos y repulsion hacia los obstaculos y las colisiones entre agentes. Luego, usamos una red neuronal para determinar si lo que detectan los robots es una vıctima. Por ultimo, utilizamos un enfoque de consenso para clasificar a las vıctimas o no vıctimas en funcion de la informacion distribuida. Por ultimo, las hormigas han sido famosas por llevar cargas que superan su tamano y capacidad de carga al cooperar. Consideramos quadrotors para transportar cargas de manera cooperativa que pueden ser suministros medicos o vıctimas en busqueda y rescate.MaestríaMagister en Ingenieria - Automatizacion IndustrialRobotic