The moderating impact of supply network topology on the effectiveness of risk management

While supply chain risk management offers a rich toolset for dealing with risk at the dyadic level, less attention has been given to the effectiveness of risk management in complex supply networks. We bridge this gap by building an agent based model to explore the relationship between topological characteristics of complex supply networks and their ability to recover through inventory mitigation and contingent rerouting. We simulate upstream supply networks, where each agent represents a supplier. Suppliers’ connectivity patterns are generated through random and preferential attachment models. Each supplier manages its inventory using an anchor-and-adjust ordering policy. We then randomly disrupt suppliers and observe how different topologies recover when risk management strategies are applied. Our results show that topology has a moderating effect on the effectiveness of risk management strategies. Scale-free supply networks generate lower costs, have higher fill-rates, and need less inventory to recover when exposed to random disruptions than random networks. Random networks need significantly more inventory distributed across the network to achieve the same fill rates as scale-free networks. Inventory mitigation improves fill-rate more than contingent rerouting regardless of network topology. Contingent rerouting is not effective for scale-free networks due to the low number of alternative suppliers, particularly for short-lasting disruptions. We also find that applying inventory mitigation to the most disrupted suppliers is only effective when the network is exposed to frequent disruptions; and not cost effective otherwise. Our work contributes to the emerging field of research on the relationship between complex supply network topology and resilience

Supply network science: Emergence of a new perspective on a classical field.

Supply networks emerge as companies procure goods from one another to produce their own products. Due to a chronic lack of data, studies on these emergent structures have long focussed on local neighbourhoods, assuming simple, chain-like structures. However, studies conducted since 2001 have shown that supply chains are indeed complex networks that exhibit similar organisational patterns to other network types. In this paper, we present a critical review of theoretical and model based studies which conceptualise supply chains from a network science perspective, showing that empirical data do not always support theoretical models that were developed, and argue that different industrial settings may present different characteristics. Consequently, a need that arises is the development and reconciliation of interpretation across different supply network layers such as contractual relations, material flow, financial links, and co-patenting, as these different projections tend to remain in disciplinary siloes. Other gaps include a lack of null models that show whether the observed properties are meaningful, a lack of dynamical models that can inform how layers evolve and adopt to changes, and a lack of studies that investigate how local decisions enable emergent outcomes. We conclude by asking the network science community to help bridge these gaps by engaging with this important area of research

Simulación de los efectos de la integración en una cadena de suministro

Existe un catálago muy amplio de herramientas de simulación. Estas herramientas pueden ser usadas para simulaciones en cualquier entorno, incluyendo el de la docencia. Beer Game, herramienta que permite simular una cadena de suministro, es usada en nuetra Escuela para la docencia, pero los tiempos actuales requerían una actualización. Una réplica de esta herramienta será la que se desarrolle a lo largo de este trabajo, desde la definición de sus políticas hasta su implementación en Excel, que será el sistema software bajo el que cuál funcionará la herramienta, por lo que resultará una aplicación de uso sencillo, en un entorno conocido e intuitivo. Por lo tanto, uno de los resultados de este trabajo se resume en el desarrollo de una herramienta que simula una cadena de suministro y que será principalmente usada como material de apoyo a la docencia. En la gestión de una cadena de suministro existe una gran cantidad de información que puede ser obviada por cada una de las entidades que participan en la cadena. Esta información puede ser de gran utilidad si es compartida por todos los miembros de la cadena para mejorar la eficiencia y así dar mejor servicio. Es aquí donde entra el papel de la integración. La integración de la información es un recurso muy utilizado en la actualidad gracias a la mejora contínua que ha tenido mediante el desarrollo de la tecnología y los resultados que genera. La importancia que ha obtenido la integración ha hecho que sea un campo sobre el que recae gran parte del peso de una empresa. Tras el desarrollo de la herramienta de simulación, se utilizará ésta para realizar una serie de simulaciones de gestión de una cadena de suministro sencilla en diferentes escenarios utilizando dos tipos de gestiones: gestión tradicional y gestión integrada. El fin es demostrar mediante los resultados obtenidos y de manera analítica las ventajas que se obtienen con la integración de una cadena de suministro.Universidad de Sevilla. Grado en Ingeniería de Tecnologías Industriale

Deep Learning and Reinforcement Learning for Inventory Control

RÉSUMÉ : La gestion d’inventaire est l’un des problèmes les plus importants dans la fabrication de produits. Les décisions de commande sont prises par des agents qui observent les demandes, stochastiques, ainsi que les informations locales tels que le niveau d’inventaire afin de prendre des décisions sur les prochaines valeurs de commande. Étant donné que l’inventaire sur place (la quantité disponible de stock en inventaire), les demandes non satisfaites (commandes en attente), et l’existence de commander sont coûteux, le problème d’optimisation est conçu afin de minimiser les coûts. Par conséquent, la fonction objective est de réduire le coût à long terme) dont les composantes sont des inventaires en stock, commandes en attente linéaires (pénalité), et des coûts de commandes fixes. Généralement, des algorithmes de processus de décision markovien, et de la programmation dynamique, ont été utilisés afin de résoudre le problème de contrôle d’inventaire. Ces algorithmes ont quelques désavantages. Ils sont conçus pour un environnement avec des informations disponibles, telles que la capacité de stockage ou elles imposent des limitations sur le nombre d’états. Résultat, les algorithmes du processus de décision markovien, et de la programmation dynamique sont inadéquats pour les situations mentionnées ci hauts, à cause de de la croissance exponentielle de l’espace d’état. En plus, les plus fameuses politique de getsion d’inventaire, telles que politiques standards et ne fonctionne que dans les systèmes où les demandes d’entrées obtiennent une distribution statistique connues. Afin de résoudre le problème, un apprentissage par renforcement approximée est développé dans le but d’éviter les défaillances mentionnées ci hauts. Ce projet applique une technique d’apprentissage de machine nommé ‘Deep Q-learning’, qui est capable d’apprendre des politiques de contrôle en utilisant directement le ‘end-to-end RL’, malgré le nombre énorme d’états. Aussi, le modèle est un ‘Deep Neural Network’ (DNN), formé avec une variante de ‘Q-learning’, dont l’entrée et la sortie sont l’information locale d’inventaire et la fonction de valeur utilisée pour estimer les récompenses futures, respectivement. Le Deep Q-learning, qui s’appelle ‘Deep Q-Network’ (DQN), est l’une des techniques pionnières ‘DRL’ qui inclut une approche à base de simulation dans laquelle les approximations d’actions sont menées en utilisant un réseau DNN. Le système prend des décisions sur les valeurs de commande. Étant donnée que la fonction de coût est calculée selon l’ordre ‘O’ et le niveau d’inventaire ‘IL’, les valeurs desquelles sont affectées par la demande ‘D’, la demande d’entrée ainsi que l’ordre et le niveau d’inventaire peuvent être considérés en tant qu’information individuelle d’inventaire. De plus, il y a un délai de mise en œuvre exprimant la latence dans l’envoi des informations et dans la réception des commandes. Le délai de mise en œuvre fournit davantage d’information locale incluant ‘IT’ et ‘OO’. Le ‘IT’ et ‘OO’ sont calculés et suivis durant les périodes de temps différents afin d’explorer plus d’informations sur l’environnement de l’agent d’inventaire. Par ailleurs, la principale information individuelle et la demande correspondante comprennent les états d’agents. Les systèmes ‘PO’ sont davantage observés dans les modèles à étapes multiples dont les agents peuvent ne pas être au courant de l’information individuelle des autres agents. Dans le but de créer une approche basée sur le ‘ML’ et fournir quelques aperçus dans la manière de résoudre le type d’agent multiple ‘PO’ du problème actuel de contrôle d’inventaire, un agent simple est étudié. Cet un agent examine si on peut mettre sur pied une technique ‘ML’ basée sur le ‘DL’ afin d’aider à trouver une décision de valeur de commande quasi optimale basée sur la demande et information individuelle sur une période à long terme. Afin de le réaliser, dans un premier temps, la différence entre la valeur de commande (action) et la demande comme résultat d’un ‘DNN’ est estimée. Ensuite, la commande est mise à jour basée sur la commande à jour et la demande suivante. Enfin, le coût total (récompense cumulative) dans chaque étape de temps est mis à jour. En conséquence, résoudre le problème de valeur de commande d’agent simple suffit pour diminuer le coût total sur le long terme. Le modèle développé est validé à l’aide de différents ratios des coefficients de coût. Aussi, le rendement de la présente méthode est considéré satisfaisant en comparaison avec le ‘RRL’ (RL de régression), la politique et le politique . Le RL de régression n’est pas capable d’apprendre aussi bien et avec autant de précision que le ‘DQN’. En dernier lieu, des recherches supplémentaires peuvent être menées afin d’observer les réseaux de chaînes d’approvisionnement multi-agents en série partiellement observables.----------ABSTRACT : Inventory control is one of the most significant problems in product manufacturing. A decision maker (agent) observes the random stochastic demands and local information of inventory such as inventory levels as its inputs to make decisions about the next ordering values as its actions. Since inventory on-hand (the available amount of stock in inventory), unmet demands (backorders), and the existence of ordering are costly, the optimization problem is designed to minimize the cost. As a result, the objective function is to reduce the long-run cost (cumulative reward) whose components are linear holding, linear backorder (penalty), and fixed ordering costs. Generally, Markov Decision Process (MDP) and Dynamic Programming (DP) algorithms have been utilized to solve the inventory control problem. These algorithms have some drawbacks. They are designed for the environment with available local information such as holding capacity or they impose limitations on the number of the states while these information and limitations are not available in some cases such as Partially Observable (PO) environments. As a result, DP or MDP algorithms are not suitable for the above-mentioned conditions due to the enormity of the state spaces. In addition, the most famous inventory management policies such as normal and policies are desirable only for the systems whose input demands obtain normal distribution. To solve the problem, an approximate Reinforcement Learning (RL) is developed so as to avoid having the afore-mentioned shortcomings. This project applies a Machine Leaning (ML) technique termed Deep Q-learning, which is able to learn control policies directly using end-to-end RL, even though the number of states is enormous. Also, the model is a Deep Neural Network (DNN), trained with a variant of Q-learning, whose input and output are the local information of inventory and the value function utilized to estimate future rewards, respectively. Deep Q-learning, which is also called Deep Q-Network (DQN), is one of the types of the pioneer Deep Reinforcement Learning (DRL) techniques that includes a simulation-based approach in which the action approximations are carried out using a Deep Neural Network (DNN). To end this, the agents observe the random stochastic demands and make decisions about the ordering values. Since the cost function is calculated in terms of Order (O) and Inventory Level (IL) whose values are affected by Demand (D), input demand as well as the order and inventory level can be considered as the individual information of the inventory. Also, there is a lead-time expressing the latency on sending information or receiving orders. The lead-time provides more local information including Inventory Transit (IT) and On-Order (OO). IT and OO are calculated and tracked during different time periods so as to explore more information about the environment of the inventory agent. Furthermore, the main individual information and the corresponding demand comprise the states of the agent. PO systems are observed more in multi-stage models whose agents can be unaware of the individual information of the other agents. In order to create a ML-based approach and provide some insight into how to resolve the PO multi-agent type of the present inventory control problem, a single-agent is studied. This agent examines if one can implement a ML technique based on Deep Learning (DL) to assist to learn near-optimal ordering value decision based on demand and individual information over long-run time. To achieve this, first, the difference between the ordering value (action) and demand as the output of a DNN is approximated. Then, the order is updated after observing the next demand. Next, the main individual information of the agent called input features of a DNN is updated based on the updated order and the following demand. Lastly, the total cost (cumulative reward) in each time step is updated. Accordingly, solving the ordering value problem of single-agent suffices to diminish the total cost over long-run time. The developed model is validated using different ratios of the cost coefficients. Also, the performance of the present method is found to be satisfactory in comparison with Regression Reinforcement Learning (Regression RL), policy, and policy. The regression RL is not able to learn as well and accurately as DQN. Finally, further research can be directed to solve the partial-observable multi-agent supply chain networks

Resilence of complex supply networks

During recent decades supply chains have grown, and became increasingly interconnected due to globalisation and outsourcing. Empirical and theoretical studies now characterise supply chains as complex networks rather than the hierarchical, linear chain structures often theorised in classical literature. Increased topological complexity resulted in an increased exposure to risk, however existing supply chain risk management methodologies are designed based on the linear structure assumption rather than interdependent network structures. There is a growing need to better understand the complexities of supply networks, and how to identify, measure and mitigate risks more efficiently. The aim of this thesis is to identify how supply network topology influences resilience. More specifically, how applying well-established supply chain risk management strategies can decrease disruption impact in different supply network topologies. The influence of supply network topology on resilience is captured using a dynamic agent-based model based on empirical and theoretical supply network structures, without a single entity controlling the whole system where each supplier is an independent decision-maker. These suppliers are then disrupted using various disruption scenarios. Suppliers in the network then apply inventory mitigation and contingent rerouting to decrease impact of disruptions on the rest of the network. To the best of author’s knowledge, this is the first time the impact of random disruptions and its reduction through risk management strategies in different supply network topologies have been assessed in a fully dynamic, interconnected environment. The main lessons from this work are as follows: It has been observed that the supply network topology plays a crucial role in reducing impact of disruptions. Some supply network topologies are more resilient to random disruptions as they better fulfil customer demand under perturbations. Under random disruptions, inventory mitigation is a well-performing shock absorption mechanism. Contingent rerouting, on the other hand, is a strategy that needs specific conditions to work well. Firstly, the strategy must be applied by companies in supply topologies where the majority of supply chain members have alternative suppliers. Secondly, contingent rerouting is only efficient in cases when the reaction time to supplier’s disruption is shorter than the duration of the disruption. It has also been observed that the topological position of the individual company who applies specific risk management strategy heavily impacts costs and fill-rates of the overall system. This property is moderated by other variables such as disruption duration, disruption frequency and the chosen risk management strategy. An additional, important lesson here is that, choosing the supplier that suffered the most from disruptions or have specific topological position in a network to apply a risk management strategy might not always decrease the costs incurred by the whole system. In contrast, it might increase it if not applied appropriately. This thesis underpins the significance of topology in supply network resilience. The results from this work are foundational to the claim that it is possible to design an extended supply network that will be able reduce the impact of certain disruption types. However, the design must consider topological properties as well as moderating variables.PhD in Manufacturin