The boomerang returns? Accounting for the impact of uncertainties on the dynamics of remanufacturing systems

Recent years have witnessed companies abandon traditional open-loop supply chain structures in favour of closed-loop variants, in a bid to mitigate environmental impacts and exploit economic opportunities. Central to the closed-loop paradigm is remanufacturing: the restoration of used products to useful life. While this operational model has huge potential to extend product life-cycles, the collection and recovery processes diminish the effectiveness of existing control mechanisms for open-loop systems. We systematically review the literature in the field of closed-loop supply chain dynamics, which explores the time-varying interactions of material and information flows in the different elements of remanufacturing supply chains. We supplement this with further reviews of what we call the three ‘pillars’ of such systems, i.e. forecasting, collection, and inventory and production control. This provides us with an interdisciplinary lens to investigate how a ‘boomerang’ effect (i.e. sale, consumption, and return processes) impacts on the behaviour of the closed-loop system and to understand how it can be controlled. To facilitate this, we contrast closed-loop supply chain dynamics research to the well-developed research in each pillar; explore how different disciplines have accommodated the supply, process, demand, and control uncertainties; and provide insights for future research on the dynamics of remanufacturing systems

Evaluator services for optimised service placement in distributed heterogeneous cloud infrastructures

Optimal placement of demanding real-time interactive applications in a distributed heterogeneous cloud very quickly results in a complex tradeoff between the application constraints and resource capabilities. This requires very detailed information of the various requirements and capabilities of the applications and available resources. In this paper, we present a mathematical model for the service optimization problem and study the concept of evaluator services as a flexible and efficient solution for this complex problem. An evaluator service is a service probe that is deployed in particular runtime environments to assess the feasibility and cost-effectiveness of deploying a specific application in such environment. We discuss how this concept can be incorporated in a general framework such as the FUSION architecture and discuss the key benefits and tradeoffs for doing evaluator-based optimal service placement in widely distributed heterogeneous cloud environments

Batch Reinforcement Learning on the Industrial Benchmark: First Experiences

The Particle Swarm Optimization Policy (PSO-P) has been recently introduced and proven to produce remarkable results on interacting with academic reinforcement learning benchmarks in an off-policy, batch-based setting. To further investigate the properties and feasibility on real-world applications, this paper investigates PSO-P on the so-called Industrial Benchmark (IB), a novel reinforcement learning (RL) benchmark that aims at being realistic by including a variety of aspects found in industrial applications, like continuous state and action spaces, a high dimensional, partially observable state space, delayed effects, and complex stochasticity. The experimental results of PSO-P on IB are compared to results of closed-form control policies derived from the model-based Recurrent Control Neural Network (RCNN) and the model-free Neural Fitted Q-Iteration (NFQ). Experiments show that PSO-P is not only of interest for academic benchmarks, but also for real-world industrial applications, since it also yielded the best performing policy in our IB setting. Compared to other well established RL techniques, PSO-P produced outstanding results in performance and robustness, requiring only a relatively low amount of effort in finding adequate parameters or making complex design decisions

E-Fulfillment and Multi-Channel Distribution – A Review

This review addresses the specific supply chain management issues of Internet fulfillment in a multi-channel environment. It provides a systematic overview of managerial planning tasks and reviews corresponding quantitative models. In this way, we aim to enhance the understanding of multi-channel e-fulfillment and to identify gaps between relevant managerial issues and academic literature, thereby indicating directions for future research. One of the recurrent patterns in today’s e-commerce operations is the combination of ‘bricks-and-clicks’, the integration of e-fulfillment into a portfolio of multiple alternative distribution channels. From a supply chain management perspective, multi-channel distribution provides opportunities for serving different customer segments, creating synergies, and exploiting economies of scale. However, in order to successfully exploit these opportunities companies need to master novel challenges. In particular, the design of a multi-channel distribution system requires a constant trade-off between process integration and separation across multiple channels. In addition, sales and operations decisions are ever more tightly intertwined as delivery and after-sales services are becoming key components of the product offering.Distribution;E-fulfillment;Literature Review;Online Retailing

Economics of soil and water conservation

The Ethiopian highlands, inhabited by the vast majority of the Ethiopian human and livestock populations, are under continuous threat from soil erosion. Land degradation induced by soil erosion is considered to be among the major factors responsible for the recurrent malnutrition and famine problems in Ethiopia. Conservation efforts during recent decades have succeeded neither in triggering voluntary adoption of conservation practices nor in mitigating soil erosion problems. The purpose of this thesis is, therefore, to understand the socio-economic aspects underlying soil and water conservation decisions in the context of subsistence farmers in the Eastern Highlands of Ethiopia. In articles I, III, and IV, the farmers’ decision problem is modeled as a utility maximization problem, and econometric models are used to link the statistical model of observed data and the economic model. Stochastic dominance criteria are used, in article I, to determine whether adoption of a conservation practice results in higher expected grain yield and income and/or reduced variability. Limited dependent variable econometric models are used in articles III and IV in order to determine factors that influence farmers’ decisions on soil and water conservation, and their preference for types of development intervention. In article II, the decision problem is modeled as an intertemporal net benefit maximization problem, and a dynamic programming optimization model is applied to determine the optimal path of investment in soil and water conservation. Findings in article I suggest that conservation results in higher expected grain yield and income, but does not support the hypothesis that conservation unambiguously results in less variability than no-conservation. In article II, it is shown that the optimal path of investment in soil and water conservation depends on the discount rate and grain prices. The results also suggest that erosive agricultural practices yield higher return in the short-term, whereas conservation yields a higher and sustainable return in the long-term. The need to design incentive mechanisms that encourage farmers to have a longer planning horizon are among important suggestions proposed in articles I and II. Results, in article III, suggest that specific physical conditions of plots and socioeconomic characteristics of farm households influence the soil and water conservation decision behavior of farmers. Article IV suggests that the perceived priority of farmers with regard to agricultural problems and socio-economic characteristics, determines their preference for the type of development intervention. The results also suggest that there exists a complementarity between different interventions and hence a need to address them simultaneously to ensure a higher return from interventions. An important lesson to be drawn from articles III and IV is that differences in farming conditions and complementarities between policy programs need to be noted in any intervention program

A Model Approximation Scheme for Planning in Partially Observable Stochastic Domains

Partially observable Markov decision processes (POMDPs) are a natural model for planning problems where effects of actions are nondeterministic and the state of the world is not completely observable. It is difficult to solve POMDPs exactly. This paper proposes a new approximation scheme. The basic idea is to transform a POMDP into another one where additional information is provided by an oracle. The oracle informs the planning agent that the current state of the world is in a certain region. The transformed POMDP is consequently said to be region observable. It is easier to solve than the original POMDP. We propose to solve the transformed POMDP and use its optimal policy to construct an approximate policy for the original POMDP. By controlling the amount of additional information that the oracle provides, it is possible to find a proper tradeoff between computational time and approximation quality. In terms of algorithmic contributions, we study in details how to exploit region observability in solving the transformed POMDP. To facilitate the study, we also propose a new exact algorithm for general POMDPs. The algorithm is conceptually simple and yet is significantly more efficient than all previous exact algorithms.Comment: See http://www.jair.org/ for any accompanying file

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page