Scenario trees and policy selection for multistage stochastic programming using machine learning

We propose a hybrid algorithmic strategy for complex stochastic optimization problems, which combines the use of scenario trees from multistage stochastic programming with machine learning techniques for learning a policy in the form of a statistical model, in the context of constrained vector-valued decisions. Such a policy allows one to run out-of-sample simulations over a large number of independent scenarios, and obtain a signal on the quality of the approximation scheme used to solve the multistage stochastic program. We propose to apply this fast simulation technique to choose the best tree from a set of scenario trees. A solution scheme is introduced, where several scenario trees with random branching structure are solved in parallel, and where the tree from which the best policy for the true problem could be learned is ultimately retained. Numerical tests show that excellent trade-offs can be achieved between run times and solution quality

Multi-stage stochastic optimization and reinforcement learning for forestry epidemic and covid-19 control planning

This dissertation focuses on developing new modeling and solution approaches based on multi-stage stochastic programming and reinforcement learning for tackling biological invasions in forests and human populations. Emerald Ash Borer (EAB) is the nemesis of ash trees. This research introduces a multi-stage stochastic mixed-integer programming model to assist forest agencies in managing emerald ash borer insects throughout the U.S. and maximize the public benets of preserving healthy ash trees. This work is then extended to present the first risk-averse multi-stage stochastic mixed-integer program in the invasive species management literature to account for extreme events. Significant computational achievements are obtained using a scenario dominance decomposition and cutting plane algorithm.The results of this work provide crucial insights and decision strategies for optimal resource allocation among surveillance, treatment, and removal of ash trees, leading to a better and healthier environment for future generations. This dissertation also addresses the computational difficulty of solving one of the most difficult classes of combinatorial optimization problems, the Multi-Dimensional Knapsack Problem (MKP). A novel 2-Dimensional (2D) deep reinforcement learning (DRL) framework is developed to represent and solve combinatorial optimization problems focusing on MKP. The DRL framework trains different agents for making sequential decisions and finding the optimal solution while still satisfying the resource constraints of the problem. To our knowledge, this is the first DRL model of its kind where a 2D environment is formulated, and an element of the DRL solution matrix represents an item of the MKP. Our DRL framework shows that it can solve medium-sized and large-sized instances at least 45 and 10 times faster in CPU solution time, respectively, with a maximum solution gap of 0.28% compared to the solution performance of CPLEX. Applying this methodology, yet another recent epidemic problem is tackled, that of COVID-19. This research investigates a reinforcement learning approach tailored with an agent-based simulation model to simulate the disease growth and optimize decision-making during an epidemic. This framework is validated using the COVID-19 data from the Center for Disease Control and Prevention (CDC). Research results provide important insights into government response to COVID-19 and vaccination strategies

The blind men and the elephant: Integrated offline/online optimization under uncertainty

open3noOptimization problems under uncertainty are traditionally solved either via offline or online methods. Offline approaches can obtain high-quality robust solutions, but have a considerable computational cost. Online algorithms can react to unexpected events once they are observed, but often run under strict time constraints, preventing the computation of optimal solutions. Many real world problems, however, have both offline and online elements: a substantial amount of time and information is frequently available (offline) before an online problem is solved (e.g. energy production forecasts, or historical travel times in routing problems); in other cases both offline (i.e. strategic) and online (i.e. operational) decisions need to be made. Surprisingly, the interplay of these offline and online phases has received little attention: like in the blind men and the elephant tale, we risk missing the whole picture, and the benefits that could come from integrated offline/online optimization. In this survey we highlight the potential shortcomings of pure methods when applied to mixed offline/online problems, we review the strategies that have been designed to take advantage of this integration, and we suggest directions for future research.openDe Filippo A.; Lombardi M.; Milano M.De Filippo A.; Lombardi M.; Milano M

A multi-stage stochastic programming for lot-sizing and scheduling under demand uncertainty

A stochastic lot-sizing and scheduling problem with demand uncertainty is studied in this paper. Lot-sizing determines the batch size for each product and scheduling decides the sequence of production. A multi-stage stochastic programming model is developed to minimize overall system costs including production cost, setup cost, inventory cost and backlog cost. We aim to find the optimal production sequence and resource allocation decisions. Demand uncertainty is represented by scenario trees using moment matching technique. Scenario reduction is used to select scenarios with the best representation of original set. A case study based on a manufacturing company has been conducted to illustrate and verify the model. We compared the two-stage stochastic programming model to the multi-stage stochastic programming model. The major motivation to adopt multi-stage stochastic programming models is that it extends the two-stage stochastic programming models by allowing revised decision at each period based on the previous realizations of uncertainty as well as decisions. Stability test and weak out-of-sample test are applied to find an appropriate scenario sample size. By using the multi-stage stochastic programming model, we improved the quality of solution by 10–13%

Dynamic Procurement of New Products with Covariate Information: The Residual Tree Method

Problem definition: We study the practice-motivated problem of dynamically procuring a new, short life-cycle product under demand uncertainty. The firm does not know the demand for the new product but has data on similar products sold in the past, including demand histories and covariate information such as product characteristics. Academic/practical relevance: The dynamic procurement problem has long attracted academic and practitioner interest, and we solve it in an innovative data-driven way with proven theoretical guarantees. This work is also the first to leverage the power of covariate data in solving this problem. Methodology:We propose a new, combined forecasting and optimization algorithm called the Residual Tree method, and analyze its performance via epi-convergence theory and computations. Our method generalizes the classical Scenario Tree method by using covariates to link historical data on similar products to construct demand forecasts for the new product. Results: We prove, under fairly mild conditions, that the Residual Tree method is asymptotically optimal as the size of the data set grows. We also numerically validate the method for problem instances derived using data from the global fashion retailer Zara. We find that ignoring covariate information leads to systematic bias in the optimal solution, translating to a 6–15% increase in the total cost for the problem instances under study. We also find that solutions based on trees using just 2–3 branches per node, which is common in the existing literature, are inadequate, resulting in 30–66% higher total costs compared with our best solution. Managerial implications: The Residual Tree is a new and generalizable approach that uses past data on similar products to manage new product inventories. We also quantify the value of covariate information and of granular demand modeling

Stochastic dynamic programming methods for the portfolio selection problem

In this thesis, we study the portfolio selection problem with multiple risky assets, linear transaction costs and a risk measure in a multi-period setting. In particular, we formulate the multi-period portfolio selection problem as a dynamic program and to solve it we construct approximate dynamic programming (ADP) algorithms, where we include Conditional-Value-at-Risk (CVaR) as a measure of risk, for different separable functional approximations of the value functions. We begin with the simple linear approximation which does not capture the nature of the portfolio selection problem since it ignores risk and leads to portfolios of only one asset. To improve it, we impose upper bound constraints on the holdings of the assets and we notice that we have more diversified portfolios. Then, we implement a piecewise linear approximation, for which we construct an update rule for the slopes of the approximate value functions that preserves concavity as well as the number of slopes. Unlike the simple linear approximation, in the piecewise linear approximation we notice that risk affects the composition of the selected portfolios. Further, unlike the linear approximation with upper bounds, here wealth flows naturally from one asset to another leading to diversified portfolios without us needing to impose any additional constraints on how much we can hold in each asset. For comparison, we consider existing portfolio selection methods, both myopic ones such as the equally weighted and a single-period portfolio models, and multi-period ones such as multistage stochastic programming. We perform extensive simulations using real-world equity data to evaluate the performance of all methods and compare all methods to a market Index. Computational results show that the piecewise linear ADP algorithm significantly outperforms the other methods as well as the market and runs in reasonable computational times. Comparative results of all methods are provided and some interesting conclusions are drawn especially when it comes to comparing the piecewise linear ADP algorithms with multistage stochastic programming

GPU-accelerated stochastic predictive control of drinking water networks

Despite the proven advantages of scenario-based stochastic model predictive control for the operational control of water networks, its applicability is limited by its considerable computational footprint. In this paper we fully exploit the structure of these problems and solve them using a proximal gradient algorithm parallelizing the involved operations. The proposed methodology is applied and validated on a case study: the water network of the city of Barcelona.Comment: 11 pages in double column, 7 figure

Integrated machine learning and optimization approaches

This dissertation focuses on the integration of machine learning and optimization. Specifically, novel machine learning-based frameworks are proposed to help solve a broad range of well-known operations research problems to reduce the solution times. The first study presents a bidirectional Long Short-Term Memory framework to learn optimal solutions to sequential decision-making problems. Computational results show that the framework significantly reduces the solution time of benchmark capacitated lot-sizing problems without much loss in feasibility and optimality. Also, models trained using shorter planning horizons can successfully predict the optimal solution of the instances with longer planning horizons. For the hardest data set, the predictions at the 25% level reduce the solution time of 70 CPU hours to less than 2 CPU minutes with an optimality gap of 0.8% and without infeasibility. In the second study, an extendable prediction-optimization framework is presented for multi-stage decision-making problems to address the key issues of sequential dependence, infeasibility, and generalization. Specifically, an attention-based encoder-decoder neural network architecture is integrated with an infeasibility-elimination and generalization framework to learn high-quality feasible solutions. The proposed framework is demonstrated to tackle the two well-known dynamic NP-Hard optimization problems: multi-item capacitated lot-sizing and multi-dimensional knapsack. The results show that models trained on shorter and smaller-dimension instances can be successfully used to predict longer and larger-dimension problems with the presented item-wise expansion algorithm. The solution time can be reduced by three orders of magnitude with an average optimality gap below 0.1%. The proposed framework can be advantageous for solving dynamic mixed-integer programming problems that need to be solved instantly and repetitively. In the third study, a deep reinforcement learning-based framework is presented for solving scenario-based two-stage stochastic programming problems, which are computationally challenging to solve. A general two-stage deep reinforcement learning framework is proposed where two learning agents sequentially learn to solve each stage of a general two-stage stochastic multi-dimensional knapsack problem. The results show that solution time can be reduced significantly with a relatively small gap. Additionally, decision-making agents can be trained with a few scenarios and solve problems with a large number of scenarios. In the fourth study, a learning-based prediction-optimization framework is proposed for solving scenario-based multi-stage stochastic programs. The issue of non-anticipativity is addressed with a novel neural network architecture that is based on a neural machine translation system. Furthermore, training the models on deterministic problems is suggested instead of solving hard and time-consuming stochastic programs. In this framework, the level of variables used for the solution is iteratively reduced to eliminate infeasibility, and a heuristic based on a linear relaxation is performed to reduce the solution time. An improved item-wise expansion strategy is introduced to generalize the algorithm to tackle instances with different sizes. The results are presented in solving stochastic multi-item capacitated lot-sizing and stochastic multi-stage multi-dimensional knapsack problems. The results show that the solution time can be reduced by a factor of 599 with an optimality gap of only 0.08%. Moreover, results demonstrate that the models can be used to predict similarly structured stochastic programming problems with a varying number of periods, items, and scenarios. The frameworks presented in this dissertation can be utilized to achieve high-quality and fast solutions to repeatedly-solved problems in various industrial and business settings, such as production and inventory management, capacity planning, scheduling, airline logistics, dynamic pricing, and emergency management