832 research outputs found
Scenario trees and policy selection for multistage stochastic programming using machine learning
We propose a hybrid algorithmic strategy for complex stochastic optimization
problems, which combines the use of scenario trees from multistage stochastic
programming with machine learning techniques for learning a policy in the form
of a statistical model, in the context of constrained vector-valued decisions.
Such a policy allows one to run out-of-sample simulations over a large number
of independent scenarios, and obtain a signal on the quality of the
approximation scheme used to solve the multistage stochastic program. We
propose to apply this fast simulation technique to choose the best tree from a
set of scenario trees. A solution scheme is introduced, where several scenario
trees with random branching structure are solved in parallel, and where the
tree from which the best policy for the true problem could be learned is
ultimately retained. Numerical tests show that excellent trade-offs can be
achieved between run times and solution quality
Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization
In this paper, we study a class of stochastic optimization problems, referred
to as the \emph{Conditional Stochastic Optimization} (CSO), in the form of
\min_{x \in \mathcal{X}}
\EE_{\xi}f_\xi\Big({\EE_{\eta|\xi}[g_\eta(x,\xi)]}\Big), which finds a wide
spectrum of applications including portfolio selection, reinforcement learning,
robust learning, causal inference and so on. Assuming availability of samples
from the distribution \PP(\xi) and samples from the conditional distribution
\PP(\eta|\xi), we establish the sample complexity of the sample average
approximation (SAA) for CSO, under a variety of structural assumptions, such as
Lipschitz continuity, smoothness, and error bound conditions. We show that the
total sample complexity improves from \cO(d/\eps^4) to \cO(d/\eps^3) when
assuming smoothness of the outer function, and further to \cO(1/\eps^2) when
the empirical function satisfies the quadratic growth condition. We also
establish the sample complexity of a modified SAA, when and are
independent. Several numerical experiments further support our theoretical
findings.
Keywords: stochastic optimization, sample average approximation, large
deviations theoryComment: Typo corrected. Reference added. Revision comments handle
The State-of-the-Art Survey on Optimization Methods for Cyber-physical Networks
Cyber-Physical Systems (CPS) are increasingly complex and frequently
integrated into modern societies via critical infrastructure systems, products,
and services. Consequently, there is a need for reliable functionality of these
complex systems under various scenarios, from physical failures due to aging,
through to cyber attacks. Indeed, the development of effective strategies to
restore disrupted infrastructure systems continues to be a major challenge.
Hitherto, there have been an increasing number of papers evaluating
cyber-physical infrastructures, yet a comprehensive review focusing on
mathematical modeling and different optimization methods is still lacking.
Thus, this review paper appraises the literature on optimization techniques for
CPS facing disruption, to synthesize key findings on the current methods in
this domain. A total of 108 relevant research papers are reviewed following an
extensive assessment of all major scientific databases. The main mathematical
modeling practices and optimization methods are identified for both
deterministic and stochastic formulations, categorizing them based on the
solution approach (exact, heuristic, meta-heuristic), objective function, and
network size. We also perform keyword clustering and bibliographic coupling
analyses to summarize the current research trends. Future research needs in
terms of the scalability of optimization algorithms are discussed. Overall,
there is a need to shift towards more scalable optimization solution
algorithms, empowered by data-driven methods and machine learning, to provide
reliable decision-support systems for decision-makers and practitioners
Multistage stochastic bid model for a wind-thermal power producer
This master thesis explore different multi-stage stochastic programming models for electricity generation companies to find optimal bid functions in electric spot markets. The explored models not only capture the uncertainty of prices of different markets and financial products, but also couples together wind and thermal generation units, offering producers that combine both technologies a more suitable approach to find their best possible bidding strategy among the space of possible actions
Integrated machine learning and optimization approaches
This dissertation focuses on the integration of machine learning and optimization. Specifically, novel machine learning-based frameworks are proposed to help solve a broad range of well-known operations research problems to reduce the solution times. The first study presents a bidirectional Long Short-Term Memory framework to learn optimal solutions to sequential decision-making problems. Computational results show that the framework significantly reduces the solution time of benchmark capacitated lot-sizing problems without much loss in feasibility and optimality. Also, models trained using shorter planning horizons can successfully predict the optimal solution of the instances with longer planning horizons. For the hardest data set, the predictions at the 25% level reduce the solution time of 70 CPU hours to less than 2 CPU minutes with an optimality gap of 0.8% and without infeasibility. In the second study, an extendable prediction-optimization framework is presented for multi-stage decision-making problems to address the key issues of sequential dependence, infeasibility, and generalization. Specifically, an attention-based encoder-decoder neural network architecture is integrated with an infeasibility-elimination and generalization framework to learn high-quality feasible solutions. The proposed framework is demonstrated to tackle the two well-known dynamic NP-Hard optimization problems: multi-item capacitated lot-sizing and multi-dimensional knapsack. The results show that models trained on shorter and smaller-dimension instances can be successfully used to predict longer and larger-dimension problems with the presented item-wise expansion algorithm. The solution time can be reduced by three orders of magnitude with an average optimality gap below 0.1%. The proposed framework can be advantageous for solving dynamic mixed-integer programming problems that need to be solved instantly and repetitively. In the third study, a deep reinforcement learning-based framework is presented for solving scenario-based two-stage stochastic programming problems, which are computationally challenging to solve. A general two-stage deep reinforcement learning framework is proposed where two learning agents sequentially learn to solve each stage of a general two-stage stochastic multi-dimensional knapsack problem. The results show that solution time can be reduced significantly with a relatively small gap. Additionally, decision-making agents can be trained with a few scenarios and solve problems with a large number of scenarios. In the fourth study, a learning-based prediction-optimization framework is proposed for solving scenario-based multi-stage stochastic programs. The issue of non-anticipativity is addressed with a novel neural network architecture that is based on a neural machine translation system. Furthermore, training the models on deterministic problems is suggested instead of solving hard and time-consuming stochastic programs. In this framework, the level of variables used for the solution is iteratively reduced to eliminate infeasibility, and a heuristic based on a linear relaxation is performed to reduce the solution time. An improved item-wise expansion strategy is introduced to generalize the algorithm to tackle instances with different sizes. The results are presented in solving stochastic multi-item capacitated lot-sizing and stochastic multi-stage multi-dimensional knapsack problems. The results show that the solution time can be reduced by a factor of 599 with an optimality gap of only 0.08%. Moreover, results demonstrate that the models can be used to predict similarly structured stochastic programming problems with a varying number of periods, items, and scenarios. The frameworks presented in this dissertation can be utilized to achieve high-quality and fast solutions to repeatedly-solved problems in various industrial and business settings, such as production and inventory management, capacity planning, scheduling, airline logistics, dynamic pricing, and emergency management
The Voice of Optimization
We introduce the idea that using optimal classification trees (OCTs) and
optimal classification trees with-hyperplanes (OCT-Hs), interpretable machine
learning algorithms developed by Bertsimas and Dunn [2017, 2018], we are able
to obtain insight on the strategy behind the optimal solution in continuous and
mixed-integer convex optimization problem as a function of key parameters that
affect the problem. In this way, optimization is not a black box anymore.
Instead, we redefine optimization as a multiclass classification problem where
the predictor gives insights on the logic behind the optimal solution. In other
words, OCTs and OCT-Hs give optimization a voice. We show on several realistic
examples that the accuracy behind our method is in the 90%-100% range, while
even when the predictions are not correct, the degree of suboptimality or
infeasibility is very low. We compare optimal strategy predictions of OCTs and
OCT-Hs and feedforward neural networks (NNs) and conclude that the performance
of OCT-Hs and NNs is comparable. OCTs are somewhat weaker but often
competitive. Therefore, our approach provides a novel insightful understanding
of optimal strategies to solve a broad class of continuous and mixed-integer
optimization problems
PARL: A Unified Framework for Policy Alignment in Reinforcement Learning
We present a novel unified bilevel optimization-based framework,
\textsf{PARL}, formulated to address the recently highlighted critical issue of
policy alignment in reinforcement learning using utility or preference-based
feedback. We identify a major gap within current algorithmic designs for
solving policy alignment due to a lack of precise characterization of the
dependence of the alignment objective on the data generated by policy
trajectories. This shortfall contributes to the sub-optimal performance
observed in contemporary algorithms. Our framework addressed these concerns by
explicitly parameterizing the distribution of the upper alignment objective
(reward design) by the lower optimal variable (optimal policy for the designed
reward). Interestingly, from an optimization perspective, our formulation leads
to a new class of stochastic bilevel problems where the stochasticity at the
upper objective depends upon the lower-level variable. To demonstrate the
efficacy of our formulation in resolving alignment issues in RL, we devised an
algorithm named \textsf{A-PARL} to solve PARL problem, establishing sample
complexity bounds of order . Our empirical results
substantiate that the proposed \textsf{PARL} can address the alignment concerns
in RL by showing significant improvements (up to 63\% in terms of required
samples) for policy alignment in large-scale environments of the Deepmind
control suite and Meta world tasks
International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book
The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions.
This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more
- …