Processing second-order stochastic dominance models using cutting-plane representations

This is the post-print version of the Article. The official published version can be accessed from the links below. Copyright @ 2011 Springer-VerlagSecond-order stochastic dominance (SSD) is widely recognised as an important decision criterion in portfolio selection. Unfortunately, stochastic dominance models are known to be very demanding from a computational point of view. In this paper we consider two classes of models which use SSD as a choice criterion. The first, proposed by Dentcheva and Ruszczyński (J Bank Finance 30:433–451, 2006), uses a SSD constraint, which can be expressed as integrated chance constraints (ICCs). The second, proposed by Roman et al. (Math Program, Ser B 108:541–569, 2006) uses SSD through a multi-objective formulation with CVaR objectives. Cutting plane representations and algorithms were proposed by Klein Haneveld and Van der Vlerk (Comput Manage Sci 3:245–269, 2006) for ICCs, and by Künzi-Bay and Mayer (Comput Manage Sci 3:3–27, 2006) for CVaR minimization. These concepts are taken into consideration to propose representations and solution methods for the above class of SSD based models. We describe a cutting plane based solution algorithm and outline implementation details. A computational study is presented, which demonstrates the effectiveness and the scale-up properties of the solution algorithm, as applied to the SSD model of Roman et al. (Math Program, Ser B 108:541–569, 2006).This study was funded by OTKA, Hungarian National Fund for Scientific Research, project 47340; by Mobile Innovation Centre, Budapest University of Technology, project 2.2; Optirisk Systems, Uxbridge, UK and by BRIEF (Brunel University Research Innovation and Enterprise Fund)

Risk-Sensitive Reinforcement Learning: A Constrained Optimization Viewpoint

The classic objective in a reinforcement learning (RL) problem is to find a policy that minimizes, in expectation, a long-run objective such as the infinite-horizon discounted or long-run average cost. In many practical applications, optimizing the expected value alone is not sufficient, and it may be necessary to include a risk measure in the optimization process, either as the objective or as a constraint. Various risk measures have been proposed in the literature, e.g., mean-variance tradeoff, exponential utility, the percentile performance, value at risk, conditional value at risk, prospect theory and its later enhancement, cumulative prospect theory. In this article, we focus on the combination of risk criteria and reinforcement learning in a constrained optimization framework, i.e., a setting where the goal to find a policy that optimizes the usual objective of infinite-horizon discounted/average cost, while ensuring that an explicit risk constraint is satisfied. We introduce the risk-constrained RL framework, cover popular risk measures based on variance, conditional value-at-risk and cumulative prospect theory, and present a template for a risk-sensitive RL algorithm. We survey some of our recent work on this topic, covering problems encompassing discounted cost, average cost, and stochastic shortest path settings, together with the aforementioned risk measures in a constrained framework. This non-exhaustive survey is aimed at giving a flavor of the challenges involved in solving a risk-sensitive RL problem, and outlining some potential future research directions

Bi-objective facility location in the presence of uncertainty

Multiple and usually conflicting objectives subject to data uncertainty are main features in many real-world problems. Consequently, in practice, decision-makers need to understand the trade-off between the objectives, considering different levels of uncertainty in order to choose a suitable solution. In this paper, we consider a two-stage bi-objective single source capacitated model as a base formulation for designing a last-mile network in disaster relief where one of the objectives is subject to demand uncertainty. We analyze scenario-based two-stage risk-neutral stochastic programming, adaptive (two-stage) robust optimization, and a two-stage risk-averse stochastic approach using conditional value-at-risk (CVaR). To cope with the bi-objective nature of the problem, we embed these concepts into two criterion space search frameworks, the $\epsilon$-constraint method and the balanced box method, to determine the Pareto frontier. Additionally, a matheuristic technique is developed to obtain high-quality approximations of the Pareto frontier for large-size instances. In an extensive computational experiment, we evaluate and compare the performance of the applied approaches based on real-world data from a Thies drought case, Senegal

A mean-risk mixed integer nonlinear program for transportation network protection

This paper focuses on transportation network protection to hedge against extreme events such as earthquakes. Traditional two-stage stochastic programming has been widely adopted to obtain solutions under a risk-neutral preference through the use of expectations in the recourse function. In reality, decision makers hold different risk preferences. We develop a mean-risk two-stage stochastic programming model that allows for greater flexibility in handling risk preferences when allocating limited resources. In particular, the first stage minimizes the retrofitting cost by making strategic retrofit decisions whereas the second stage minimizes the travel cost. The conditional value-at-risk (CVaR) is included as the risk measure for the total system cost. The two-stage model is equivalent to a nonconvex mixed integer nonlinear program (MINLP). To solve this model using the Generalized Benders Decomposition (GBD) method, we derive a convex reformulation of the second-stage problem to overcome algorithmic challenges embedded in the non-convexity, nonlinearity, and non-separability of first- and second-stage variables. The model is used for developing retrofit strategies for networked highway bridges, which is one of the research areas that can significantly benefit from mean-risk models. We first justify the model using a hypothetical nine-node network. Then we evaluate our decomposition algorithm by applying the model to the Sioux Falls network, which is a large-scale benchmark network in the transportation research community. The effects of the chosen risk measure and critical parameters on optimal solutions are empirically explored

Robust Modeling Framework for Transportation Infrastructure System Protection Under Uncertainty

This dissertation presents a modelling framework that will be useful for decision makers at federal and state levels to establish efficient resource allocation schemes to transportation infrastructures on both strategic and tactical levels. In particular, at the upper level, the highway road network carries traffic flows that rely on the performance of individual bridge infrastructure which is optimized through robust design at lower level. A system optimization model is developed to allocate resources to infrastructure systems considering traffic impact, which aims to reduce infrastructure rehabilitation cost, long term economic cost including travel delays due to realization of future natural disasters such as earthquakes. At the lower level, robust design for each individual bridge is confined by the resources allocated from upper level network optimization model, where optimal rehabilitation strategies are selected to improve its resiliency to hedge against potential disasters. The above two decision making processes are interdependent, thus should not be treated separately. Thus, the resultant modeling framework will be a step forward in the disaster management for transportation infrastructure network. This dissertation first presents a novel formulation and a solution algorithm of network level resource allocation problem. A mean-risk two-stage stochastic programming model is developed with the first-stage considering resources allocation and second-stages shows the response from system travel delays, where the conditional value-at-risk (CVaR) is specified as the risk measure. A decomposition method based on generalized Benders decomposition is developed to solve the model, with a concerted effort on overcoming the algorithmic challenges imbedded in non-convexity, nonlinearity and non-separability of first- and second- stage variables. The network level model focusing on traffic optimization is further integrated into a bi-level modeling framework. For lower level, a method using finite element analysis to generate a nonlinear relationship between structural performances of bridges and retrofit levels. This relationship was converted to traffic capacity-cost relationship and used as an input for the upper-level model. Results from the Sioux Falls transportation network demonstrated that the integration of both network and FE modeling for individual structure enhanced the effectiveness of retrofit strategies, compared to linear traffic capacity-cost estimation and conventional engineering practice which prioritizes bridges according to the severity of expected damages of bridges. This dissertation also presents a minimax regret formulation of network protection problem that is integrated with earthquake simulations. The lower level model incorporates a seismic analysis component into the framework such that bridge columns are subject to a set of ground motions. Results of seismic response of bridge structures are used to develop a Pareto front of cost-safety-robustness relationship from which bridge damage scenarios are generated as an input of the network level model