Multi-Objective Approaches to Markov Decision Processes with Uncertain Transition Parameters

Markov decision processes (MDPs) are a popular model for performance analysis and optimization of stochastic systems. The parameters of stochastic behavior of MDPs are estimates from empirical observations of a system; their values are not known precisely. Different types of MDPs with uncertain, imprecise or bounded transition rates or probabilities and rewards exist in the literature. Commonly, analysis of models with uncertainties amounts to searching for the most robust policy which means that the goal is to generate a policy with the greatest lower bound on performance (or, symmetrically, the lowest upper bound on costs). However, hedging against an unlikely worst case may lead to losses in other situations. In general, one is interested in policies that behave well in all situations which results in a multi-objective view on decision making. In this paper, we consider policies for the expected discounted reward measure of MDPs with uncertain parameters. In particular, the approach is defined for bounded-parameter MDPs (BMDPs) [8]. In this setting the worst, best and average case performances of a policy are analyzed simultaneously, which yields a multi-scenario multi-objective optimization problem. The paper presents and evaluates approaches to compute the pure Pareto optimal policies in the value vector space.Comment: 9 pages, 5 figures, preprint for VALUETOOLS 201

A similarity-based cooperative co-evolutionary algorithm for dynamic interval multi-objective optimization problems

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Dynamic interval multi-objective optimization problems (DI-MOPs) are very common in real-world applications. However, there are few evolutionary algorithms that are suitable for tackling DI-MOPs up to date. A framework of dynamic interval multi-objective cooperative co-evolutionary optimization based on the interval similarity is presented in this paper to handle DI-MOPs. In the framework, a strategy for decomposing decision variables is first proposed, through which all the decision variables are divided into two groups according to the interval similarity between each decision variable and interval parameters. Following that, two sub-populations are utilized to cooperatively optimize decision variables in the two groups. Furthermore, two response strategies, rgb0.00,0.00,0.00i.e., a strategy based on the change intensity and a random mutation strategy, are employed to rapidly track the changing Pareto front of the optimization problem. The proposed algorithm is applied to eight benchmark optimization instances rgb0.00,0.00,0.00as well as a multi-period portfolio selection problem and compared with five state-of-the-art evolutionary algorithms. The experimental results reveal that the proposed algorithm is very competitive on most optimization instances

Portfolio implementation risk management using evolutionary multiobjective optimization

Portfoliomanagementbasedonmean-varianceportfoliooptimizationissubjecttodifferent sources of uncertainty. In addition to those related to the quality of parameter estimates used in the optimization process, investors face a portfolio implementation risk. The potential temporary discrepancybetweentargetandpresentportfolios,causedbytradingstrategies,mayexposeinvestors to undesired risks. This study proposes an evolutionary multiobjective optimization algorithm aiming at regions with solutions more tolerant to these deviations and, therefore, more reliable. The proposed approach incorporates a user’s preference and seeks a fine-grained approximation of the most relevant efficient region. The computational experiments performed in this study are based on a cardinality-constrained problem with investment limits for eight broad-category indexes and 15 years of data. The obtained results show the ability of the proposed approach to address the robustness issue and to support decision making by providing a preferred part of the efficient set. The results reveal that the obtained solutions also exhibit a higher tolerance to prediction errors in asset returns and variance–covariance matrix.Sandra Garcia-Rodriguez and David Quintana acknowledge financial support granted by the Spanish Ministry of Economy and Competitivity under grant ENE2014-56126-C2-2-R. Roman Denysiuk and Antonio Gaspar-Cunha were supported by the Portuguese Foundation for Science and Technology under grant PEst-C/CTM/LA0025/2013 (Projecto Estratégico-LA 25-2013-2014-Strategic Project-LA 25-2013-2014).info:eu-repo/semantics/publishedVersio

A Neuroevolutionary Approach to Stochastic Inventory Control in Multi-Echelon Systems

Stochastic inventory control in multi-echelon systems poses hard problems in optimisation under uncertainty. Stochastic programming can solve small instances optimally, and approximately solve larger instances via scenario reduction techniques, but it cannot handle arbitrary nonlinear constraints or other non-standard features. Simulation optimisation is an alternative approach that has recently been applied to such problems, using policies that require only a few decision variables to be determined. However, to find optimal or near-optimal solutions we must consider exponentially large scenario trees with a corresponding number of decision variables. We propose instead a neuroevolutionary approach: using an artificial neural network to compactly represent the scenario tree, and training the network by a simulation-based evolutionary algorithm. We show experimentally that this method can quickly find high-quality plans using networks of a very simple form

Decision support for build-to-order supply chain management through multiobjective optimization

This paper aims to identify the gaps in decision-making support based on multiobjective optimization for build-to-order supply chain management (BTOSCM). To this end, it reviews the literature available on modelling build-to-order supply chains (BTO-SC) with the focus on adopting multiobjective optimization (MOO) techniques as a decision support tool. The literature has been classified based on the nature of the decisions in different part of the supply chain, and the key decision areas across a typical BTO-SC are discussed in detail. Available software packages suitable for supporting decision making in BTO supply chains are also identified and their related solutions are outlined. The gap between the modelling and optimization techniques developed in the literature and the decision support needed in practice are highlighted and future research directions to better exploit the decision support capabilities of MOO are proposed