A Practical Guide to Multi-Objective Reinforcement Learning and Planning

Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems

A practical guide to multi-objective reinforcement learning and planning

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems. © 2022, The Author(s)

Determining Feasibility Resilience: Set Based Design Iteration Evaluation Through Permutation Stability Analysis

The goal of robust design is to select a design that will still perform satisfactorily even with unexpected variation in design parameters. A resilient design will accommodate unanticipated future system requirements. Through studying the variations of system parameters through the use of multi objective optimization, a designer hopes to locate a robustly resilient design, which performs current mission well even with varying system parameters and is able to be easily repurposed to new missions. This ability to withstand changes is critical because it is common for the product of a design to undergo changes throughout its life cycle. This subject has been an active area of research in industrial design and systems engineering but most methodologies rest upon exhaustive understanding of design, manufacturing and mission variance. The thrust of this research is to develop new methodologies for estimating robust resilience given imperfect information. In this work we will apply new methodologies for locating resilient designs within a dataset derive from a study performed by the Small Surface Combatant Task Force in order to improve upon a state of the art design process. Two new methodologies, permutation stability analysis and mutation stability analysis, are presented along with results and discussion as applied to the SSCTF dataset. It is demonstrated that these new methods improve upon the state of the art by providing insight into the robustness and resilience of selected system properties. These methodologies, although applied to the SSCTF dataset are posed more generally for wider application in system design

Robust Solutions to Uncertain Multiobjective Programs

Decision making in the presence of uncertainty and multiple conflicting objec-tives is a real-life issue, especially in the fields of engineering, public policy making, business management, and many others. The conflicting goals may originate from the variety of ways to assess a system’s performance such as cost, safety, and affordability, while uncertainty may result from inaccurate or unknown data, limited knowledge, or future changes in the environment. To address optimization problems that incor-porate these two aspects, we focus on the integration of robust and multiobjective optimization. Although the uncertainty may present itself in many different ways due to a diversity of sources, we address the situation of objective-wise uncertainty only in the coefficients of the objective functions, which is drawn from a finite set of scenarios. Among the numerous concepts of robust solutions that have been proposed and de-veloped, we concentrate on a strict concept referred to as highly robust efficiency in which a feasible solution is highly robust efficient provided that it is efficient with respect to every realization of the uncertain data. The main focus of our study is uncertain multiobjective linear programs (UMOLPs), however, nonlinear problems are discussed as well. In the course of our study, we develop properties of the highly robust efficient set, provide its characterization using the cone of improving directions associated with the UMOLP, derive several bound sets on the highly robust efficient set, and present a robust counterpart for a class of UMOLPs. As various results rely on the polar and strict polar of the cone of improving directions, as well as the acuteness of this cone, we derive properties and closed-form representations of the (strict) polar and also propose methods to verify the property of acuteness. Moreover, we undertake the computation of highly robust efficient solutions. We provide methods for checking whether or not the highly robust efficient set is empty, computing highly robust efficient points, and determining whether a given solution of interest is highly robust efficient. An application in the area of bank management is included