MONISE - Many Objective Non-Inferior Set Estimation

This work proposes a novel many objective optimization approach that globally finds a set of efficient solutions, also known as Pareto-optimal solutions, by automatically formulating and solving a sequence of weighted problems. The approach is called MONISE (Many-Objective NISE), because it represents an extension of the well-known non-inferior set estimation (NISE) algorithm, which was originally conceived to deal with two-dimensional objective spaces. Looking for theoretical support, we demonstrate that being a solution of the weighted problem is a necessary condition, and it will also be a sufficient condition at the convex hull of the feasible set. The proposal is conceived to operate in more than two dimensions, thus properly supporting many objectives. Moreover, specifically deal with two objectives, some nice additional properties are portrayed for the estimated non-inferior set. Experimental results are used to validate the proposal and have indicated that MONISE is competitive both in terms of computational cost and considering the overall quality of the non-inferior set, measured by the hypervolume.Comment: 36 page

The proximal point method for locally lipschitz functions in multiobjective optimization with application to the compromise problem

This paper studies the constrained multiobjective optimization problem of finding Pareto critical points of vector-valued functions. The proximal point method considered by Bonnel, Iusem, and Svaiter [SIAM J. Optim., 15 (2005), pp. 953–970] is extended to locally Lipschitz functions in the finite dimensional multiobjective setting. To this end, a new (scalarization-free) approach for convergence analysis of the method is proposed where the first-order optimality condition of the scalarized problem is replaced by a necessary condition for weak Pareto points of a multiobjective problem. As a consequence, this has allowed us to consider the method without any assumption of convexity over the constraint sets that determine the vectorial improvement steps. This is very important for applications; for example, to extend to a dynamic setting the famous compromise problem in management sciences and game theory.Fundação de Amparo à Pesquisa do Estado de GoiásConselho Nacional de Desenvolvimento Científico e TecnológicoCoordenação de Aperfeiçoamento de Pessoal de Nivel SuperiorMinisterio de Economía y CompetitividadAgence nationale de la recherch

Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes

In industrial applications of adaptive optimal control often multiple contrary objectives have to be considered. The weights (relative importance) of the objectives are often not known during the design of the control and can change with changing production conditions and requirements. In this work a novel model-free multiobjective reinforcement learning approach for adaptive optimal control of manufacturing processes is proposed. The approach enables sample-efficient learning in sequences of control configurations, given by particular objective weights.Comment: Conference, Preprint, 978-1-5386-5925-0/18/$31.00 \c{opyright} 2018 IEE

Controllable Pareto Multi-Task Learning

A multi-task learning (MTL) system aims at solving multiple related tasks at the same time. With a fixed model capacity, the tasks would be conflicted with each other, and the system usually has to make a trade-off among learning all of them together. For many real-world applications where the trade-off has to be made online, multiple models with different preferences over tasks have to be trained and stored. This work proposes a novel controllable Pareto multi-task learning framework, to enable the system to make real-time trade-off control among different tasks with a single model. To be specific, we formulate the MTL as a preference-conditioned multiobjective optimization problem, with a parametric mapping from preferences to the corresponding trade-off solutions. A single hypernetwork-based multi-task neural network is built to learn all tasks with different trade-off preferences among them, where the hypernetwork generates the model parameters conditioned on the preference. For inference, MTL practitioners can easily control the model performance based on different trade-off preferences in real-time. Experiments on different applications demonstrate that the proposed model is efficient for solving various MTL problems

Pareto Multi-Task Learning

Multi-task learning is a powerful method for solving multiple correlated tasks simultaneously. However, it is often impossible to find one single solution to optimize all the tasks, since different tasks might conflict with each other. Recently, a novel method is proposed to find one single Pareto optimal solution with good trade-off among different tasks by casting multi-task learning as multiobjective optimization. In this paper, we generalize this idea and propose a novel Pareto multi-task learning algorithm (Pareto MTL) to find a set of well-distributed Pareto solutions which can represent different trade-offs among different tasks. The proposed algorithm first formulates a multi-task learning problem as a multiobjective optimization problem, and then decomposes the multiobjective optimization problem into a set of constrained subproblems with different trade-off preferences. By solving these subproblems in parallel, Pareto MTL can find a set of well-representative Pareto optimal solutions with different trade-off among all tasks. Practitioners can easily select their preferred solution from these Pareto solutions, or use different trade-off solutions for different situations. Experimental results confirm that the proposed algorithm can generate well-representative solutions and outperform some state-of-the-art algorithms on many multi-task learning applications.Comment: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canad

Explicit multiobjective model predictive control for nonlinear systems with symmetries

Model predictive control is a prominent approach to construct a feedback control loop for dynamical systems. Due to real-time constraints, the major challenge in MPC is to solve model-based optimal control problems in a very short amount of time. For linear-quadratic problems, Bemporad et al.~have proposed an explicit formulation where the underlying optimization problems are solved a priori in an offline phase. In this article, we present an extension of this concept in two significant ways. We consider nonlinear problems and -- more importantly -- problems with multiple conflicting objective functions. In the offline phase, we build a library of Pareto optimal solutions from which we then obtain a valid compromise solution in the online phase according to a decision maker's preference. Since the standard multi-parametric programming approach is no longer valid in this situation, we instead use interpolation between different entries of the library. To reduce the number of problems that have to be solved in the offline phase, we exploit symmetries in the dynamical system and the corresponding multiobjective optimal control problem. The results are verified using two different examples from autonomous driving

A linear bound on the number of scalarizations needed to solve discrete tricriteria optimization problems

General multi-objective optimization problems are often solved by a sequence of parametric single objective problems, so-called scalarizations. If the set of nondominated points is finite, and if an appropriate scalarization is employed, the entire nondominated set can be generated in this way. In the bicriteria case it is well known that this can be realized by an adaptive approach which, given an appropriate initial search space, requires the solution of a number of subproblems which is at most two times the number of nondominated points. For higher dimensional problems, no linear methods were known up to now. We present a new procedure for finding the entire nondominated set of tricriteria optimization problems for which the number of scalarized subproblems to be solved is at most three times the number of nondominated points of the underlying problem. The approach includes an iterative update of the search space that, given a (sub-)set of nondominated points, describes the area in which additional nondominated points may be located. In particular, we show that the number of boxes, into which the search space is decomposed, depends linearly on the number of nondominated points.Comment: 32 pages, 8 figures, Journal of Global Optimization, 201

Optimal Scalarizations for Sublinear Hypervolume Regret

Scalarization is a general technique that can be deployed in any multiobjective setting to reduce multiple objectives into one, such as recently in RLHF for training reward models that align human preferences. Yet some have dismissed this classical approach because linear scalarizations are known to miss concave regions of the Pareto frontier. To that end, we aim to find simple non-linear scalarizations that can explore a diverse set of $k$ objectives on the Pareto frontier, as measured by the dominated hypervolume. We show that hypervolume scalarizations with uniformly random weights are surprisingly optimal for provably minimizing the hypervolume regret, achieving an optimal sublinear regret bound of $O(T^{-1/k})$, with matching lower bounds that preclude any algorithm from doing better asymptotically. As a theoretical case study, we consider the multiobjective stochastic linear bandits problem and demonstrate that by exploiting the sublinear regret bounds of the hypervolume scalarizations, we can derive a novel non-Euclidean analysis that produces improved hypervolume regret bounds of $\tilde{O}( d T^{-1/2} + T^{-1/k})$. We support our theory with strong empirical performance of using simple hypervolume scalarizations that consistently outperforms both the linear and Chebyshev scalarizations, as well as standard multiobjective algorithms in bayesian optimization, such as EHVI.Comment: ICML 2023 Worksho

Topology of Pareto sets of strongly convex problems

A multiobjective optimization problem is simplicial if the Pareto set and front are homeomorphic to a simplex and, under the homeomorphisms, each face of the simplex corresponds to the Pareto set and front of a subproblem. In this paper, we show that strongly convex problems are simplicial under a mild assumption on the ranks of the differentials of the objective mappings. We further prove that one can make any strongly convex problem satisfy the assumption by a generic linear perturbation, provided that the dimension of the source is sufficiently larger than that of the target. We demonstrate that the location problems, a biological modeling, and the ridge regression can be reduced to multiobjective strongly convex problems via appropriate transformations preserving the Pareto ordering and the topology.Comment: 21 pages. Remarks 4.4 and 4.5 are added. A new application is given in section 5.3. Introduction is also revised accordingl

A bi-criteria path planning algorithm for robotics applications

Realistic path planning applications often require optimizing with respect to several criteria simultaneously. Here we introduce an efficient algorithm for bi-criteria path planning on graphs. Our approach is based on augmenting the state space to keep track of the "budget" remaining to satisfy the constraints on secondary cost. The resulting augmented graph is acyclic and the primary cost can be then minimized by a simple upward sweep through budget levels. The efficiency and accuracy of our algorithm is tested on Probabilistic Roadmap graphs to minimize the distance of travel subject to a constraint on the overall threat exposure of the robot. We also present the results from field experiments illustrating the use of this approach on realistic robotic systems.Comment: 19 pages, 12 figures; submitted for publication to IEEE Transactions on Automation Science and Engineerin