The proximal point method for locally lipschitz functions in multiobjective optimization with application to the compromise problem

This paper studies the constrained multiobjective optimization problem of finding Pareto critical points of vector-valued functions. The proximal point method considered by Bonnel, Iusem, and Svaiter [SIAM J. Optim., 15 (2005), pp. 953–970] is extended to locally Lipschitz functions in the finite dimensional multiobjective setting. To this end, a new (scalarization-free) approach for convergence analysis of the method is proposed where the first-order optimality condition of the scalarized problem is replaced by a necessary condition for weak Pareto points of a multiobjective problem. As a consequence, this has allowed us to consider the method without any assumption of convexity over the constraint sets that determine the vectorial improvement steps. This is very important for applications; for example, to extend to a dynamic setting the famous compromise problem in management sciences and game theory.Fundação de Amparo à Pesquisa do Estado de GoiásConselho Nacional de Desenvolvimento Científico e TecnológicoCoordenação de Aperfeiçoamento de Pessoal de Nivel SuperiorMinisterio de Economía y CompetitividadAgence nationale de la recherch

A dynamic gradient approach to Pareto optimization with nonsmooth convex objective functions

In a general Hilbert framework, we consider continuous gradient-like dynamical systems for constrained multiobjective optimization involving non-smooth convex objective functions. Our approach is in the line of a previous work where was considered the case of convex di erentiable objective functions. Based on the Yosida regularization of the subdi erential operators involved in the system, we obtain the existence of strong global trajectories. We prove a descent property for each objective function, and the convergence of trajectories to weak Pareto minima. This approach provides a dynamical endogenous weighting of the objective functions. Applications are given to cooperative games, inverse problems, and numerical multiobjective optimization

Monotonicity for Multiobjective Accelerated Proximal Gradient Methods

Accelerated proximal gradient methods, which are also called fast iterative shrinkage-thresholding algorithms (FISTA) are known to be efficient for many applications. Recently, Tanabe et al. proposed an extension of FISTA for multiobjective optimization problems. However, similarly to the single-objective minimization case, the objective functions values may increase in some iterations, and inexact computations of subproblems can also lead to divergence. Motivated by this, here we propose a variant of the FISTA for multiobjective optimization, that imposes some monotonicity of the objective functions values. In the single-objective case, we retrieve the so-called MFISTA, proposed by Beck and Teboulle. We also prove that our method has global convergence with rate $O(1/k^2)$, where $k$ is the number of iterations, and show some numerical advantages in requiring monotonicity.Comment: - Added new numerical experiment

On multiobjective optimization from the nonsmooth perspective

Practical applications usually have multiobjective nature rather than having only one objective to optimize. A multiobjective problem cannot be solved with a single-objective solver as such. On the other hand, optimization of only one objective may lead to an arbitrary bad solutions with respect to other objectives. Therefore, special techniques for multiobjective optimization are vital. In addition to multiobjective nature, many real-life problems have nonsmooth (i.e. not continuously differentiable) structure. Unfortunately, many smooth (i.e. continuously differentiable) methods adopt gradient-based information which cannot be used for nonsmooth problems. Since both of these characteristics are relevant for applications, we focus here on nonsmooth multiobjective optimization. As a research topic, nonsmooth multiobjective optimization has gained only limited attraction while the fields of nonsmooth single-objective and smooth multiobjective optimization distinctively have attained greater interest. This dissertation covers parts of nonsmooth multiobjective optimization in terms of theory, methodology and application. Bundle methods are widely considered as effective and reliable solvers for single-objective nonsmooth optimization. Therefore, we investigate the use of the bundle idea in the multiobjective framework with three different methods. The first one generalizes the single-objective proximal bundle method for the nonconvex multiobjective constrained problem. The second method adopts the ideas from the classical steepest descent method into the convex unconstrained multiobjective case. The third method is designed for multiobjective problems with constraints where both the objectives and constraints can be represented as a difference of convex (DC) functions. Beside the bundle idea, all three methods are descent, meaning that they produce better values for each objective at each iteration. Furthermore, all of them utilize the improvement function either directly or indirectly. A notable fact is that none of these methods use scalarization in the traditional sense. With the scalarization we refer to the techniques transforming a multiobjective problem into the single-objective one. As the scalarization plays an important role in multiobjective optimization, we present one special family of achievement scalarizing functions as a representative of this category. In general, the achievement scalarizing functions suit well in the interactive framework. Thus, we propose the interactive method using our special family of achievement scalarizing functions. In addition, this method utilizes the above mentioned descent methods as tools to illustrate the range of optimal solutions. Finally, this interactive method is used to solve the practical case studies of the scheduling the final disposal of the spent nuclear fuel in Finland.Käytännön optimointisovellukset ovat usein luonteeltaan ennemmin moni- kuin yksitavoitteisia. Erityisesti monitavoitteisille tehtäville suunnitellut menetelmät ovat tarpeen, sillä monitavoitteista optimointitehtävää ei sellaisenaan pysty ratkaisemaan yksitavoitteisilla menetelmillä eikä vain yhden tavoitteen optimointi välttämättä tuota mielekästä ratkaisua muiden tavoitteiden suhteen. Monitavoitteisuuden lisäksi useat käytännön tehtävät ovat myös epäsileitä siten, etteivät niissä esiintyvät kohde- ja rajoitefunktiot välttämättä ole kaikkialla jatkuvasti differentioituvia. Kuitenkin monet optimointimenetelmät hyödyntävät gradienttiin pohjautuvaa tietoa, jota ei epäsileille funktioille ole saatavissa. Näiden molempien ominaisuuksien ollessa keskeisiä sovelluksia ajatellen, keskitytään tässä työssä epäsileään monitavoiteoptimointiin. Tutkimusalana epäsileä monitavoiteoptimointi on saanut vain vähän huomiota osakseen, vaikka sekä sileä monitavoiteoptimointi että yksitavoitteinen epäsileä optimointi erikseen ovat aktiivisia tutkimusaloja. Tässä työssä epäsileää monitavoiteoptimointia on käsitelty niin teorian, menetelmien kuin käytännön sovelluksien kannalta. Kimppumenetelmiä pidetään yleisesti tehokkaina ja luotettavina menetelminä epäsileän optimointitehtävän ratkaisemiseen ja siksi tätä ajatusta hyödynnetään myös tässä väitöskirjassa kolmessa eri menetelmässä. Ensimmäinen näistä yleistää yksitavoitteisen proksimaalisen kimppumenetelmän epäkonveksille monitavoitteiselle rajoitteiselle tehtävälle sopivaksi. Toinen menetelmä hyödyntää klassisen nopeimman laskeutumisen menetelmän ideaa konveksille rajoitteettomalle tehtävälle. Kolmas menetelmä on suunniteltu erityisesti monitavoitteisille rajoitteisille tehtäville, joiden kohde- ja rajoitefunktiot voidaan ilmaista kahden konveksin funktion erotuksena. Kimppuajatuksen lisäksi kaikki kolme menetelmää ovat laskevia eli ne tuottavat joka kierroksella paremman arvon jokaiselle tavoitteelle. Yhteistä on myös se, että nämä kaikki hyödyntävät parannusfunktiota joko suoraan sellaisenaan tai epäsuorasti. Huomattavaa on, ettei yksikään näistä menetelmistä hyödynnä skalarisointia perinteisessä merkityksessään. Skalarisoinnilla viitataan menetelmiin, joissa usean tavoitteen tehtävä on muutettu sopivaksi yksitavoitteiseksi tehtäväksi. Monitavoiteoptimointimenetelmien joukossa skalarisoinnilla on vankka jalansija. Esimerkkinä skalarisoinnista tässä työssä esitellään yksi saavuttavien skalarisointifunktioiden perhe. Yleisesti saavuttavat skalarisointifunktiot soveltuvat hyvin interaktiivisten menetelmien rakennuspalikoiksi. Täten kuvaillaan myös esiteltyä skalarisointifunktioiden perhettä hyödyntävä interaktiivinen menetelmä, joka lisäksi hyödyntää laskevia menetelmiä optimaalisten ratkaisujen havainnollistamisen apuna. Lopuksi tätä interaktiivista menetelmää käytetään aikatauluttamaan käytetyn ydinpolttoaineen loppusijoitusta Suomessa

An accelerated proximal gradient method for multiobjective optimization

This paper presents an accelerated proximal gradient method for multiobjective optimization, in which each objective function is the sum of a continuously differentiable, convex function and a closed, proper, convex function. Extending first-order methods for multiobjective problems without scalarization has been widely studied, but providing accelerated methods with accurate proofs of convergence rates remains an open problem. Our proposed method is a multiobjective generalization of the accelerated proximal gradient method, also known as the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), for scalar optimization. The key to this successful extension is solving a subproblem with terms exclusive to the multiobjective case. This approach allows us to demonstrate the global convergence rate of the proposed method ($O(1 / k^2)$), using a merit function to measure the complexity. Furthermore, we present an efficient way to solve the subproblem via its dual representation, and we confirm the validity of the proposed method through some numerical experiments

A multiobjective continuation method to compute the regularization path of deep neural networks

Sparsity is a highly desired feature in deep neural networks (DNNs) since it ensures numerical efficiency, improves the interpretability of models (due to the smaller number of relevant features), and robustness. In machine learning approaches based on linear models, it is well known that there exists a connecting path between the sparsest solution in terms of the $\ell^1$ norm (i.e., zero weights) and the non-regularized solution, which is called the regularization path. Very recently, there was a first attempt to extend the concept of regularization paths to DNNs by means of treating the empirical loss and sparsity ($\ell^1$ norm) as two conflicting criteria and solving the resulting multiobjective optimization problem. However, due to the non-smoothness of the $\ell^1$ norm and the high number of parameters, this approach is not very efficient from a computational perspective. To overcome this limitation, we present an algorithm that allows for the approximation of the entire Pareto front for the above-mentioned objectives in a very efficient manner. We present numerical examples using both deterministic and stochastic gradients. We furthermore demonstrate that knowledge of the regularization path allows for a well-generalizing network parametrization.Comment: 7 pages, 6 figure