On multiobjective optimization from the nonsmooth perspective

Practical applications usually have multiobjective nature rather than having only one objective to optimize. A multiobjective problem cannot be solved with a single-objective solver as such. On the other hand, optimization of only one objective may lead to an arbitrary bad solutions with respect to other objectives. Therefore, special techniques for multiobjective optimization are vital. In addition to multiobjective nature, many real-life problems have nonsmooth (i.e. not continuously differentiable) structure. Unfortunately, many smooth (i.e. continuously differentiable) methods adopt gradient-based information which cannot be used for nonsmooth problems. Since both of these characteristics are relevant for applications, we focus here on nonsmooth multiobjective optimization. As a research topic, nonsmooth multiobjective optimization has gained only limited attraction while the fields of nonsmooth single-objective and smooth multiobjective optimization distinctively have attained greater interest. This dissertation covers parts of nonsmooth multiobjective optimization in terms of theory, methodology and application. Bundle methods are widely considered as effective and reliable solvers for single-objective nonsmooth optimization. Therefore, we investigate the use of the bundle idea in the multiobjective framework with three different methods. The first one generalizes the single-objective proximal bundle method for the nonconvex multiobjective constrained problem. The second method adopts the ideas from the classical steepest descent method into the convex unconstrained multiobjective case. The third method is designed for multiobjective problems with constraints where both the objectives and constraints can be represented as a difference of convex (DC) functions. Beside the bundle idea, all three methods are descent, meaning that they produce better values for each objective at each iteration. Furthermore, all of them utilize the improvement function either directly or indirectly. A notable fact is that none of these methods use scalarization in the traditional sense. With the scalarization we refer to the techniques transforming a multiobjective problem into the single-objective one. As the scalarization plays an important role in multiobjective optimization, we present one special family of achievement scalarizing functions as a representative of this category. In general, the achievement scalarizing functions suit well in the interactive framework. Thus, we propose the interactive method using our special family of achievement scalarizing functions. In addition, this method utilizes the above mentioned descent methods as tools to illustrate the range of optimal solutions. Finally, this interactive method is used to solve the practical case studies of the scheduling the final disposal of the spent nuclear fuel in Finland.Käytännön optimointisovellukset ovat usein luonteeltaan ennemmin moni- kuin yksitavoitteisia. Erityisesti monitavoitteisille tehtäville suunnitellut menetelmät ovat tarpeen, sillä monitavoitteista optimointitehtävää ei sellaisenaan pysty ratkaisemaan yksitavoitteisilla menetelmillä eikä vain yhden tavoitteen optimointi välttämättä tuota mielekästä ratkaisua muiden tavoitteiden suhteen. Monitavoitteisuuden lisäksi useat käytännön tehtävät ovat myös epäsileitä siten, etteivät niissä esiintyvät kohde- ja rajoitefunktiot välttämättä ole kaikkialla jatkuvasti differentioituvia. Kuitenkin monet optimointimenetelmät hyödyntävät gradienttiin pohjautuvaa tietoa, jota ei epäsileille funktioille ole saatavissa. Näiden molempien ominaisuuksien ollessa keskeisiä sovelluksia ajatellen, keskitytään tässä työssä epäsileään monitavoiteoptimointiin. Tutkimusalana epäsileä monitavoiteoptimointi on saanut vain vähän huomiota osakseen, vaikka sekä sileä monitavoiteoptimointi että yksitavoitteinen epäsileä optimointi erikseen ovat aktiivisia tutkimusaloja. Tässä työssä epäsileää monitavoiteoptimointia on käsitelty niin teorian, menetelmien kuin käytännön sovelluksien kannalta. Kimppumenetelmiä pidetään yleisesti tehokkaina ja luotettavina menetelminä epäsileän optimointitehtävän ratkaisemiseen ja siksi tätä ajatusta hyödynnetään myös tässä väitöskirjassa kolmessa eri menetelmässä. Ensimmäinen näistä yleistää yksitavoitteisen proksimaalisen kimppumenetelmän epäkonveksille monitavoitteiselle rajoitteiselle tehtävälle sopivaksi. Toinen menetelmä hyödyntää klassisen nopeimman laskeutumisen menetelmän ideaa konveksille rajoitteettomalle tehtävälle. Kolmas menetelmä on suunniteltu erityisesti monitavoitteisille rajoitteisille tehtäville, joiden kohde- ja rajoitefunktiot voidaan ilmaista kahden konveksin funktion erotuksena. Kimppuajatuksen lisäksi kaikki kolme menetelmää ovat laskevia eli ne tuottavat joka kierroksella paremman arvon jokaiselle tavoitteelle. Yhteistä on myös se, että nämä kaikki hyödyntävät parannusfunktiota joko suoraan sellaisenaan tai epäsuorasti. Huomattavaa on, ettei yksikään näistä menetelmistä hyödynnä skalarisointia perinteisessä merkityksessään. Skalarisoinnilla viitataan menetelmiin, joissa usean tavoitteen tehtävä on muutettu sopivaksi yksitavoitteiseksi tehtäväksi. Monitavoiteoptimointimenetelmien joukossa skalarisoinnilla on vankka jalansija. Esimerkkinä skalarisoinnista tässä työssä esitellään yksi saavuttavien skalarisointifunktioiden perhe. Yleisesti saavuttavat skalarisointifunktiot soveltuvat hyvin interaktiivisten menetelmien rakennuspalikoiksi. Täten kuvaillaan myös esiteltyä skalarisointifunktioiden perhettä hyödyntävä interaktiivinen menetelmä, joka lisäksi hyödyntää laskevia menetelmiä optimaalisten ratkaisujen havainnollistamisen apuna. Lopuksi tätä interaktiivista menetelmää käytetään aikatauluttamaan käytetyn ydinpolttoaineen loppusijoitusta Suomessa

On the Relationship Between the Value Function and the Efficient Frontier of a Mixed Integer Linear Optimization Problem

In this paper, we investigate the connection between the efficient frontier (EF) of a general multiobjective mixed integer linear optimization problem (MILP) and the so-called restricted value function (RVF) of a closely related single-objective MILP. We demonstrate that the EF of the multiobjective MILP is comprised of points on the boundary of the epigraph of the RVF so that any description of the EF suffices to describe the RVF and vice versa. In the first part of the paper, we describe the mathematical structure of the RVF, including characterizing the set of points at which it is differentiable, the gradients at such points, and the subdifferential at all nondifferentiable points. Because of the close relationship of the RVF to the EF, we observe that methods for constructing so-called value functions and methods for constructing the EF of a multiobjective optimization problem, each of which have been developed in separate communities, are effectively interchangeable. By exploiting this relationship, we propose a generalized cutting plane algorithm for constructing the EF of a multiobjective MILP based on a generalization of an existing algorithm for constructing the classical value function. We prove that the algorithm is finite under a standard boundedness assumption and comes with a performance guarantee if terminated early

Nonconvex and mixed integer multiobjective optimization with an application to decision uncertainty

Multiobjective optimization problems commonly arise in different fields like economics or engineering. In general, when dealing with several conflicting objective functions, there is an infinite number of optimal solutions which cannot usually be determined analytically. This thesis presents new branch-and-bound-based approaches for computing the globally optimal solutions of multiobjective optimization problems of various types. New algorithms are proposed for smooth multiobjective nonconvex optimization problems with convex constraints as well as for multiobjective mixed integer convex optimization problems. Both algorithms guarantee a certain accuracy of the computed solutions, and belong to the first deterministic algorithms within their class of optimization problems. Additionally, a new approach to compute a covering of the optimal solution set of multiobjective optimization problems with decision uncertainty is presented. The three new algorithms are tested numerically. The results are evaluated in this thesis as well. The branch-and-bound based algorithms deal with box partitions and use selection rules, discarding tests and termination criteria. The discarding tests are the most important aspect, as they give criteria whether a box can be discarded as it does not contain any optimal solution. We present discarding tests which combine techniques from global single objective optimization with outer approximation techniques from multiobjective convex optimization and with the concept of local upper bounds from multiobjective combinatorial optimization. The new discarding tests aim to find appropriate lower bounds of subsets of the image set in order to compare them with known upper bounds numerically.Multikriterielle Optimierungprobleme sind in diversen Anwendungsgebieten wie beispielsweise in den Wirtschafts- oder Ingenieurwissenschaften zu finden. Da hierbei mehrere konkurrierende Zielfunktionen auftreten, ist die Lösungsmenge eines derartigen Optimierungsproblems im Allgemeinen unendlich groß und kann meist nicht in analytischer Form berechnet werden. In dieser Dissertation werden neue Branch-and-Bound basierte Algorithmen zur Lösung verschiedener Klassen von multikriteriellen Optimierungsproblemen entwickelt und vorgestellt. Der Branch-and-Bound Ansatz ist eine typische Methode der globalen Optimierung. Einer der neuen Algorithmen löst glatte multikriterielle nichtkonvexe Optimierungsprobleme mit konvexen Nebenbedingungen, während ein zweiter zur Lösung multikriterieller gemischt-ganzzahliger konvexer Optimierungsprobleme dient. Beide Algorithmen garantieren eine gewisse Genauigkeit der berechneten Lösungen und gehören damit zu den ersten deterministischen Algorithmen ihrer Art. Zusätzlich wird ein Algorithmus zur Berechnung einer Überdeckung der Lösungsmenge multikriterieller Optimierungsprobleme mit Entscheidungsunsicherheit vorgestellt. Alle drei Algorithmen wurden numerisch getestet. Die Ergebnisse werden ebenfalls in dieser Arbeit ausgewertet. Die neuen Algorithmen arbeiten alle mit Boxunterteilungen und nutzen Auswahlregeln, sowie Verwerfungs- und Terminierungskriterien. Dabei spielen gute Verwerfungskriterien eine zentrale Rolle. Diese entscheiden, ob eine Box verworfen werden kann, da diese sicher keine Optimallösung enthält. Die neuen Verwerfungskriterien nutzen Methoden aus der globalen skalarwertigen Optimierung, Approximationstechniken aus der multikriteriellen konvexen Optimierung sowie ein Konzept aus der kombinatorischen Optimierung. Dabei werden stets untere Schranken der Bildmengen konstruiert, die mit bisher berechneten oberen Schranken numerisch verglichen werden können

Multiobjective Optimization for Complex Systems

Complex systems are becoming more and more apparent in a variety of disciplines, making solution methods for these systems valuable tools. The solution of complex systems requires two significant skills. The first challenge of developing mathematical models for these systems is followed by the difficulty of solving these models to produce preferred solutions for the overall systems. Both issues are addressed by this research. This study of complex systems focuses on two distinct aspects. First, models of complex systems with multiobjective formulations and a variety of structures are proposed. Using multiobjective optimization theory, relationships between the efficient solutions of the overall system and the efficient solutions of its subproblems are derived. A system with a particular structure is then selected and further analysis is performed regarding the connection between the original system and its decomposable counterpart. The analysis is based on Kuhn-Tucker efficiency conditions. The other aspect of this thesis pertains to the study of a class of complex systems with a structure that is amenable for use with analytic target cascading (ATC), a decomposition and coordination approach of special interest to engineering design. Two types of algorithms are investigated. Modifications to a subgradient optimization algorithm are proposed and shown to improve the speed of the algorithm. A new family of biobjective algorithms showing considerable promise for ATC-decomposable problems is introduced for two-level systems, and convergence results for a specified algorithm are given. Numerical examples showing the effectiveness of all algorithms are included

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more

Decomposition methods for mixed-integer nonlinear programming

En esta tesis se pueden distinguir dos líneas principales de investigación. La primera se ocupa de los métodos de Aproximación Externa (Outer Approximation), mientras que la segunda estudia un solución basada en el método de Generación de Columnas (Column Generation). En esta tesis investigamos y analizamos aspectos teóricos y prácticos de ambas ideas dentro del marco de la descomposición. El objetivo principal de este estudio es desarrollar métodos sistemáticos basados en la descomposición para resolver problemas de gran escala utilizando los métodos de Aproximación Externa y Generación de Columnas. En el capítulo 1 se introduce un concepto importante necesario para la descomposición. Este concepto consiste en una reformulación separable en bloques del problema de programación no lineal de enteros mixtos. En el capítulo 1 también se hace una descripción de los métodos mencionados anteriormente, incluyendo los de Ramificación y Acotación, además de otros conceptos clave que son necesarios para esta tesis, como por ejemplo los de Aproximación Interior, etc. Los capítulos 2, 3 y 4 investigan el uso del concepto de Aproximación Externa. Específicamente, en el capítulo 2 se presenta un algoritmo de Aproximación Externa basado en descomposición para resolver problemas de programación no-lineales convexos enteros-mixtos, basados en la construcción de hiperplanos soporte para un conjunto factible. El capítulo 3 amplia el marco de aplicación de un algoritmo de Aproximación Externa basado en descomposición, a problemas de programación no lineales no convexos enteros mixtos, introduciendo una Aproximación Externa convexa por partes de un conjunto factible no convexo. Otra perspectiva de la definición de Aproximación Externa para problemas no convexos se considera en el capítulo 4, que presenta un algoritmo de Refinamiento Interno y Externo basado en descomposición, que construye una Aproximación Externa al mismo tiempo que calcula la Aproximación Interna usando Generación de Columnas. La Aproximación Externa usada en el algoritmo de Refinamiento Interno y Externo se basa en la visión multiobjetivo de la denominada versión recursos restringidos del problema original. Dos capítulos están dedicados a la Generación de Columnas. En el capítulo 4 se presenta un algoritmo de Generación de Columnas para calcular una Aproximación Interna del problema original. Además se describe un algoritmo heurístico basado en particiones que usa un refinamiento de la Aproximación Interna. El capítulo 5 analiza varias técnicas de aceleración para la Generación de Columnas, donde se describe un algoritmo heurístico general basado en la Generación de Columnas, que puede generar varias soluciones candidatas de alta calidad. El capítulo 6 contiene una breve descripción de la implementación en Python de DECOGO (software de programación no lineal de enteros mixtos).La programación no lineal de enteros mixtos es un campo de optimización importante y desafiante. Este tipo de problemas pueden contener variables continuas e enteras, así como restricciones lineales y no lineales. Esta clase de problemas tiene un papel fundamental en la ciencia y la industria, ya que proporcionan una forma precisa de describir fenómenos en diferentes áreas como ingeniería química y mecánica, cadena de suministro, gestión, etc. La mayoría de los algoritmos de última generación para resolver los problemas de programación no lineal de enteros mixtos no convexos están basados en los métodos de ramificación y acotación. El principal inconveniente de este enfoque es que el árbol de búsqueda puede crecer muy rápido impidiendo que el algoritmo encuentre una solución de alta calidad en un tiempo razonable. Una posible alternativa que evite la generación de grandes árboles consiste en hacer uso del concepto de descomposición para hacer que el procedimiento sea más manejable. La descomposición proporciona un marco general en el que el problema original se divide en pequeños subproblemas y sus resultados se combinan en un problema maestro más sencillo. Esta tesis analiza los métodos de descomposición para la programación no lineal de enteros mixtos. El principal objetivo de esta tesis es desarrollar métodos alternativos al de ramificación y acotación, basados en el concepto de descomposición. Para la industria y la ciencia, es importante calcular una solución óptima, o al menos, mejorar la mejor solución disponible hasta ahora. Además, esto debe hacerse en un plazo de tiempo razonable. Por lo tanto, el objetivo de esta tesis es diseñar algoritmos eficientes que permitan resolver problemas de gran escala que tienen una aplicación práctica directa. En particular, nos centraremos en modelos que pueden ser aplicados en la planificación y operación de sistemas energéticos

Proceedings of the XIII Global Optimization Workshop: GOW'16

[Excerpt] Preface: Past Global Optimization Workshop shave been held in Sopron (1985 and 1990), Szeged (WGO, 1995), Florence (GO’99, 1999), Hanmer Springs (Let’s GO, 2001), Santorini (Frontiers in GO, 2003), San José (Go’05, 2005), Mykonos (AGO’07, 2007), Skukuza (SAGO’08, 2008), Toulouse (TOGO’10, 2010), Natal (NAGO’12, 2012) and Málaga (MAGO’14, 2014) with the aim of stimulating discussion between senior and junior researchers on the topic of Global Optimization. In 2016, the XIII Global Optimization Workshop (GOW’16) takes place in Braga and is organized by three researchers from the University of Minho. Two of them belong to the Systems Engineering and Operational Research Group from the Algoritmi Research Centre and the other to the Statistics, Applied Probability and Operational Research Group from the Centre of Mathematics. The event received more than 50 submissions from 15 countries from Europe, South America and North America. We want to express our gratitude to the invited speaker Panos Pardalos for accepting the invitation and sharing his expertise, helping us to meet the workshop objectives. GOW’16 would not have been possible without the valuable contribution from the authors and the International Scientific Committee members. We thank you all. This proceedings book intends to present an overview of the topics that will be addressed in the workshop with the goal of contributing to interesting and fruitful discussions between the authors and participants. After the event, high quality papers can be submitted to a special issue of the Journal of Global Optimization dedicated to the workshop. [...

New variational principles with applications to optimization theory and algorithms

In this dissertation we investigate some applications of variational analysis in optimization theory and algorithms. In the first part we develop new extremal principles in variational analysis that deal with finite and infinite systems of convex and nonconvex sets. The results obtained, under the name of tangential extremal principles and rated extremal principles, combine primal and dual approaches to the study of variational systems being in fact first extremal principles applied to infinite systems of sets. These developments are in the core geometric theory of variational analysis. Our study includes the basic theory and applications to problems of semi-infinite programming and multiobjective optimization. The second part of this dissertation concerns developing numerical methods of the Newton-type to solve systems of nonlinear equations. We propose and justify a new generalized Newton algorithm based on graphical derivatives. Based on advanced tools of variational analysis and generalized differentiation, we establish the well-posedness and convergence results of the algorithm. Besides, we present a new generalized damped Newton algorithm, which is also known as Newton\u27s method with line-search. Some global convergence results are also justified

Inverse Optimization and Inverse Multiobjective Optimization

Inverse optimization is a powerful paradigm for learning preferences and restrictions that explain the behavior of a decision maker, based on a set of external signal and the corresponding decision pairs. However, most inverse optimization algorithms are designed specifically in a batch setting, where all data is available in advance. As a consequence, there has been rare use of these methods in an online setting that is more suitable for real-time applications. To change such a situation, we propose a general framework for inverse optimization through online learning. Specifically, we develop an online learning algorithm that uses an implicit update rule which can handle noisy data. We also note that the majority of existing studies assumes that the decision making problem is with a single objective function, and attributes data divergence to noises, errors or bounded rationality, which, however, could lead to a corrupted inference when decisions are tradeoffs among multiple criteria. We take a data-driven approach and design a more sophisticated inverse optimization formulation to explicitly infer parameters of a multiobjective decision making problem from noisy observations. This framework, together with our mathematical analyses and advanced algorithm developments, demonstrates a strong capacity in estimating critical parameters, decoupling interpretable components from noises or errors, deriving the denoised optimal decisions, and ensuring statistical significance. In particular, for the whole decision maker population, if suitable conditions hold, we will be able to understand the overall diversity and the distribution of their preferences over multiple criteria. Additionally, we propose a distributionally robust approach to inverse multiobjective optimization. Specifically, we study the problem of learning the objective functions or constraints of a multiobjective decision making model, based on a set of observed decisions. In particular, these decisions might not be exact and possibly carry measurement noises or are generated with the bounded rationality of decision makers. We use the Wasserstein metric to construct the uncertainty set centered at the empirical distribution of these decisions. We show that this framework has statistical performance guarantees. We also develop an algorithm to solve the resulting minmax problem and prove its finite convergence