Nonsmooth Optimization and Descent Methods

Nonsmooth optimization is a field of research actively pursued at IIASA. In this paper, we show "what" it is; a thing that cannot be guessed easily from its definition by a negative statement. Also, we show "why" it exists at IIASA, by exhibiting a large field of applications ranging from the theory of nonlinear programming to the computation of economic equilibria, including the general concept of decentralization. Finally, we show "how" it can be done, outlining the state of the art, and developing a new algorithm that realizes a synthesis between the concepts commonly used in differentiable as well as nondifferentiable optimization. Our approach is as non-technical as possible, and we hope that a nonacquainted reader will be able to follow a non-negligible part of our development. In Section 1, we give the basic concepts underlying nonsmooth optimization and show what it consists of. We also outline the classical methods, which have existed since 1959, aimed at optimizing nondifferentiable problems. In Section 2, we give a list of possible applications, including acceleration of gradient type methods, general decomposition--by prices, by resources, and Benders decomposition--minimax problems, and computation of economic equilibria. In Section 3, we give the most modern methods for nonsmooth optimization, defined around 1975, which were the first general descent methods. In Section 4, we develop a new descent method, which is based on concepts of variable metric, cutting plan approximation and feasible directions. We study its motivation, its convergence, and its flexibility

Nonsmooth Optimization: Use of the Code DYNEPS

One of the aims of the Optimization Task of the System and Decision Sciences Area is to provide computer codes that help to solve certain numerical problems. This paper describes the use of such a code which is being used successfully on a number of IIASA problems, in particular for the Food and Agriculture Program and Human Settlements and Services

Nonsmooth Optimization; Proceedings of an IIASA Workshop, March 28 - April 8, 1977

Optimization, a central methodological tool of systems analysis, is used in many of IIASA's research areas, including the Energy Systems and Food and Agriculture Programs. IIASA's activity in the field of optimization is strongly connected with nonsmooth or nondifferentiable extreme problems, which consist of searching for conditional or unconditional minima of functions that, due to their complicated internal structure, have no continuous derivatives. Particularly significant for these kinds of extreme problems in systems analysis is the strong link between nonsmooth or nondifferentiable optimization and the decomposition approach to large-scale programming. This volume contains the report of the IIASA workshop held from March 28 to April 8, 1977, entitled Nondifferentiable Optimization. However, the title was changed to Nonsmooth Optimization for publication of this volume as we are concerned not only with optimization without derivatives, but also with problems having functions for which gradients exist almost everywhere but are not continous, so that the usual gradient-based methods fail. Because of the small number of participants and the unusual length of the workshop, a substantial exchange of information was possible. As a result, details of the main developments in nonsmooth optimization are summarized in this volume, which might also be considered a guide for inexperienced users. Eight papers are presented: three on subgradient optimization, four on descent methods, and one on applicability. The report also includes a set of nonsmooth optimization test problems and a comprehensive bibliography

Differentiability of a Support Function of an E-Subgradient

Directional differentiability of a support function of an epsilon-subgradient set-valued mapping is proved and formula for a directional derivative is given

Computing Economic Equilibria through Nonsmooth Optimization

In the first section the problem of nonsmooth optimization is described in general terms, setting the precise hypothesis in mathematical language. Section 2 describes the principles of an example which arises in the context of the linkage of national models of food and agriculture. The general methodology is presented in Section 3, where the algorithm of solution is outlined. Section 4 reports on an extensive set of numerical experiments, both on problems known in the literature, and on the example of Section 2. Finally the paper concludes with some remarks about improvements of the algorithm, which motivate further research on the subject

Some Numerical Experiments with Variable Storage Quasi-Newton Algorithms

This paper relates some numerical experiments with variable storage quasi-Newton methods for the optimization of large-scale models. The basic idea of the recommended algorithm is to start bfgs updates on a diagonal matrix, itself generated by an update formula and adjusted to Rayleigh's ellipsoid of the local Hessian of the objective function in the direction of the change in the gradient. A variational derivation of some rank one and rank two updates in Hilbert spaces is also given

An inexact conic bundle variant suited to column generation

Final version to appear in Mathematical Programming Available in www.springerlink.com DOI 10.1007/s10107-007-0187-4We give a bundle method for constrained convex optimization. Instead of using penalty functions, it shifts iterates towards feasibility, by way of a Slater point, assumed to be known. Besides, the method accepts an oracle delivering function and subgradient values with unknown accuracy. Our approach is motivated by a number of applications in column generation, in which constraints are positively homogeneous -- so that 0 is a natural Slater point -- and an exact oracle may be time consuming. Finally, our convergence analysis employs arguments which have been little used so far in the bundle community. The method is illustrated on a number of cutting-stock problems

New variants of bundle methods

An updated version of this paper has appeared in Mathematical Programming, no 69, (1995), pp. 111-147, DOI 10.1007/BF01585555Résumé disponible dans le fichier PD

On the equivalence between complementarity systems, projected systems and differential inclusions

International audienceIn this note, we prove the equivalence, under appropriate conditions, between several dynamical formalisms: projected dynamical systems, two types of differential inclusions, and a class of complementarity dynamical systems. Each of these dynamical systems can also be considered as a hybrid dynamical system. This work both generalizes previous results and sheds some new light on the relationship between known formalisms; besides, it exclusively uses tools from convex analysis