Inexact Semimonotonic Augmented Lagrangians with Optimal Feasibility Convergence for Convex Bound and Equality Constrained Quadratic Programming

A variant of the augmented Lagrangian-type algorithm for strictly convex quadratic programming problems with bounds and equality constraints is considered. The algorithm exploits the adaptive precision control in the solution of auxiliary bound constraint problems in the inner loop while the Lagrange multipliers for the equality constraints are updated in the outer loop. The update rule for the penalty parameter is introduced that depends on the increase of the augmented Lagrangian. Global convergence is proved and an explicit bound on the penalty parameter is given. A qualitatively new feature of our algorithm is a bound on the feasibility error that is independent of conditioning of the constraints. When applied to the class of problems with the spectrum of the Hessian matrix in a given interval, the algorithm returns the solution in O(1) matrix-vector multiplications. The results are valid even for linearly dependent constraints. Theoretical results are illustrated by numerical experiments including the solution of an elliptic variational inequality

Fragments d'Optimisation Différentiable - Théories et Algorithmes

MasterLecture Notes (in French) of optimization courses given at ENSTA (Paris, next Saclay), ENSAE (Paris) and at the universities Paris I, Paris VI and Paris Saclay (979 pages).Syllabus d’enseignements délivrés à l’ENSTA (Paris, puis Saclay), à l’ENSAE (Paris) et aux universités Paris I, Paris VI et Paris Saclay (979 pages)

Augmented Lagrangian Methods Under The Constant Positive Linear Dependence Constraint Qualification

Two Augmented Lagrangian algorithms for solving KKT systems are introduced. The algorithms differ in the way in which penalty parameters are updated. Possibly infeasible accumulation points are characterized. It is proved that feasible limit points that satisfy the Constant Positive Linear Dependence constraint qualification are KKT solutions. Boundedness of the penalty parameters is proved under suitable assumptions. Numerical experiments are presented. © Springer-Verlag 2007.1111-2532Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L., On Augmented Lagrangian methods with general lower-level constraints (2005), Technical Report MCDO-050303, Department of Applied Mathematics, UNICAMP, BrazilAndreani, R., Martínez, J.M., Schuverdt, M.L., On the relation between Constant Positive linear dependence condition and quasinormality constraint qualification (2005) J. Optim. Theory Appl, 125, pp. 473-485Bertsekas, D.P., (2003) Nonlinear programming, , 2nd edn. Athena Scientific, BelmontBirgin, E.G., Castillo, R., Martínez, J.M., Numerical comparison of Augmented Lagrangian algorithms for nonconvex problems (2005) Comput. Optim. Appl, 31, pp. 31-56Birgin, E.G., Martínez, J.M., Large-scale active-set box-constrained optimization method with spectral projected gradients (2002) Comput. Optim. Appl, 23, pp. 101-125Birgin, E.G., Martínez, J.M., Raydan, M., Nonmonotone spectral projected gradient methods on convex sets (2000) SIAM J. Optim, 10, pp. 1196-1211Birgin, E.G., Martínez, J.M., Raydan, M., Algorithm 813: SPG - Software for convex-constrained optimization (2001) ACM Trans. Math. Softw, 27, pp. 340-349Birgin, E.G., Martínez, J.M., Raydan, M., (2003) Inexact spectral projected gradient methods on convex sets. IMAI Numer, 23, pp. 539-559. , AnalBongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.L., CUTE: Constrained and unconstrained testing environment (1995) ACM Trans. Math. Softw, 21, pp. 123-160Conn, A.R., Gould, N.I.M., Toint, P.L., LANCELOT, A., (1992) Fortran Package for Large scale Nonlinear Optimization, , Springer, Berlin Heidelberg New YorkConn, A.R., Gould, N.I.M., Sartenaer, A., Toint, P.L., Convergence properties of an Augmented Lagrangian algorithm for optimization with a combination of general equality and linear constraints (1996) SIAM J. Optim, 6, pp. 674-703Conn, A.R., Gould, N.I.M., Toint, P.L., A globally convergent Augmented Lagrangian algorithm for optimization with general constraints and simple bounds (1991) SIAM J. Numer. Anal, 28, pp. 545-572Diniz-Ehrhardt, M.A., Gomes-Ruggiero, M.A., Martínez, J.M., Santos, S.A., Augmented Lagrangian algorithms based on the spectral projected gradient for solving nonlinear programming problems (2004) J. Optim. Theory Appl, 123, pp. 497-517Dirkse, S.P., Ferris, M.C., The PATH solver: A non-monotone stabilization scheme for mixed complementarity problems (1995) Optim. Methods Softw, 5, pp. 123-156Dolan, E.D., Moré, J.J., Benchmarking optimization software with performance profiles (2002) Math. Program, 91, pp. 201-213Dostál, Z., Inexact semimonotonic augmented Lagrangians with optimal feasibility convergence for convex bound and equality constrained quadratic programming (2005) SIAM J. Numer. Anal, 43, pp. 96-115Dostál, Z., Friedlander, A., Santos, S.A., Augmented Lagrangian with adaptive precision control for quadratic programming with simple bounds and equality constraints (2003) SIAM J. Optim, 13, pp. 1120-1140Dostál, Z., Gomes, F.A.M., Santos, S.A., Duality based domain decomposition with natural coarse space for variational inequalities (2000) J. Comput. Appl. Math, 126, pp. 397-415Facchinei, F., Fischer, A., Kanzow, C., Regularity properties of a semismooth reformulation of variational inequalities (1998) SIAM J. Optim, 8, pp. 850-869Facchinei, F., Fischer, A., Kanzow, C: A semismooth Newton method for variational inequalities: The case of box constraints. In: Ferris, M.C., Pang, J.-S. (eds.) Complementarity and Variational Problems: State of the Art., pp. 76-90. SIAM, Philadelphia (1997)Facchinei, F., Fischer, A., Kanzow, C., Peng, J.-M., A simply constrained reformulation of KKT systems arising from variational inequalities (1999) Appl. Math. Optim, 40, pp. 19-37Facchinei, F., Pang, J.-S., Finite-Dimensional Variational Inequalities and Complementary Problems (2000) Springer Series in Operation Research, , Springer, Berlin Heidelberg New YorkFerris, M.C., Munson, T.S.: PATH 4.6 User Manual, http://www.gams.com/ solvers/solvers.htm#PATHFletcher, R., (1987) Practical Methods of Optimization, , Academic, LondonHager, W.W., Analysis and implementation of a dual algorithm for constrained optimization (1993) J. Optim. Theory Appl, 79, pp. 37-71Harker, P.T., Pang, J.-S., Finite-dimensional variational inequality and nonlinear complementary problems: A survey of theory, algorithms and applications (1990) Math. Program, 48, pp. 161-220Luo, Z.-Q., Pang, J.-S., Ralph, D., (1996) Mathematical programs with equilibrium constraints, , Cambridge University Press, CambridgeMangasarian, O.L., Fromovitz, S., The Fritz-John necessary optimality conditions in presence of equality and inequality constraints (1967) J. Math. Anal. Appl, 17, pp. 37-47Qi, L., Wei, Z., On the constant positive linear dependence condition and its application to SQP methods (2000) SIAM J. Optim, 10, pp. 963-981Raydan, M., On the Barzilai and Borwein choice of steplength for the gradient method (1993) IMA J. Numer. Anal, 13, pp. 321-326Raydan, M., The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem (1997) SIAM J. Optim, 7, pp. 26-33Rockafellar, R.T., The multiplier method of Hestenes and Powell applied to convex programming (1973) J. Optim. Theory Appl, 12, pp. 555-562Rockafellar, R.T., Lagrange multipliers and optimality (1993) SIAM Rev, 35, pp. 183-238Shapiro, A., Sensitivity analysis of parameterized variational inequalities (2005) Math. Oper. Res, 30, pp. 109-12