This document is an introduction to the Matlab package SDLS (Semi-Definite
Least-Squares) for solving least-squares problems over convex symmetric cones.
The package is shortly presented through the addressed problem, a sketch of the
implemented algorithm, the syntax and calling sequences, a simple numerical
example and some more advanced features. The implemented method consists in
solving the dual problem with a quasi-Newton algorithm. We note that SDLS is
not the most competitive implementation of this algorithm: efficient, robust,
commercial implementations are available (contact the authors). Our main goal
with this Matlab SDLS package is to provide a simple, user-friendly software
for solving and experimenting with semidefinite least-squares problems. Up to
our knowledge, no such freeware exists at this date

Henrion, Didier

Malick, Jerome

English

arXiv

This document is an introduction to the Matlab package SDLS (Semi-Definite Least-Squares) for solving least-squares problems over convex symmetric cones. The package is shortly presented through the addressed problem, a sketch of the implemented algorithm, the syntax and calling sequences, a simple numerical example and some more advanced features. The implemented method consists in solving the dual problem with a quasi-Newton algorithm. We note that SDLS is not the most competitive implementation of this algorithm: efficient, robust, commercial implementations are available (contact the authors). Our main goal with this Matlab SDLS package is to provide a simple, user-friendly software for solving and experimenting with semidefinite least-squares problems. Up to our knowledge, no such freeware exists at this date

Malick, Jérôme

Scientific Publications of the University of Toulouse II Le Mirail

SDLS: a Matlab package for solving conic least-squares problems

INRIA a CCSD electronic archive server

HAL Id: hal-00172439https://hal.archives-ouvertes.fr/hal-00172439Preprint submitted on 17 Sep 2007HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.SDLS: a Matlab package for solving conic least-squaresproblemsDidier Henrion, Jérôme MalickTo cite this version:Didier Henrion, Jérôme Malick. SDLS: a Matlab package for solving conic least-squares problems.2007. ￿hal-00172439￿hal-00172439, version 1 - 17 Sep 2007SDLS: a Matlab package forsolving conic least-squares problemsDidier Henrion1,2 Jérôme Malick3September 17, 2007AbstractThis document is an introduction to the Matlab package SDLS (Semi-DefiniteLeast-Squares) for solving least-squares problems over convex symmetric cones. Thepackage is shortly presented through the addressed problem, a sketch of the imple-mented algorithm, the syntax and calling sequences, a simple numerical exampleand some more advanced features. The implemented method consists in solving thedual problem with a quasi-Newton algorithm. We note that SDLS is not the mostcompetitive implementation of this algorithm: efficient, robust, commercial imple-mentations are available (contact the authors). Our main goal with this MatlabSDLS package is to provide a simple, user-friendly software for solving and exper-imenting with semidefinite least-squares problems. Up to our knowledge, no suchfreeware exists at this date.1 General presentationSDLS is a Matlab freeware solving approximately convex conic-constrained least-squaresproblems. Geometrically, such problems amount to finding the best approximation of apoint in the intersection of a convex symmetric cone with an affine subspace. In mathe-matical terms, those problems can be cast as follows:minx12‖x − c‖2s.t. Ax = bx ∈ K(1)where x ∈ Rn has to be found, A ∈ Rm×n, b ∈ Rm, c ∈ Rn are given data and K is aconvex symmetric cone. The norm appearing in the objective function is ‖x‖ =√xT x,the Euclidean norm associated with the standard inner product in Rn. Note that thereexists a unique solution to this optimization problem.In practice, K must be expressed as a direct product of linear, quadratic and semidef-inite cones. It is always assumed that matrix A has full row-rank, otherwise the problemcan be reformulated by eliminating redundant equality constraints.1LAAS-CNRS, University of Toulouse (France)2Czech Technical University in Prague (Czech Republic)3CNRS, Laboratoire Jean Kunztmann, University of Grenoble (France)12 AlgorithmSDLS is an implementation of the algorithm described in [2], which consists in solvingthe dual to problem (1). Specifically, the dual problem is (up to the sign and an additiveconstant)miny f(y) =12‖pK(c + AT y)‖2 − bT y (2)where pK(z) denotes the orthogonal projection of vector z onto the cone K. It is easy toprove that this dual problem is smooth and convex, so it can be solved by classical op-timization algorithms. Among them, quasi-Newton algorithms are known to be efficient,and SDLS uses a Matlab implementation of the BFGS algorithm available in the Matlabfreeware HANSO [1]. The other key numerical component of SDLS is eigenvalue decom-position for symmetric matrices achieved by Matlab’s built-in linear algebra function eig.We insist on the two following features:• Simplicity: the SDLS package consists of only 4 Matlab interpreted m-files callingone external package and a built-in linear algebra function.• Easy-to-use: as explained in the next section, the syntax follows standard patterns.The implementation of the algorithm of [2] in SDLS is probably not the most efficient.It is not meant to outperform neither the professional or commercial implementations ofthis method, nor other methods solving the same problem. Our main goal is to provide asimple, user-friendly, free software for solving and experimenting with semidefinite least-squares problems. Up to our knowledge, no such freeware exists at this date.3 SyntaxSDLS has the same syntax as the widely used freeware SeDuMi for solving linear problemsover convex symmetric cones [3]. Even though SeDuMi and SDLS share the same callingsyntax, note that SeDuMi is aimed at solving a different problem. Namely, SeDuMiminimizes a linear objective function cT x while SDLS minimizes a quadratic objectivefunction as in (1).The basic calling syntax is:>> [x,y] = sdls(A,b,c,K)Input arguments A, b, c are real-valued matrices as in (1). SDLS exploits data sparsity,so that A, b and c can be Matlab objects of class sparse. If the third input argument isempty or not specified, then c is set to zero.Input argument K is a structure describing the components of the cone:• K.f is the number of free, i.e. unrestricted components. E.g. if K.f=2 then x(1:2)is unrestricted. These are always the first components in x• K.l is the dimension of the linear cone, i.e. the number of nonnegative components.E.g. if K.f=2 and K.l=8 then x(3:10)>=02• K.q lists the dimensions of the quadratic cones. E.g. if K.l=10 and K.q=[3 7]then x(11)>=norm(x(12:13)) and x(14)>=norm(x(15:20)). The correspondingcomponents of x immediately follow the K.l nonnegative ones• K.s lists the dimensions of the semidefinite cones. E.g. if K.l=20 and K.s=[4 3],then reshape(x(21:36),4,4) and reshape(x(37:45),3,3) are symmetric positivesemidefinite matrices. These components are always the last entries in xNote that rotated quadratic cones (K.r) and complex Hermitian components (K.xcomplex,K.scomplex, K.ycomplex) are currently not supported. Please refer to SeDuMi’s docu-mentation [3] for more details on the meaning of the fields in structure K. If the fourthinput argument is empty or not specified, then K is the linear cone, e.g. K.f=size(A,2).Output argument x (same dimension as c) is the solution to SDLS problem (1),whereas y (same dimension as b) is the corresponding dual vector solving the smoothproblem (2).4 InstallationSDLS 1.0 is developed for Matlab version 7.2 (release 2006). It consists of a tar.gz orzip archive that can be downloaded fromwww.laas.fr/∼henrion/software/sdlsUnpacking the archive creates a subdirectory sdls that should be activated under yourMatlab environment, either with the inline command addpath or with the Set Path entryof the File menu.The package HANSO, available atwww.cs.nyu.edu/overton/software/hansomust also be installed and activated under your Matlab environment.If SDLS is correctly installed, it should generate the following output:>> x = sdls([1, 0],1)x =105 ExampleAs a simple illustrative example of the use of SDLS we consider the problem of calibratingcorrelation matrices.Correlation matrices are symmetric positive semidefinite matrices with ones along thediagonal. They play an important role in probability and statistics but also in combinato-rial optimization. Consider a symmetric matrix C obtained from a correlation matrix aftermodifications of some entries, for example due to measurement errors or post-processing.Those modifications may alter the positive semidefiniteness of the matrix. We want to3restore basic properties (positive semidefiniteness and ones along the diagonal) by com-puting the correlation matrix nearest to C in the Euclidean sense. This problem canwritten in the form (1).For illustration, consider the problem of computing a nearest correlation matrix ofsize 3. Define>> A=sparse(3,3); A(1,1)=1; A(2,5)=1; A(3,9)=1;>> b=ones(3,1); % ones along diagonal>> C=[1 1/2 1;1/2 1 1/4;1 1/4 1]C =1.0000 0.5000 1.00000.5000 1.0000 0.25001.0000 0.2500 1.0000>> c=C(:);>> K=[]; K.s=3Matrix C was obtained by adding 1/2 to entries (1,3) and (3,1) of a correlation matrix.Modifying these 2 entries destroyed the positive semidefiniteness:>> min(eig(C))ans =-0.0349Then we run the calibration process using SDLS:>> [x,y] = sdls(A,b,c,K);>> X = reshape(x,K.s,K.s)X =1.0000 0.4910 0.96840.4910 1.0000 0.25820.9684 0.2582 1.0000This way, we do not recover the original correlation matrix, but we get the correlationmatrix nearest to C in the Euclidean sense. Indeed we observe that the diagonal onesremain whereas positive semidefiniteness is restored:>> min(eig(X))ans =3.0097e-166 Advanced use6.1 Input argumentsOptionally, tuning parameters can be specified as a fifth input argument:>> [x,y] = sdls(A,b,c,K,pars)Input argument pars is a structure containing the following fields:4• pars.eps: relative accuracy (default 1e-6)• pars.fid: screen output (0 for no output, default 1)• pars.maxit: maximum number of iterations for the BFGS solver (default 200)• pars.reg: regularization parameter (default 0)• pars.scaling: scaling of the linear equation (false for no scaling, default true)The stopping rule of the algorithm is norm(A*x-b) <= max(1,norm(b))*pars.epswhere pars.eps is the expected relative accuracy. Since BFGS is essentially a first-orderalgorithm, pars.eps should not be set too small. Typical values are between 1e-4 and1e-8, depending on the problem dimensions and scaling.The regularization parameter pars.reg should be used if one suspects that the qual-ification constraint does not hold. That is, the affine subspace may be tangent to thecone, or equivalently there may be no point satisfying the affine constraint in the interiorof the cone. A typical value is pars.reg=1e-6.When pars.scaling=true, the scaling consists simply in dividing input data A andb by max(1,norm(b)). Otherwise, the input data are left unchanged.6.2 Output argumentsAn additional third output argument:>> [x,y,info] = sdls(A,b,c,K)can be retrieved. Argument info contains information on the objective function of dualproblem (2), as provided by the BFGS solver:• info.f: function value f(y) at the optimum• info.g: gradient of f(y) at the optimum• info.H: approximation of inverse Hessian of f(y) at the optimum• info.time: CPU timeThe norm of the residual norm(A*x-b) is equal to the norm of the gradient norm(info.g).It should be less than the absolute accuracy max(1,norm(b))*pars.eps (see above) andit can be used as an estimate of the quality of the solution produced by SDLS.References[1] J.V. Burke, A.S. Lewis and M.L. Overton. HANSO (hybrid algorithmfor non-smooth optimization): a Matlab package based on BFGS, bun-dle and gradient sampling methods. Version 1.0 released in 2006. Seewww.cs.nyu.edu/faculty/overton/software/hanso5[2] J. Malick. A dual approach to semidefinite least-squares problems. SIAM J. MatrixAnal. Appl. Vol. 26, No. 1, pp. 272-284, 2004.[3] J.F. Sturm. Using SeDuMi 1.02, a Matlab toolbox for optimization over symmet-ric cones. Optimization Methods and Software. Vol. 11-12, pp. 625–653, 1999. Seesedumi.mcmaster.ca6

Hal - Université Grenoble Alpes

HAL-INSA Toulouse

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.244.7455

SDLS: a Matlab package for solving conic least-squares problems

Abstract

Similar works

Full text

Available Versions

Scientific Publications of the University of Toulouse II Le Mirail

INRIA a CCSD electronic archive server

Hal - Université Grenoble Alpes

HAL-INSA Toulouse