International audienceSparse signal reconstruction is performed by minimizing the sum of a least-squares fit regularized with a piecewise rational approximation of 0. We show the benefit of an oracle that yields the sign of the signal when using a recent methodology for global polynomial or semi-algebraic minimization. The computational time and memory cost are both decrease

Castella, Marc

Marmin, Arthur

Pesquet, Jean-Christophe

HAL-CentraleSupelec

Sparse signal reconstruction with a sign oracle

INRIA a CCSD electronic archive server

HAL Id: hal-02196881https://hal.archives-ouvertes.fr/hal-02196881Submitted on 29 Jul 2019HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Sparse signal reconstruction with a sign oracleArthur Marmin, Marc Castella, Jean-Christophe PesquetTo cite this version:Arthur Marmin, Marc Castella, Jean-Christophe Pesquet. Sparse signal reconstruction with a signoracle. SPARS 2019 - Signal Processing with Adaptive Sparse Structured Representations - Workshop,Jul 2019, Toulouse, France. ￿hal-02196881￿1Sparse Signal Reconstruction with a Sign OracleArthur Marmin∗, Marc Castella†, Jean-Christophe Pesquet∗∗ Center for Visual Computing, CentraleSupélec, INRIA, Université Paris-Saclay, Gif-sur-Yvette, France{arthur.marmin, jean-christophe.pesquet}@centralesupelec.fr† SAMOVAR, Télécom SudParis, CNRS Université Paris-Saclay, Evry, Francemarc.castella@telecom-sudparis.euAbstract—Sparse signal reconstruction is performed by mini-mizing the sum of a least-squares fit regularized with a piecewiserational approximation of `0. We show the benefit of an oraclethat yields the sign of the signal when using a recent methodol-ogy for global polynomial or semi-algebraic minimization. Thecomputational time and memory cost are both decreased.I. INTRODUCTIONWith the exponential growth of numerical data acquisitionand storage requirements, compressive sensing has becomea fundamental tool. The standard approach to promote sparsesolutions consists in adding an `0 penalization to a data-fit costfunction [1]. Since this yields NP hard optimization problems,surrogates for `0 that are continuous, exact and that do not addundesirable bias were proposed [2]–[7]. Nevertheless, they arenon convex and lead to intricate optimization problems whichcan be solved only approximately [5], [8]–[13]. Fortunately,many criteria are piecewise rational and the problem can thenbe reformulated in terms of polynomial optimization problemsbefore being solved exactly following a recent approach [14]–[16]. In this work, we explore the advantage provided by ana priori knowledge on the sign of the estimated signal.II. PROBLEM MODELLING AND METHODOLOGYWe consider the reconstruction of an unknown sparsediscrete-time signal x ∈ RT from observations y ∈ RT givenby the following linear degradation model y = Hx+w, wherew ∈ RT is a zero-mean white Gaussian noise and H ∈ RT×Tis a Toeplitz band matrix associated to a convolution filterof length L. We estimate x as a minimizer of the followingregularized criterion(∀x ∈ RT ) J (x) = 12‖y −Hx‖2 +T∑t=1Ψλ(|xt|) , (1)where Ψλ is a piecewise rational regularization that promotessparsity and depends on a parameter λ ∈]0,+∞[. Manycontinuous exact relaxations of `0 can be expressed in theabove form [2]–[7]. In particular, the penalization is generallyan even separable function which depends on |xt| only andcan be written Ψλ(|xt|). The minimization of (1) is firstreformulated as a polynomial optimization problem and thensolved using Lasserre’s method [16]. Our methodology isapplicable to any piecewise rational function. Since x is areal-valued signal, in our previous work, we used additionalvariables u ∈ RT such that u2 = |x|2 , u ≥ 0 as a substitutefor the absolute value in Ψλ.Lasserre’s method constructs a hierarchy of convex SDPproblems indexed by the relaxation order k. Solving each SDPproblem yields both a lower bound J ∗k on the criterion Jand a minimizer x̂ [17]. (J ∗k )k∈N is an increasing convergentsequence whose limit is the true optimal value of (1) [14].III. USE OF AN ORACLE ON THE SIGN OF xFor real-valued signals, convergence is observed for ordersk for which building and solving the corresponding SDPproblems is highly demanding in terms of computation andmemory storage. However, when x is a positive signal, weobserve [16] convergence at a lower order k. This thus sug-gests a method yielding similar results for real-valued signalsusing an oracle. Instead of (1), we minimize(∀v ∈ RT+) J̃ (v) =12∥∥∥y − H̃v∥∥∥2 + T∑t=1Ψλ(vt) ,where v ∈ RT+, H̃ = HDiag(ε), ε ∈ {−1, 1}T is thesign vector of x provided by the oracle, and Diag(ε) is thescalar matrix with diagonal elements ε. We build our oracleby solving (1) with lasso [18], i.e. Ψλ = λ Id. The availabilityof an oracle allows us to restrict the minimization of (1) topositive signals thanks to the new convolution matrix H̃. Anexact solution is thus retrieved by solving a SDP problemof fair dimension. Moreover, the use of an oracle decreasessignificantly the computation time since additional variablesu are no longer required.IV. NUMERICAL SIMULATIONSWe generate 100 Monte-Carlo realizations of the signalx with size T = 50 and 100 and 10% non-zero ele-ments drawn randomly from the uniform distribution on[−1,−2/3] ∪ [2/3, 1]. The length L of the filter is set to3. We use the penalization SCAD [5]. We solve the SDPcorresponding to relaxation orders 2, 3, and 4.Figure 1 shows the value of the criterion at the minimizerand the lower bound for different cases. When the oracle isavailable, we observe that the convergence is reached at k = 3since J̃ ∗k = J̃ (x̂our oracle). Conversely, when no informationon the sign of x is available, there is still a gap between J ∗kand J (x̂no oracle) at k = 4. Figure 2 shows the quality of ouroracle relative to the original criterion (1). Table I shows thecorresponding average computational times. Using our oraclesecures convergence for a lower relaxation order and thus leadsto a significant decrease of computational time. Table II showsthe relative error for the different estimated signals x̂.210 20 30 40 50 60 70 80 90 1000.40.60.8110 20 30 40 50 60 70 80 90 1000.40.60.81(a) With our oracle: in red J̃ (x̂our oracle), in blue J̃ ∗kfor k = 2 (top), 3 (bottom)20 40 60 80 10000.5120 40 60 80 1000.5120 40 60 80 1000.51(b) Without oracle: in red J (x̂no oracle), in blue J ∗kfor k = 2 (top), 3 (middle), and 4 (bottom)Fig. 1. Comparison between the lower bound and the optimal value of thecriterion for 100 tests with T = 50. The superposition of the two curvesindicates the convergence of Lasserre’s hierarchy. For the sake of clarity, thecases are ordered by increasing value of the lower bound.20 40 60 80 1000.40.60.811.2Fig. 2. Value of the original criterion J (x̂) for the different estimates x̂on 100 tests with T = 50 and k = 3. For the sake of clarity, the cases areordered by increasing value of J ∗3 .10 20 30 40 50 60 70 80 90 100-10110 20 30 40 50 60 70 80 90 100-10110 20 30 40 50 60 70 80 90 100-10110 20 30 40 50 60 70 80 90 100-101Fig. 3. Comparison of the different estimates of x (T = 100 and k = 3)TABLE IAVERAGE COMPUTATIONAL TIMEST = 50 T = 100Our oracle No oracle Our oracle No oraclek = 2 9s 9s 15s 16sk = 3 19s 25s 46s 1mink = 4 15min53s 27min38s 2h11s 3h18min43sTABLE IIAVERAGE RELATIVE RECONSTRUCTION ERROR‖x̂−x‖‖x‖ No oracle Our oracle Perfect oracle Lassok = 2 0.765 0.694 0.545 0.809k = 3 0.710 0.710 0.564REFERENCES[1] M. Nikolova, “Description of the minimizers of least squares regularizedwith `0 norm. Uniqueness of the global minimizer,” SIAM J. ImagingSci., vol. 6, no. 2, pp. 904–937, Jan. 2013.[2] T. Zhang, “Analysis of multi-stage convex relaxation for sparse regular-ization,” J. Mach. Learn. Res., vol. 11, pp. 1081–1107, Mar. 2010.[3] M. Artina, M. Fornasier, and F. Solombrino, “Linearly constrainednonsmooth and nonconvex minimization,” SIAM J. Optim., vol. 23, no. 3,pp. 1904–1937, Jan. 2013.[4] A. Jezierska, H. Talbot, O. Veksler, and D. Wesierski, “A fast solverfor truncated-convex priors: Quantized-convex split moves,” in LectureNotes in Computer Science. Springer Berlin Heidelberg, 2011, pp.45–58.[5] J. Fan and R. Li, “Variable selection via nonconcave penalized likelihoodand its oracle properties,” J. Am. Stat. Assoc., vol. 96, no. 456, pp. 1348–1360, Dec. 2001.[6] C.-H. Zhang, “Nearly unbiased variable selection under minimax con-cave penalty,” Ann. Appl. Stat., vol. 38, no. 2, pp. 894–942, Apr. 2010.[7] E. Soubies, L. Blanc-Féraud, and G. Aubert, “A continuous exact `0penalty (CEL0) for least squares regularized problem,” SIAM J. ImagingSci., vol. 8, no. 3, pp. 1607–1639, jan 2015.[8] P. Ochs, A. Dosovitskiy, T. Brox, and T. Pock, “On iteratively reweightedalgorithms for nonsmooth nonconvex optimization in computer vision,”SIAM J. Imaging Sci., vol. 8, no. 1, pp. 331–372, Jan. 2015.[9] E. J. Candès, M. B. Wakin, and S. P. Boyd, “Enhancing sparsity byreweighted `1 minimization,” J. Fourier Anal. Appl., vol. 14, no. 5-6,pp. 877–905, Oct. 2008.[10] P. Breheny and J. Huang, “Coordinate descent algorithms for nonconvexpenalized regression, with applications to biological feature selection,”Ann. Appl. Stat., vol. 5, no. 1, pp. 232–253, Mar. 2011.[11] A. Patrascu and I. Necoara, “Random coordinate descent methods for `0regularized convex optimization,” IEEE Trans. Automat. Contr., vol. 60,no. 7, pp. 1811–1824, July 2015.[12] T. Blumensath and M. E. Davies, “Iterative thresholding for sparseapproximations,” J. Fourier Anal. Appl., vol. 14, no. 5-6, pp. 629–654,Sept. 2008.[13] I. Selesnick, “Sparse regularization via convex analysis,” IEEE Trans.Signal Process., vol. 65, no. 17, pp. 4481–4494, Sept. 2017.[14] J. B. Lasserre, “Global optimization with polynomials and the problemof moments,” SIAM J. Optim., vol. 11, no. 3, pp. 796–817, Jan. 2001.[15] F. Bugarin, D. Henrion, and J. B. Lasserre, “Minimizing the sum ofmany rational functions,” Math. Program. Comput., vol. 8, no. 1, pp.83–111, Aug. 2015.[16] M. Castella, J.-C. Pesquet, and A. Marmin, “Rational optimization fornonlinear reconstruction with approximate `0 penalization,” IEEE Trans.Signal Process., vol. 67, pp. 1–1, 2018.[17] D. Henrion and J.-B. Lasserre, “Detecting global optimality and ex-tracting solutions in GloptiPoly,” in Positive Polynomials in Control.Springer Berlin Heidelberg, Sept. 2005, vol. 312, pp. 293–310.[18] R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. R.Stat. Soc. Ser. B Stat. Methodol., vol. 58, no. 1, pp. 267–288, 1996.

Archive Ouverte en Sciences de l'Information et de la Communication

https://hal.archives-ouvertes.fr/hal-02196881/document

Sparse signal reconstruction with a sign oracle

Abstract

Similar works

Full text

Available Versions

HAL-CentraleSupelec

INRIA a CCSD electronic archive server

Archive Ouverte en Sciences de l'Information et de la Communication