On the performance of algorithms for the minimization of ℓ1\ell_1-penalized functionals

The problem of assessing the performance of algorithms used for the minimization of an $\ell_1$-penalized least-squares functional, for a range of penalty parameters, is investigated. A criterion that uses the idea of `approximation isochrones' is introduced. Five different iterative minimization algorithms are tested and compared, as well as two warm-start strategies. Both well-conditioned and ill-conditioned problems are used in the comparison, and the contrast between these two categories is highlighted.Comment: 18 pages, 10 figures; v3: expanded version with an additional synthetic test problem

Approximation Complexity of Complex-Weighted Degree-Two Counting Constraint Satisfaction Problems

Constraint satisfaction problems have been studied in numerous fields with practical and theoretical interests. In recent years, major breakthroughs have been made in a study of counting constraint satisfaction problems (or #CSPs). In particular, a computational complexity classification of bounded-degree #CSPs has been discovered for all degrees except for two, where the "degree" of an input instance is the maximal number of times that each input variable appears in a given set of constraints. Despite the efforts of recent studies, however, a complexity classification of degree-2 #CSPs has eluded from our understandings. This paper challenges this open problem and gives its partial solution by applying two novel proof techniques--T_{2}-constructibility and parametrized symmetrization--which are specifically designed to handle "arbitrary" constraints under randomized approximation-preserving reductions. We partition entire constraints into four sets and we classify the approximation complexity of all degree-2 #CSPs whose constraints are drawn from two of the four sets into two categories: problems computable in polynomial-time or problems that are at least as hard as #SAT. Our proof exploits a close relationship between complex-weighted degree-2 #CSPs and Holant problems, which are a natural generalization of complex-weighted #CSPs.Comment: A4, 10pt, 23 pages. This is a complete version of the paper that appeared in the Proceedings of the 17th Annual International Computing and Combinatorics Conference (COCOON 2011), Lecture Notes in Computer Science, vol.6842, pp.122-133, Dallas, Texas, USA, August 14-16, 201

A Dichotomy Theorem for the Approximate Counting of Complex-Weighted Bounded-Degree Boolean CSPs

We determine the computational complexity of approximately counting the total weight of variable assignments for every complex-weighted Boolean constraint satisfaction problem (or CSP) with any number of additional unary (i.e., arity 1) constraints, particularly, when degrees of input instances are bounded from above by a fixed constant. All degree-1 counting CSPs are obviously solvable in polynomial time. When the instance's degree is more than two, we present a dichotomy theorem that classifies all counting CSPs admitting free unary constraints into exactly two categories. This classification theorem extends, to complex-weighted problems, an earlier result on the approximation complexity of unweighted counting Boolean CSPs of bounded degree. The framework of the proof of our theorem is based on a theory of signature developed from Valiant's holographic algorithms that can efficiently solve seemingly intractable counting CSPs. Despite the use of arbitrary complex weight, our proof of the classification theorem is rather elementary and intuitive due to an extensive use of a novel notion of limited T-constructibility. For the remaining degree-2 problems, in contrast, they are as hard to approximate as Holant problems, which are a generalization of counting CSPs.Comment: A4, 10pt, 20 pages. This revised version improves its preliminary version published under a slightly different title in the Proceedings of the 4th International Conference on Combinatorial Optimization and Applications (COCOA 2010), Lecture Notes in Computer Science, Springer, Vol.6508 (Part I), pp.285--299, Kailua-Kona, Hawaii, USA, December 18--20, 201

FPTAS for Counting Monotone CNF

A monotone CNF formula is a Boolean formula in conjunctive normal form where each variable appears positively. We design a deterministic fully polynomial-time approximation scheme (FPTAS) for counting the number of satisfying assignments for a given monotone CNF formula when each variable appears in at most $5$ clauses. Equivalently, this is also an FPTAS for counting set covers where each set contains at most $5$ elements. If we allow variables to appear in a maximum of $6$ clauses (or sets to contain $6$ elements), it is NP-hard to approximate it. Thus, this gives a complete understanding of the approximability of counting for monotone CNF formulas. It is also an important step towards a complete characterization of the approximability for all bounded degree Boolean #CSP problems. In addition, we study the hypergraph matching problem, which arises naturally towards a complete classification of bounded degree Boolean #CSP problems, and show an FPTAS for counting 3D matchings of hypergraphs with maximum degree $4$. Our main technique is correlation decay, a powerful tool to design deterministic FPTAS for counting problems defined by local constraints among a number of variables. All previous uses of this design technique fall into two categories: each constraint involves at most two variables, such as independent set, coloring, and spin systems in general; or each variable appears in at most two constraints, such as matching, edge cover, and holant problem in general. The CNF problems studied here have more complicated structures than these problems and require new design and proof techniques. As it turns out, the technique we developed for the CNF problem also works for the hypergraph matching problem. We believe that it may also find applications in other CSP or more general counting problems.Comment: 24 pages, 2 figures. version 1=>2: minor edits, highlighted the picture of set cover/packing, and an implication of our previous result in 3D matchin

Approximation of high-dimensional parametric PDEs

Parametrized families of PDEs arise in various contexts such as inverse problems, control and optimization, risk assessment, and uncertainty quantification. In most of these applications, the number of parameters is large or perhaps even infinite. Thus, the development of numerical methods for these parametric problems is faced with the possible curse of dimensionality. This article is directed at (i) identifying and understanding which properties of parametric equations allow one to avoid this curse and (ii) developing and analyzing effective numerical methodd which fully exploit these properties and, in turn, are immune to the growth in dimensionality. The first part of this article studies the smoothness and approximability of the solution map, that is, the map $a\mapsto u(a)$ where $a$ is the parameter value and $u(a)$ is the corresponding solution to the PDE. It is shown that for many relevant parametric PDEs, the parametric smoothness of this map is typically holomorphic and also highly anisotropic in that the relevant parameters are of widely varying importance in describing the solution. These two properties are then exploited to establish convergence rates of $n$-term approximations to the solution map for which each term is separable in the parametric and physical variables. These results reveal that, at least on a theoretical level, the solution map can be well approximated by discretizations of moderate complexity, thereby showing how the curse of dimensionality is broken. This theoretical analysis is carried out through concepts of approximation theory such as best $n$-term approximation, sparsity, and $n$-widths. These notions determine a priori the best possible performance of numerical methods and thus serve as a benchmark for concrete algorithms. The second part of this article turns to the development of numerical algorithms based on the theoretically established sparse separable approximations. The numerical methods studied fall into two general categories. The first uses polynomial expansions in terms of the parameters to approximate the solution map. The second one searches for suitable low dimensional spaces for simultaneously approximating all members of the parametric family. The numerical implementation of these approaches is carried out through adaptive and greedy algorithms. An a priori analysis of the performance of these algorithms establishes how well they meet the theoretical benchmarks

Krylov Methods for Large Sparse Systems: A Comprehensive Overview

In this paper are analyzed behavior and properties for different Krylov methods applied in different categories of problems. These categories often include PDEs, econometrics and network models, which are represented by large sparse systems. For our empirical analysis are taken into consideration size, the density of non-zero elements, symmetry/un-symmetry, eigenvalue distribution, also well/ill-conditioned and random systems. Convergence, approximation error and residuals are compared for the full version of methods, some restarted methods and preconditioned methods. Two preconditioners are considered respectively, ILU(0) and IC(0) by using at least five preconditioning techniques. In each case, empirical results show which technique is best to use based on properties of the system and are backed up by general theoretical information already found on Krylov space methods