Greedy Algorithms for Cone Constrained Optimization with Convergence Guarantees

Greedy optimization methods such as Matching Pursuit (MP) and Frank-Wolfe (FW) algorithms regained popularity in recent years due to their simplicity, effectiveness and theoretical guarantees. MP and FW address optimization over the linear span and the convex hull of a set of atoms, respectively. In this paper, we consider the intermediate case of optimization over the convex cone, parametrized as the conic hull of a generic atom set, leading to the first principled definitions of non-negative MP algorithms for which we give explicit convergence rates and demonstrate excellent empirical performance. In particular, we derive sublinear ($\mathcal{O}(1/t)$) convergence on general smooth and convex objectives, and linear convergence ($\mathcal{O}(e^{-t})$) on strongly convex objectives, in both cases for general sets of atoms. Furthermore, we establish a clear correspondence of our algorithms to known algorithms from the MP and FW literature. Our novel algorithms and analyses target general atom sets and general objective functions, and hence are directly applicable to a large variety of learning settings.Comment: NIPS 201

On finding exact solutions of linear programs in the oracle model

We consider linear programming in the oracle model: mincT x s.t. x ∊ P, where the polyhedron P = {x ∊ ℝn: Ax ≤ b} is given by a separation oracle that returns violated inequalities from the system Ax ≤ b. We present an algorithm that finds exact primal and dual solutions using O(n2 log(n/δ)) oracle calls and O(n4 log(n/δ) + n6 log log(1/δ)) arithmetic operations, where δ is a geometric condition number associated with the system (A, b). These bounds do not depend on the cost vector c. The algorithm works in a black box manner, requiring a subroutine for approximate primal and dual solutions; the above running times are achieved when using the cutting plane method of Jiang, Lee, Song, and Wong (STOC 2020) for this subroutine. Whereas approximate solvers may return primal solutions only, we develop a general framework for extracting dual certificates based on the work of Burrell and Todd (Math. Oper. Res. 1985). Our algorithm works in the real model of computation, and extends results by Grötschel, Lovász, and Schrijver (Prog. Comb. Opt. 1984), and by Frank and Tardos (Combinatorica 1987) on solving LPs in the bit-complexity model. We show that under a natural assumption, simultaneous Diophantine approximation in these results can be avoided

Post-Processing with Projection and Rescaling Algorithms for Semidefinite Programming

We propose the algorithm that solves the symmetric cone programs (SCPs) by iteratively calling the projection and rescaling methods the algorithms for solving exceptional cases of SCP. Although our algorithm can solve SCPs by itself, we propose it intending to use it as a post-processing step for interior point methods since it can solve the problems more efficiently by using an approximate optimal (interior feasible) solution. We also conduct numerical experiments to see the numerical performance of the proposed algorithm when used as a post-processing step of the solvers implementing interior point methods, using several instances where the symmetric cone is given by a direct product of positive semidefinite cones. Numerical results show that our algorithm can obtain approximate optimal solutions more accurately than the solvers. When at least one of the primal and dual problems did not have an interior feasible solution, the performance of our algorithm was slightly reduced in terms of optimality. However, our algorithm stably returned more accurate solutions than the solvers when the primal and dual problems had interior feasible solutions.Comment: 78 page

Analytic Torsion on Manifolds with Edges

Let (M,g) be an odd-dimensional incomplete compact Riemannian singular space with a simple edge singularity. We study the analytic torsion on M, and in particular consider how it depends on the metric g. If g is an admissible edge metric, we prove that the torsion zeta function is holomorphic near s = 0, hence the torsion is well-defined, but possibly depends on g. In general dimensions, we prove that the analytic torsion depends only on the asymptotic structure of g near the singular stratum of M; when the dimension of the edge is odd, we prove that the analytic torsion is independent of the choice of admissible edge metric. The main tool is the construction, via the methodology of geometric microlocal analysis, of the heat kernel for the Friedrichs extension of the Hodge Laplacian in all degrees. In this way we obtain detailed asymptotics of this heat kernel and its trace.Comment: 36 pages, 5 figures, v2: minor improvement

Lagrangian fibrations on blowups of toric varieties and mirror symmetry for hypersurfaces

We consider mirror symmetry for (essentially arbitrary) hypersurfaces in (possibly noncompact) toric varieties from the perspective of the Strominger-Yau-Zaslow (SYZ) conjecture. Given a hypersurface $H$ in a toric variety $V$ we construct a Landau-Ginzburg model which is SYZ mirror to the blowup of $V\times\mathbb{C}$ along $H\times 0$, under a positivity assumption. This construction also yields SYZ mirrors to affine conic bundles, as well as a Landau-Ginzburg model which can be naturally viewed as a mirror to $H$. The main applications concern affine hypersurfaces of general type, for which our results provide a geometric basis for various mirror symmetry statements that appear in the recent literature. We also obtain analogous results for complete intersections.Comment: 83 pages; v2: added appendix discussing the analytic structure on moduli of objects in the Fukaya category; v3: further clarifications in response to referee report; v4: further clarifications throughout, especially sections 4 and 7 and appendix A; added appendix B on the geometry of reduced space