On the von Neumann and Frank-Wolfe Algorithms with Away Steps

The von Neumann algorithm is a simple coordinate-descent algorithm to determine whether the origin belongs to a polytope generated by a finite set of points. When the origin is in the of the polytope, the algorithm generates a sequence of points in the polytope that converges linearly to zero. The algorithm's rate of convergence depends on the radius of the largest ball around the origin contained in the polytope. We show that under the weaker condition that the origin is in the polytope, possibly on its boundary, a variant of the von Neumann algorithm that includes generates a sequence of points in the polytope that converges linearly to zero. The new algorithm's rate of convergence depends on a certain geometric parameter of the polytope that extends the above radius but is always positive. Our linear convergence result and geometric insights also extend to a variant of the Frank-Wolfe algorithm with away steps for minimizing a strongly convex function over a polytope

Frank-Wolfe Algorithms for Saddle Point Problems

We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW optimization, we provide the first proof of convergence of a FW-type saddle point solver over polytopes, thereby partially answering a 30 year-old conjecture. We also survey other convergence results and highlight gaps in the theoretical underpinnings of FW-style algorithms. Motivating applications without known efficient alternatives are explored through structured prediction with combinatorial penalties as well as games over matching polytopes involving an exponential number of constraints.Comment: Appears in: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017). 39 page

Rescaling algorithms for linear conic feasibility

We propose simple polynomial-time algorithms for two linear conic feasibility problems. For a matrix A ∈ R m× n, the kernel problem requires a positive vector in the kernel of A, and the image problem requires a positive vector in the image of A T. Both algorithms iterate between simple first-order steps and rescaling steps. These rescalings improve natural geometric potentials. If Goffin's condition measure ρ A is negative, then the kernel problem is feasible, and the worst-case complexity of the kernel algorithm is O((m 3n + mn 2)log|ρ A| −1); if ρ A > 0, then the image problem is feasible, and the image algorithm runs in time O(m 2n 2 log ρ A −1). We also extend the image algorithm to the oracle setting. We address the degenerate case ρA = 0 by extending our algorithms to find maximum support nonnegative vectors in the kernel of A and in the image of A T. In this case, the running time bounds are expressed in the bit-size model of computation: for an input matrix A with integer entries and total encoding length L, the maximum support kernel algorithm runs in time O((m 3n + mn 2)L), whereas the maximum support image algorithm runs in time O(m 2n 2L). The standard linear programming feasibility problem can be easily reduced to either maximum support problems, yielding polynomial-time algorithms for linear programming