Some preconditioners for systems of linear inequalities

We show that a combination of two simple preprocessing steps would generally improve the conditioning of a homogeneous system of linear inequalities. Our approach is based on a comparison among three different but related notions of conditioning for linear inequalities

Rescaled coordinate descent methods for linear programming

We propose two simple polynomial-time algorithms to find a positive solution to Ax=0Ax=0 . Both algorithms iterate between coordinate descent steps similar to von Neumann’s algorithm, and rescaling steps. In both cases, either the updating step leads to a substantial decrease in the norm, or we can infer that the condition measure is small and rescale in order to improve the geometry. We also show how the algorithms can be extended to find a solution of maximum support for the system Ax=0Ax=0 , x≥0x≥0 . This is an extended abstract. The missing proofs will be provided in the full version

Faster Margin Maximization Rates for Generic Optimization Methods

First-order optimization methods tend to inherently favor certain solutions over others when minimizing a given training objective with multiple local optima. This phenomenon, known as implicit bias, plays a critical role in understanding the generalization capabilities of optimization algorithms. Recent research has revealed that gradient-descent-based methods exhibit an implicit bias for the $\ell_2$-maximal margin classifier in the context of separable binary classification. In contrast, generic optimization methods, such as mirror descent and steepest descent, have been shown to converge to maximal margin classifiers defined by alternative geometries. However, while gradient-descent-based algorithms demonstrate fast implicit bias rates, the implicit bias rates of generic optimization methods have been relatively slow. To address this limitation, in this paper, we present a series of state-of-the-art implicit bias rates for mirror descent and steepest descent algorithms. Our primary technique involves transforming a generic optimization algorithm into an online learning dynamic that solves a regularized bilinear game, providing a unified framework for analyzing the implicit bias of various optimization methods. The accelerated rates are derived leveraging the regret bounds of online learning algorithms within this game framework

Rescaling algorithms for linear conic feasibility

We propose simple polynomial-time algorithms for two linear conic feasibility problems. For a matrix A ∈ R m× n, the kernel problem requires a positive vector in the kernel of A, and the image problem requires a positive vector in the image of A T. Both algorithms iterate between simple first-order steps and rescaling steps. These rescalings improve natural geometric potentials. If Goffin's condition measure ρ A is negative, then the kernel problem is feasible, and the worst-case complexity of the kernel algorithm is O((m 3n + mn 2)log|ρ A| −1); if ρ A > 0, then the image problem is feasible, and the image algorithm runs in time O(m 2n 2 log ρ A −1). We also extend the image algorithm to the oracle setting. We address the degenerate case ρA = 0 by extending our algorithms to find maximum support nonnegative vectors in the kernel of A and in the image of A T. In this case, the running time bounds are expressed in the bit-size model of computation: for an input matrix A with integer entries and total encoding length L, the maximum support kernel algorithm runs in time O((m 3n + mn 2)L), whereas the maximum support image algorithm runs in time O(m 2n 2L). The standard linear programming feasibility problem can be easily reduced to either maximum support problems, yielding polynomial-time algorithms for linear programming

A New Extension of Chubanov's Method to Symmetric Cones

We propose a new variant of Chubanov's method for solving the feasibility problem over the symmetric cone by extending Roos's method (2018) of solving the feasibility problem over the nonnegative orthant. The proposed method considers a feasibility problem associated with a norm induced by the maximum eigenvalue of an element and uses a rescaling focusing on the upper bound for the sum of eigenvalues of any feasible solution to the problem. Its computational bound is (i) equivalent to that of Roos's original method (2018) and superior to that of Louren\c{c}o et al.'s method (2019) when the symmetric cone is the nonnegative orthant, (ii) superior to that of Louren\c{c}o et al.'s method (2019) when the symmetric cone is a Cartesian product of second-order cones, (iii) equivalent to that of Louren\c{c}o et al.'s method (2019) when the symmetric cone is the simple positive semidefinite cone, and (iv) superior to that of Pena and Soheili's method (2017) for any simple symmetric cones under the feasibility assumption of the problem imposed in Pena and Soheili's method (2017). We also conduct numerical experiments that compare the performance of our method with existing methods by generating instances in three types: strongly (but ill-conditioned) feasible instances, weakly feasible instances, and infeasible instances. For any of these instances, the proposed method is rather more efficient than the existing methods in terms of accuracy and execution time.Comment: 44 pages; Department of Policy and Planning Sciences Discussion Paper Series No. 1378, University of Tsukub

Post-Processing with Projection and Rescaling Algorithms for Semidefinite Programming

We propose the algorithm that solves the symmetric cone programs (SCPs) by iteratively calling the projection and rescaling methods the algorithms for solving exceptional cases of SCP. Although our algorithm can solve SCPs by itself, we propose it intending to use it as a post-processing step for interior point methods since it can solve the problems more efficiently by using an approximate optimal (interior feasible) solution. We also conduct numerical experiments to see the numerical performance of the proposed algorithm when used as a post-processing step of the solvers implementing interior point methods, using several instances where the symmetric cone is given by a direct product of positive semidefinite cones. Numerical results show that our algorithm can obtain approximate optimal solutions more accurately than the solvers. When at least one of the primal and dual problems did not have an interior feasible solution, the performance of our algorithm was slightly reduced in terms of optimality. However, our algorithm stably returned more accurate solutions than the solvers when the primal and dual problems had interior feasible solutions.Comment: 78 page