48 research outputs found
Faster Algorithms for Structured Linear and Kernel Support Vector Machines
Quadratic programming is a ubiquitous prototype in convex programming. Many
combinatorial optimizations on graphs and machine learning problems can be
formulated as quadratic programming; for example, Support Vector Machines
(SVMs). Linear and kernel SVMs have been among the most popular models in
machine learning over the past three decades, prior to the deep learning era.
Generally, a quadratic program has an input size of , where
is the number of variables. Assuming the Strong Exponential Time Hypothesis
(), it is known that no algorithm exists
(Backurs, Indyk, and Schmidt, NIPS'17). However, problems such as SVMs usually
feature much smaller input sizes: one is given data points, each of
dimension , with . Furthermore, SVMs are variants with only
linear constraints. This suggests that faster algorithms are feasible, provided
the program exhibits certain underlying structures.
In this work, we design the first nearly-linear time algorithm for solving
quadratic programs whenever the quadratic objective has small treewidth or
admits a low-rank factorization, and the number of linear constraints is small.
Consequently, we obtain a variety of results for SVMs:
* For linear SVM, where the quadratic constraint matrix has treewidth ,
we can solve the corresponding program in time ;
* For linear SVM, where the quadratic constraint matrix admits a low-rank
factorization of rank-, we can solve the corresponding program in time
;
* For Gaussian kernel SVM, where the data dimension and
the squared dataset radius is small, we can solve it in time
. We also prove that when the squared dataset
radius is large, then time is required.Comment: New results: almost-linear time algorithm for Gaussian kernel SVM and
complementary lower bounds. Abstract shortened to meet arxiv requiremen
Research on Bending Fatigue Properties of Reinforced Macadam Foundation
Existing macadam base structures have poor resistance to bending deformation of pavement. The structural layer is challenged with serious cracking due to the long-term dynamic loads. Despite these issues, however, studies on reinforcement techniques for macadam base structure have been scarce. No reasonable and feasible method on how to improve the bending resistance of flexible base structures (e.g. macadam base structure) and prolong their bending fatigue life has yet been reported. Thus, this study proposed a method for reinforcing macadam base structures with geogrids to strengthen bending performance, anti-fatigue performance and dynamic stability of flexible base structures under different loading levels. The reinforced macadam base structure was investigated through a laboratory test of flexural-tensile strength and a three-point fatigue bending test. A cyclic loading test of reinforced and non-reinforced girder specimens was carried out under five stress levels. The effect of such a new reinforcement method in improving the bending fatigue properties of macadam base structures was analyzed by comparing the yield curves and fatigue lives of reinforced and non-reinforced specimens under different loading levels. Results demonstrate that the non-reinforced macadam base has poor bending resistance. The yield curve and fatigue life of non-reinforced specimens attenuated quickly as it approached the ultimate loading level. Non-reinforced specimens have low-cycle fatigue failures when the loads reach the 0.8 stress level and the ultimate displacement at failure is relatively small, accompanied by obvious failure surface. After geogrids are added, the fatigue life of specimens improved significantly. The yield curve shows no significant attenuation at a high loading level. The reinforced specimens can still maintain high-cycle fatigue failure under the 0.8 stress level and avoid the occurrence of low-cycle fatigue failures. Moreover, reinforced specimens can inhibit transmission of reflection cracks at midspan of cement-stabilized macadam base, delay the propagation of oblique shearing fractures close to the support, and strengthen the dynamic stability and durability of pavement structure with a macadam base. The conclusions of this study provide theoretical references to practical engineering applications of such new reinforcement technology for macadam bases
Discrete Element Simulation of Bending Deformation of Geogrid-Reinforced Macadam Base
The pavement bending deformation resistance of the existing macadam base structure is poor. The geogrid-reinforced macadam base can effectively strengthen the bending resistance of the pavement, but no international consensus has been reached at present over bending failure laws of reinforced macadam base structure. Discrete element models of semi-rigid base pavement structure, macadam base pavement structure, and geogrid-reinforced macadam base pavement structure were built based on MATDEM discrete element simulation program; loading calculation of the three models was conducted by taking their centers as loading positions; and model displacement nephogram, strain nephogram, and effects of different spans on their bending deformation were analyzed to reveal bending failure laws of reinforced macadam base and improvement effect of the geogrid on the anti-bending performance of the macadam structural layer. Finally, bending deformation laws of the three pavement structures and improvement effect of geogrid reinforcement on bending properties of the macadam base structure were established. The results show that under bending deformation of semi-rigid base, the vertical strain at the contract surface between the baseplate and soil base and horizontal strain at midspan position reach the maximum, which can easily lead to fracture and shear failure, and the macadam base layer can effectively isolate the tensile strain transmitted from bottom up. Through their own deformation, grids can transform surface pressure load into frictional resistance at the geogrid/soil interface and partial kinetic energy in the system into their own elastic potential energy to reduce the kinetic energy at the subbase layer. Geogrid reinforcement can improve the nonlinearity of macadam materials, reduce the fluctuation amplitude of the strain curve and displacement curve, lengthen the service life of the macadam base pavement structure, and improve its structural soundness under bending deformation. This study can provide a theoretical reference for numerical simulation of bending failure of geogrid-reinforced macadam base
Streaming Semidefinite Programs: Passes, Small Space and Fast Runtime
We study the problem of solving semidefinite programs (SDP) in the streaming
model. Specifically, constraint matrices and a target matrix , all of
size together with a vector are streamed to us
one-by-one. The goal is to find a matrix such
that is maximized, subject to
for all and . Previous algorithmic studies of SDP
primarily focus on \emph{time-efficiency}, and all of them require a
prohibitively large space in order to store \emph{all the
constraints}. Such space consumption is necessary for fast algorithms as it is
the size of the input. In this work, we design an interior point method (IPM)
that uses space, which is strictly sublinear in the
regime . Our algorithm takes passes, which
is standard for IPM. Moreover, when is much smaller than , our algorithm
also matches the time complexity of the state-of-the-art SDP solvers. To
achieve such a sublinear space bound, we design a novel sketching method that
enables one to compute a spectral approximation to the Hessian matrix in
space. To the best of our knowledge, this is the first method that
successfully applies sketching technique to improve SDP algorithm in terms of
space (also time)
Dynamic Tensor Product Regression
In this work, we initiate the study of \emph{Dynamic Tensor Product
Regression}. One has matrices and a label vector , and the goal is to solve the regression problem with the design matrix
being the tensor product of the matrices i.e.
. At each time step, one matrix receives a sparse change, and
the goal is to maintain a sketch of the tensor product so that the regression solution can be updated quickly.
Recomputing the solution from scratch for each round is very slow and so it is
important to develop algorithms which can quickly update the solution with the
new design matrix. Our main result is a dynamic tree data structure where any
update to a single matrix can be propagated quickly throughout the tree. We
show that our data structure can be used to solve dynamic versions of not only
Tensor Product Regression, but also Tensor Product Spline regression (which is
a generalization of ridge regression) and for maintaining Low Rank
Approximations for the tensor product.Comment: NeurIPS 202
Accelerating Frank-Wolfe Algorithm using Low-Dimensional and Adaptive Data Structures
In this paper, we study the problem of speeding up a type of optimization
algorithms called Frank-Wolfe, a conditional gradient method. We develop and
employ two novel inner product search data structures, improving the prior
fastest algorithm in [Shrivastava, Song and Xu, NeurIPS 2021].
* The first data structure uses low-dimensional random projection to reduce
the problem to a lower dimension, then uses efficient inner product data
structure. It has preprocessing time and
per iteration cost for small constant .
* The second data structure leverages the recent development in adaptive
inner product search data structure that can output estimations to all inner
products. It has preprocessing time and per iteration cost
.
The first algorithm improves the state-of-the-art (with preprocessing time
and per iteration cost ) in all
cases, while the second one provides an even faster preprocessing time and is
suitable when the number of iterations is small
Efficient Algorithm for Solving Hyperbolic Programs
Hyperbolic polynomials is a class of real-roots polynomials that has wide
range of applications in theoretical computer science. Each hyperbolic
polynomial also induces a hyperbolic cone that is of particular interest in
optimization due to its generality, as by choosing the polynomial properly, one
can easily recover the classic optimization problems such as linear programming
and semidefinite programming. In this work, we develop efficient algorithms for
hyperbolic programming, the problem in each one wants to minimize a linear
objective, under a system of linear constraints and the solution must be in the
hyperbolic cone induced by the hyperbolic polynomial. Our algorithm is an
instance of interior point method (IPM) that, instead of following the central
path, it follows the central Swath, which is a generalization of central path.
To implement the IPM efficiently, we utilize a relaxation of the hyperbolic
program to a quadratic program, coupled with the first four moments of the
hyperbolic eigenvalues that are crucial to update the optimization direction.
We further show that, given an evaluation oracle of the polynomial, our
algorithm only requires oracle calls, where is the number
of variables and is the degree of the polynomial, with extra arithmetic operations, where is the number of constraints
Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
Given a matrix , the low rank matrix completion
problem asks us to find a rank- approximation of as for and by only observing a
few entries specified by a set of entries . In
particular, we examine an approach that is widely used in practice -- the
alternating minimization framework. Jain, Netrapalli and Sanghavi~\cite{jns13}
showed that if has incoherent rows and columns, then alternating
minimization provably recovers the matrix by observing a nearly linear in
number of entries. While the sample complexity has been subsequently
improved~\cite{glz17}, alternating minimization steps are required to be
computed exactly. This hinders the development of more efficient algorithms and
fails to depict the practical implementation of alternating minimization, where
the updates are usually performed approximately in favor of efficiency.
In this paper, we take a major step towards a more efficient and error-robust
alternating minimization framework. To this end, we develop an analytical
framework for alternating minimization that can tolerate moderate amount of
errors caused by approximate updates. Moreover, our algorithm runs in time
, which is nearly linear in the time to verify the
solution while preserving the sample complexity. This improves upon all prior
known alternating minimization approaches which require time.Comment: Improve the runtime from to $O|\Omega| k)
Convex Minimization with Integer Minima in Time
Given a convex function on with an integer minimizer, we
show how to find an exact minimizer of using calls to a
separation oracle and time. The previous best polynomial time
algorithm for this problem given in [Jiang, SODA 2021, JACM 2022] achieves
oracle complexity. However, the overall runtime of
Jiang's algorithm is at least , due to expensive
sub-routines such as the Lenstra-Lenstra-Lov\'asz (LLL) algorithm [Lenstra,
Lenstra, Lov\'asz, Math. Ann. 1982] and random walk based cutting plane method
[Bertsimas, Vempala, JACM 2004]. Our significant speedup is obtained by a
nontrivial combination of a faster version of the LLL algorithm due to
[Neumaier, Stehl\'e, ISSAC 2016] that gives similar guarantees, the volumetric
center cutting plane method (CPM) by [Vaidya, FOCS 1989] and its fast
implementation given in [Jiang, Lee, Song, Wong, STOC 2020].
For the special case of submodular function minimization (SFM), our result
implies a strongly polynomial time algorithm for this problem using calls to an evaluation oracle and additional arithmetic
operations. Both the oracle complexity and the number of arithmetic operations
of our more general algorithm are better than the previous best-known runtime
algorithms for this specific problem given in [Lee, Sidford, Wong, FOCS 2015]
and [Dadush, V\'egh, Zambelli, SODA 2018, MOR 2021].Comment: SODA 202