12 research outputs found
Recovery Guarantees for Quadratic Tensors with Limited Observations
We consider the tensor completion problem of predicting the missing entries
of a tensor. The commonly used CP model has a triple product form, but an
alternate family of quadratic models which are the sum of pairwise products
instead of a triple product have emerged from applications such as
recommendation systems. Non-convex methods are the method of choice for
learning quadratic models, and this work examines their sample complexity and
error guarantee. Our main result is that with the number of samples being only
linear in the dimension, all local minima of the mean squared error objective
are global minima and recover the original tensor accurately. The techniques
lead to simple proofs showing that convex relaxation can recover quadratic
tensors provided with linear number of samples. We substantiate our theoretical
results with experiments on synthetic and real-world data, showing that
quadratic models have better performance than CP models in scenarios where
there are limited amount of observations available
Static Data Structure Lower Bounds Imply Rigidity
We show that static data structure lower bounds in the group (linear) model
imply semi-explicit lower bounds on matrix rigidity. In particular, we prove
that an explicit lower bound of on the cell-probe
complexity of linear data structures in the group model, even against
arbitrarily small linear space , would already imply a
semi-explicit () construction of rigid matrices with
significantly better parameters than the current state of art (Alon, Panigrahy
and Yekhanin, 2009). Our results further assert that polynomial () data structure lower bounds against near-optimal space, would
imply super-linear circuit lower bounds for log-depth linear circuits (a
four-decade open question). In the succinct space regime , we show
that any improvement on current cell-probe lower bounds in the linear model
would also imply new rigidity bounds. Our results rely on a new connection
between the "inner" and "outer" dimensions of a matrix (Paturi and Pudlak,
2006), and on a new reduction from worst-case to average-case rigidity, which
is of independent interest
Block Rigidity: Strong Multiplayer Parallel Repetition Implies Super-Linear Lower Bounds for Turing Machines
We prove that a sufficiently strong parallel repetition theorem for a special
case of multiplayer (multiprover) games implies super-linear lower bounds for
multi-tape Turing machines with advice. To the best of our knowledge, this is
the first connection between parallel repetition and lower bounds for time
complexity and the first major potential implication of a parallel repetition
theorem with more than two players.
Along the way to proving this result, we define and initiate a study of block
rigidity, a weakening of Valiant's notion of rigidity. While rigidity was
originally defined for matrices, or, equivalently, for (multi-output) linear
functions, we extend and study both rigidity and block rigidity for general
(multi-output) functions. Using techniques of Paul, Pippenger, Szemer\'edi and
Trotter, we show that a block-rigid function cannot be computed by multi-tape
Turing machines that run in linear (or slightly super-linear) time, even in the
non-uniform setting, where the machine gets an arbitrary advice tape.
We then describe a class of multiplayer games, such that, a sufficiently
strong parallel repetition theorem for that class of games implies an explicit
block-rigid function. The games in that class have the following property that
may be of independent interest: for every random string for the verifier
(which, in particular, determines the vector of queries to the players), there
is a unique correct answer for each of the players, and the verifier accepts if
and only if all answers are correct. We refer to such games as independent
games. The theorem that we need is that parallel repetition reduces the value
of games in this class from to , where is the number of
repetitions.
As another application of block rigidity, we show conditional size-depth
tradeoffs for boolean circuits, where the gates compute arbitrary functions
over large sets.Comment: 17 pages, ITCS 202
Range Avoidance for Constant-Depth Circuits: Hardness and Algorithms
Range Avoidance (AVOID) is a total search problem where, given a Boolean
circuit , , the task is to find a
outside the range of . For an integer ,
-AVOID is a special case of AVOID where each output bit of
depends on at most input bits. While there is a very natural randomized
algorithm for AVOID, a deterministic algorithm for the problem would have many
interesting consequences. Ren, Santhanam, and Wang (FOCS 2022) and Guruswami,
Lyu, and Wang (RANDOM 2022) proved that explicit constructions of functions of
high formula complexity, rigid matrices, and optimal linear codes, reduce to
-AVOID, thus establishing conditional hardness of the
-AVOID problem. On the other hand, -AVOID
admits polynomial-time algorithms, leaving the question about the complexity of
-AVOID open.
We give the first reduction of an explicit construction question to
-AVOID. Specifically, we prove that a polynomial-time
algorithm (with an oracle) for -AVOID for the
case of would imply an explicit construction of a rigid matrix,
and, thus, a super-linear lower bound on the size of log-depth circuits.
We also give deterministic polynomial-time algorithms for all
-AVOID problems for . Prior work
required an oracle, and required larger stretch, .Comment: 19 page
Range Avoidance for Constant Depth Circuits: Hardness and Algorithms
Range Avoidance (Avoid) is a total search problem where, given a Boolean circuit ?: {0,1}? ? {0,1}^m, m > n, the task is to find a y ? {0,1}^m outside the range of ?. For an integer k ? 2, NC?_k-Avoid is a special case of Avoid where each output bit of ? depends on at most k input bits. While there is a very natural randomized algorithm for Avoid, a deterministic algorithm for the problem would have many interesting consequences. Ren, Santhanam, and Wang (FOCS 2022) and Guruswami, Lyu, and Wang (RANDOM 2022) proved that explicit constructions of functions of high formula complexity, rigid matrices, and optimal linear codes, reduce to NC??-Avoid, thus establishing conditional hardness of the NC??-Avoid problem. On the other hand, NC??-Avoid admits polynomial-time algorithms, leaving the question about the complexity of NC??-Avoid open.
We give the first reduction of an explicit construction question to NC??-Avoid. Specifically, we prove that a polynomial-time algorithm (with an NP oracle) for NC??-Avoid for the case of m = n+n^{2/3} would imply an explicit construction of a rigid matrix, and, thus, a super-linear lower bound on the size of log-depth circuits.
We also give deterministic polynomial-time algorithms for all NC?_k-Avoid problems for m ? n^{k-1}/log(n). Prior work required an NP oracle, and required larger stretch, m ? n^{k-1}
Recommended from our members
Fourier and Circulant Matrices Are Not Rigid
The concept of matrix rigidity was first introduced by Valiant in [Friedman, 1993]. Roughly speaking, a matrix is rigid if its rank cannot be reduced significantly by changing a small number of entries. There has been extensive interest in rigid matrices as Valiant showed in [Friedman, 1993] that rigidity can be used to prove arithmetic circuit lower bounds.
In a surprising result, Alman and Williams showed that the (real valued) Hadamard matrix, which was conjectured to be rigid, is actually not very rigid. This line of work was extended by [Dvir and Edelman, 2017] to a family of matrices related to the Hadamard matrix, but over finite fields. In our work, we take another step in this direction and show that for any abelian group G and function f:G - > {C}, the matrix given by M_{xy} = f(x - y) for x,y in G is not rigid. In particular, we get that complex valued Fourier matrices, circulant matrices, and Toeplitz matrices are all not rigid and cannot be used to carry out Valiant\u27s approach to proving circuit lower bounds. This complements a recent result of Goldreich and Tal [Goldreich and Tal, 2016] who showed that Toeplitz matrices are nontrivially rigid (but not enough for Valiant\u27s method). Our work differs from previous non-rigidity results in that those works considered matrices whose underlying group of symmetries was of the form {F}_p^n with p fixed and n tending to infinity, while in the families of matrices we study, the underlying group of symmetries can be any abelian group and, in particular, the cyclic group {Z}_N, which has very different structure. Our results also suggest natural new candidates for rigidity in the form of matrices whose symmetry groups are highly non-abelian.
Our proof has four parts. The first extends the results of [Josh Alman and Ryan Williams, 2016; Dvir and Edelman, 2017] to generalized Hadamard matrices over the complex numbers via a new proof technique. The second part handles the N x N Fourier matrix when N has a particularly nice factorization that allows us to embed smaller copies of (generalized) Hadamard matrices inside of it. The third part uses results from number theory to bootstrap the non-rigidity for these special values of N and extend to all sufficiently large N. The fourth and final part involves using the non-rigidity of the Fourier matrix to show that the group algebra matrix, given by M_{xy} = f(x - y) for x,y in G, is not rigid for any function f and abelian group G
Matrix Rigidity Depends on the Target Field
The rigidity of a matrix A for target rank r is the minimum number of entries of A that need to be changed in order to obtain a matrix of rank at most r (Valiant, 1977).
We study the dependence of rigidity on the target field. We consider especially two natural regimes: when one is allowed to make changes only from the field of definition of the matrix ("strict rigidity"), and when the changes are allowed to be in an arbitrary extension field ("absolute rigidity").
We demonstrate, apparently for the first time, a separation between these two concepts. We establish a gap of a factor of 3/2-o(1) between strict and absolute rigidities.
The question seems especially timely because of recent results by Dvir and Liu (Theory of Computing, 2020) where important families of matrices, previously expected to be rigid, are shown not to be absolutely rigid, while their strict rigidity remains open. Our lower-bound method combines elementary arguments from algebraic geometry with "untouched minors" arguments.
Finally, we point out that more families of long-time rigidity candidates fall as a consequence of the results of Dvir and Liu. These include the incidence matrices of projective planes over finite fields, proposed by Valiant as candidates for rigidity over ??
Matrix Multiplication Verification Using Coding Theory
We study the Matrix Multiplication Verification Problem (MMV) where the goal
is, given three matrices , , and as input, to decide
whether . A classic randomized algorithm by Freivalds (MFCS, 1979)
solves MMV in time, and a longstanding challenge is to
(partially) derandomize it while still running in faster than matrix
multiplication time (i.e., in time).
To that end, we give two algorithms for MMV in the case where is
sparse. Specifically, when has at most non-zero
entries for a constant , we give (1) a deterministic
-time algorithm for constant , and (2) a randomized -time
algorithm using random bits. The former
algorithm is faster than the deterministic algorithm of K\"{u}nnemann (ESA,
2018) when , and the latter algorithm uses fewer random bits
than the algorithm of Kimbrel and Sinha (IPL, 1993), which runs in the same
time and uses random bits (in turn fewer than Freivalds's
algorithm).
We additionally study the complexity of MMV. We first show that all
algorithms in a natural class of deterministic linear algebraic algorithms for
MMV (including ours) require time. We also show a barrier
to proving a super-quadratic running time lower bound for matrix multiplication
(and hence MMV) under the Strong Exponential Time Hypothesis (SETH). Finally,
we study relationships between natural variants and special cases of MMV (with
respect to deterministic -time reductions)