5 research outputs found
What Circuit Classes Can Be Learned with Non-Trivial Savings?
Despite decades of intensive research, efficient - or even sub-exponential time - distribution-free PAC learning algorithms are not known for many important Boolean function classes. In this work we suggest a new perspective on these learning problems, inspired by a surge of recent research in complexity theory, in which the goal is to determine whether and how much of a savings over a naive 2^n runtime can be achieved.
We establish a range of exploratory results towards this end. In more detail,
(1) We first observe that a simple approach building on known uniform-distribution learning results gives non-trivial distribution-free learning algorithms for several well-studied classes including AC0, arbitrary functions of a few linear threshold functions (LTFs), and AC0 augmented with mod_p gates.
(2) Next we present an approach, based on the method of random restrictions from circuit complexity, which can be used to obtain several distribution-free learning algorithms that do not appear to be achievable by approach (1) above. The results achieved in this way include learning algorithms with non-trivial savings for LTF-of-AC0 circuits and improved savings for learning parity-of-AC0 circuits.
(3) Finally, our third contribution is a generic technique for converting lower bounds proved using Neciporuk\u27s method to learning algorithms with non-trivial savings. This technique, which is the most involved of our three approaches, yields distribution-free learning algorithms for a range of classes where previously even non-trivial uniform-distribution learning algorithms were not known; these classes include full-basis formulas, branching programs, span programs, etc. up to some fixed polynomial size
Algorithms and lower bounds for de Morgan formulas of low-communication leaf gates
The class consists of Boolean functions
computable by size- de Morgan formulas whose leaves are any Boolean
functions from a class . We give lower bounds and (SAT, Learning,
and PRG) algorithms for , for classes
of functions with low communication complexity. Let
be the maximum -party NOF randomized communication
complexity of . We show:
(1) The Generalized Inner Product function cannot be computed in
on more than fraction of inputs
for As a corollary, we get an average-case lower bound for
against .
(2) There is a PRG of seed length that -fools . For
, we get the better seed length . This gives the first
non-trivial PRG (with seed length ) for intersections of half-spaces
in the regime where .
(3) There is a randomized -time SAT algorithm for , where In particular, this implies a nontrivial
#SAT algorithm for .
(4) The Minimum Circuit Size Problem is not in .
On the algorithmic side, we show that can be
PAC-learned in time
Agnostic Membership Query Learning with Nontrivial Savings: New Results, Techniques
(Abridged) Designing computationally efficient algorithms in the agnostic
learning model (Haussler, 1992; Kearns et al., 1994) is notoriously difficult.
In this work, we consider agnostic learning with membership queries for
touchstone classes at the frontier of agnostic learning, with a focus on how
much computation can be saved over the trivial runtime of 2^n$. This approach
is inspired by and continues the study of ``learning with nontrivial savings''
(Servedio and Tan, 2017). To this end, we establish multiple agnostic learning
algorithms, highlighted by:
1. An agnostic learning algorithm for circuits consisting of a sublinear
number of gates, which can each be any function computable by a sublogarithmic
degree k polynomial threshold function (the depth of the circuit is bounded
only by size). This algorithm runs in time 2^{n -s(n)} for s(n) \approx
n/(k+1), and learns over the uniform distribution over unlabelled examples on
\{0,1\}^n.
2. An agnostic learning algorithm for circuits consisting of a sublinear
number of gates, where each can be any function computable by a \sym^+ circuit
of subexponential size and sublogarithmic degree k. This algorithm runs in time
2^{n-s(n)} for s(n) \approx n/(k+1), and learns over distributions of
unlabelled examples that are products of k+1 arbitrary and unknown
distributions, each over \{0,1\}^{n/(k+1)} (assume without loss of generality
that k+1 divides n)
Quantum learning algorithms imply circuit lower bounds
We establish the first general connection between the design of quantum
algorithms and circuit lower bounds. Specifically, let be a
class of polynomial-size concepts, and suppose that can be
PAC-learned with membership queries under the uniform distribution with error
by a time quantum algorithm. We prove that if , then , where
is an exponential-time analogue of
. This result is optimal in both and , since it is
not hard to learn any class of functions in (classical) time (with no error), or in quantum time with error at
most via Fourier sampling. In other words, even a
marginal improvement on these generic learning algorithms would lead to major
consequences in complexity theory.
Our proof builds on several works in learning theory, pseudorandomness, and
computational complexity, and crucially, on a connection between non-trivial
classical learning algorithms and circuit lower bounds established by Oliveira
and Santhanam (CCC 2017). Extending their approach to quantum learning
algorithms turns out to create significant challenges. To achieve that, we show
among other results how pseudorandom generators imply learning-to-lower-bound
connections in a generic fashion, construct the first conditional pseudorandom
generator secure against uniform quantum computations, and extend the local
list-decoding algorithm of Impagliazzo, Jaiswal, Kabanets and Wigderson (SICOMP
2010) to quantum circuits via a delicate analysis. We believe that these
contributions are of independent interest and might find other applications