162 research outputs found
Learning DNFs under product distributions via {\mu}-biased quantum Fourier sampling
We show that DNF formulae can be quantum PAC-learned in polynomial time under
product distributions using a quantum example oracle. The best classical
algorithm (without access to membership queries) runs in superpolynomial time.
Our result extends the work by Bshouty and Jackson (1998) that proved that DNF
formulae are efficiently learnable under the uniform distribution using a
quantum example oracle. Our proof is based on a new quantum algorithm that
efficiently samples the coefficients of a {\mu}-biased Fourier transform.Comment: 17 pages; v3 based on journal version; minor corrections and
clarification
Learning DNF Expressions from Fourier Spectrum
Since its introduction by Valiant in 1984, PAC learning of DNF expressions
remains one of the central problems in learning theory. We consider this
problem in the setting where the underlying distribution is uniform, or more
generally, a product distribution. Kalai, Samorodnitsky and Teng (2009) showed
that in this setting a DNF expression can be efficiently approximated from its
"heavy" low-degree Fourier coefficients alone. This is in contrast to previous
approaches where boosting was used and thus Fourier coefficients of the target
function modified by various distributions were needed. This property is
crucial for learning of DNF expressions over smoothed product distributions, a
learning model introduced by Kalai et al. (2009) and inspired by the seminal
smoothed analysis model of Spielman and Teng (2001).
We introduce a new approach to learning (or approximating) a polynomial
threshold functions which is based on creating a function with range [-1,1]
that approximately agrees with the unknown function on low-degree Fourier
coefficients. We then describe conditions under which this is sufficient for
learning polynomial threshold functions. Our approach yields a new, simple
algorithm for approximating any polynomial-size DNF expression from its "heavy"
low-degree Fourier coefficients alone. Our algorithm greatly simplifies the
proof of learnability of DNF expressions over smoothed product distributions.
We also describe an application of our algorithm to learning monotone DNF
expressions over product distributions. Building on the work of Servedio
(2001), we give an algorithm that runs in time \poly((s \cdot
\log{(s/\eps)})^{\log{(s/\eps)}}, n), where is the size of the target DNF
expression and \eps is the accuracy. This improves on \poly((s \cdot
\log{(ns/\eps)})^{\log{(s/\eps)} \cdot \log{(1/\eps)}}, n) bound of Servedio
(2001).Comment: Appears in Conference on Learning Theory (COLT) 201
Categorical invariance and structural complexity in human concept learning
An alternative account of human concept learning based on an invariance measure of the categorical\ud
stimulus is proposed. The categorical invariance model (CIM) characterizes the degree of structural\ud
complexity of a Boolean category as a function of its inherent degree of invariance and its cardinality or\ud
size. To do this we introduce a mathematical framework based on the notion of a Boolean differential\ud
operator on Boolean categories that generates the degrees of invariance (i.e., logical manifold) of the\ud
category in respect to its dimensions. Using this framework, we propose that the structural complexity\ud
of a Boolean category is indirectly proportional to its degree of categorical invariance and directly\ud
proportional to its cardinality or size. Consequently, complexity and invariance notions are formally\ud
unified to account for concept learning difficulty. Beyond developing the above unifying mathematical\ud
framework, the CIM is significant in that: (1) it precisely predicts the key learning difficulty ordering of\ud
the SHJ [Shepard, R. N., Hovland, C. L.,&Jenkins, H. M. (1961). Learning and memorization of classifications.\ud
Psychological Monographs: General and Applied, 75(13), 1-42] Boolean category types consisting of three\ud
binary dimensions and four positive examples; (2) it is, in general, a good quantitative predictor of the\ud
degree of learning difficulty of a large class of categories (in particular, the 41 category types studied\ud
by Feldman [Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature,\ud
407, 630-633]); (3) it is, in general, a good quantitative predictor of parity effects for this large class of\ud
categories; (4) it does all of the above without free parameters; and (5) it is cognitively plausible (e.g.,\ud
cognitively tractable)
Complexity of Equivalence and Learning for Multiplicity Tree Automata
We consider the complexity of equivalence and learning for multiplicity tree
automata, i.e., weighted tree automata over a field. We first show that the
equivalence problem is logspace equivalent to polynomial identity testing, the
complexity of which is a longstanding open problem. Secondly, we derive lower
bounds on the number of queries needed to learn multiplicity tree automata in
Angluin's exact learning model, over both arbitrary and fixed fields.
Habrard and Oncina (2006) give an exact learning algorithm for multiplicity
tree automata, in which the number of queries is proportional to the size of
the target automaton and the size of a largest counterexample, represented as a
tree, that is returned by the Teacher. However, the smallest
tree-counterexample may be exponential in the size of the target automaton.
Thus the above algorithm does not run in time polynomial in the size of the
target automaton, and has query complexity exponential in the lower bound.
Assuming a Teacher that returns minimal DAG representations of
counterexamples, we give a new exact learning algorithm whose query complexity
is quadratic in the target automaton size, almost matching the lower bound, and
improving the best previously-known algorithm by an exponential factor
Towards a Law of Invariance in Human Concept Learning
Invariance principles underlie many key theories in modern science. They provide the explanatory and predictive framework necessary for the rigorous study of natural phenomena ranging from the structure of crystals, to magnetism, to relativistic mechanics. Vigo (2008, 2009)introduced a new general notion and principle of invariance from which two parameter-free (ratio and exponential) models were derived to account for human conceptual behavior. Here we introduce a new parameterized \ud
exponential “law” based on the same invariance principle. The law accurately predicts the subjective degree of difficulty that humans experience when learning different types of concepts. In addition, it precisely fits the data from a large-scale experiment which examined a total of 84 category structures across 10 category families (R-Squared =.97, p < .0001; r= .98, p < .0001). Moreover, it overcomes seven key challenges that had, hitherto, been grave obstacles for theories of concept learning
DNF Sparsification and a Faster Deterministic Counting Algorithm
Given a DNF formula on n variables, the two natural size measures are the
number of terms or size s(f), and the maximum width of a term w(f). It is
folklore that short DNF formulas can be made narrow. We prove a converse,
showing that narrow formulas can be sparsified. More precisely, any width w DNF
irrespective of its size can be -approximated by a width DNF with
at most terms.
We combine our sparsification result with the work of Luby and Velikovic to
give a faster deterministic algorithm for approximately counting the number of
satisfying solutions to a DNF. Given a formula on n variables with poly(n)
terms, we give a deterministic time algorithm
that computes an additive approximation to the fraction of
satisfying assignments of f for \epsilon = 1/\poly(\log n). The previous best
result due to Luby and Velickovic from nearly two decades ago had a run-time of
.Comment: To appear in the IEEE Conference on Computational Complexity, 201
- …