Categorical invariance and structural complexity in human concept learning

An alternative account of human concept learning based on an invariance measure of the categorical\ud stimulus is proposed. The categorical invariance model (CIM) characterizes the degree of structural\ud complexity of a Boolean category as a function of its inherent degree of invariance and its cardinality or\ud size. To do this we introduce a mathematical framework based on the notion of a Boolean differential\ud operator on Boolean categories that generates the degrees of invariance (i.e., logical manifold) of the\ud category in respect to its dimensions. Using this framework, we propose that the structural complexity\ud of a Boolean category is indirectly proportional to its degree of categorical invariance and directly\ud proportional to its cardinality or size. Consequently, complexity and invariance notions are formally\ud unified to account for concept learning difficulty. Beyond developing the above unifying mathematical\ud framework, the CIM is significant in that: (1) it precisely predicts the key learning difficulty ordering of\ud the SHJ [Shepard, R. N., Hovland, C. L.,&Jenkins, H. M. (1961). Learning and memorization of classifications.\ud Psychological Monographs: General and Applied, 75(13), 1-42] Boolean category types consisting of three\ud binary dimensions and four positive examples; (2) it is, in general, a good quantitative predictor of the\ud degree of learning difficulty of a large class of categories (in particular, the 41 category types studied\ud by Feldman [Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature,\ud 407, 630-633]); (3) it is, in general, a good quantitative predictor of parity effects for this large class of\ud categories; (4) it does all of the above without free parameters; and (5) it is cognitively plausible (e.g.,\ud cognitively tractable)

Complexity of Equivalence and Learning for Multiplicity Tree Automata

We consider the complexity of equivalence and learning for multiplicity tree automata, i.e., weighted tree automata over a field. We first show that the equivalence problem is logspace equivalent to polynomial identity testing, the complexity of which is a longstanding open problem. Secondly, we derive lower bounds on the number of queries needed to learn multiplicity tree automata in Angluin's exact learning model, over both arbitrary and fixed fields. Habrard and Oncina (2006) give an exact learning algorithm for multiplicity tree automata, in which the number of queries is proportional to the size of the target automaton and the size of a largest counterexample, represented as a tree, that is returned by the Teacher. However, the smallest tree-counterexample may be exponential in the size of the target automaton. Thus the above algorithm does not run in time polynomial in the size of the target automaton, and has query complexity exponential in the lower bound. Assuming a Teacher that returns minimal DAG representations of counterexamples, we give a new exact learning algorithm whose query complexity is quadratic in the target automaton size, almost matching the lower bound, and improving the best previously-known algorithm by an exponential factor

Towards a Law of Invariance in Human Concept Learning

Invariance principles underlie many key theories in modern science. They provide the explanatory and predictive framework necessary for the rigorous study of natural phenomena ranging from the structure of crystals, to magnetism, to relativistic mechanics. Vigo (2008, 2009)introduced a new general notion and principle of invariance from which two parameter-free (ratio and exponential) models were derived to account for human conceptual behavior. Here we introduce a new parameterized \ud exponential “law” based on the same invariance principle. The law accurately predicts the subjective degree of difficulty that humans experience when learning different types of concepts. In addition, it precisely fits the data from a large-scale experiment which examined a total of 84 category structures across 10 category families (R-Squared =.97, p < .0001; r= .98, p < .0001). Moreover, it overcomes seven key challenges that had, hitherto, been grave obstacles for theories of concept learning

DNF Sparsification and a Faster Deterministic Counting Algorithm

Given a DNF formula on n variables, the two natural size measures are the number of terms or size s(f), and the maximum width of a term w(f). It is folklore that short DNF formulas can be made narrow. We prove a converse, showing that narrow formulas can be sparsified. More precisely, any width w DNF irrespective of its size can be $\epsilon$-approximated by a width $w$ DNF with at most $(w\log(1/\epsilon))^{O(w)}$ terms. We combine our sparsification result with the work of Luby and Velikovic to give a faster deterministic algorithm for approximately counting the number of satisfying solutions to a DNF. Given a formula on n variables with poly(n) terms, we give a deterministic $n^{\tilde{O}(\log \log(n))}$ time algorithm that computes an additive $\epsilon$ approximation to the fraction of satisfying assignments of f for \epsilon = 1/\poly(\log n). The previous best result due to Luby and Velickovic from nearly two decades ago had a run-time of $n^{\exp(O(\sqrt{\log \log n}))}$.Comment: To appear in the IEEE Conference on Computational Complexity, 201

Learning Unions of ω(1)\omega(1)-Dimensional Rectangles

We consider the problem of learning unions of rectangles over the domain $[b]^n$, in the uniform distribution membership query learning setting, where both b and n are "large". We obtain poly$(n, \log b)$-time algorithms for the following classes: - poly$(n \log b)$-way Majority of $O(\frac{\log(n \log b)} {\log \log(n \log b)})$-dimensional rectangles. - Union of poly$(\log(n \log b))$ many $O(\frac{\log^2 (n \log b)} {(\log \log(n \log b) \log \log \log (n \log b))^2})$-dimensional rectangles. - poly$(n \log b)$-way Majority of poly$(n \log b)$-Or of disjoint $O(\frac{\log(n \log b)} {\log \log(n \log b)})$-dimensional rectangles. Our main algorithmic tool is an extension of Jackson's boosting- and Fourier-based Harmonic Sieve algorithm [Jackson 1997] to the domain $[b]^n$, building on work of [Akavia, Goldwasser, Safra 2003]. Other ingredients used to obtain the results stated above are techniques from exact learning [Beimel, Kushilevitz 1998] and ideas from recent work on learning augmented $AC^{0}$ circuits [Jackson, Klivans, Servedio 2002] and on representing Boolean functions as thresholds of parities [Klivans, Servedio 2001].Comment: 25 pages. Some corrections. Recipient of E. M. Gold award ALT 2006. To appear in Journal of Theoretical Computer Scienc

Towards Optimal Tree Construction of Monotone Functions

This thesis focuses on finding counterexamples for the conjecture suggested by Dr. Jackson that if two Boolean variables i and j in a monotone Boolean function have the relation such that if i is relevant in only one sub-tree with j as root while j is relevant in both sub-trees with i as root, then the optimal tree size (defined as the number of leaves in the tree) with j as root is as least as small as the optimal tree size with i as root. All distinct monotone Boolean functions of up to 6 variables and some interesting functions of 7 and 8 variables are tested; no counterexample has been found

Pre-Reduction Graph Products: Hardnesses of Properly Learning DFAs and Approximating EDP on DAGs

The study of graph products is a major research topic and typically concerns the term $f(G*H)$, e.g., to show that $f(G*H)=f(G)f(H)$. In this paper, we study graph products in a non-standard form $f(R[G*H]$ where $R$ is a "reduction", a transformation of any graph into an instance of an intended optimization problem. We resolve some open problems as applications. (1) A tight $n^{1-\epsilon}$-approximation hardness for the minimum consistent deterministic finite automaton (DFA) problem, where $n$ is the sample size. Due to Board and Pitt [Theoretical Computer Science 1992], this implies the hardness of properly learning DFAs assuming $NP\neq RP$ (the weakest possible assumption). (2) A tight $n^{1/2-\epsilon}$ hardness for the edge-disjoint paths (EDP) problem on directed acyclic graphs (DAGs), where $n$ denotes the number of vertices. (3) A tight hardness of packing vertex-disjoint $k$-cycles for large $k$. (4) An alternative (and perhaps simpler) proof for the hardness of properly learning DNF, CNF and intersection of halfspaces [Alekhnovich et al., FOCS 2004 and J. Comput.Syst.Sci. 2008]