Efficiency versus Convergence of Boolean Kernels for On-Line Learning Algorithms

The paper studies machine learning problems where each example is described using a set of Boolean features and where hypotheses are represented by linear threshold elements. One method of increasing the expressiveness of learned hypotheses in this context is to expand the feature set to include conjunctions of basic features. This can be done explicitly or where possible by using a kernel function. Focusing on the well known Perceptron and Winnow algorithms, the paper demonstrates a tradeoff between the computational efficiency with which the algorithm can be run over the expanded feature space and the generalization ability of the corresponding learning algorithm. We first describe several kernel functions which capture either limited forms of conjunctions or all conjunctions. We show that these kernels can be used to efficiently run the Perceptron algorithm over a feature space of exponentially many conjunctions; however we also show that using such kernels, the Perceptron algorithm can provably make an exponential number of mistakes even when learning simple functions. We then consider the question of whether kernel functions can analogously be used to run the multiplicative-update Winnow algorithm over an expanded feature space of exponentially many conjunctions. Known upper bounds imply that the Winnow algorithm can learn Disjunctive Normal Form (DNF) formulae with a polynomial mistake bound in this setting. However, we prove that it is computationally hard to simulate Winnows behavior for learning DNF over such a feature set. This implies that the kernel functions which correspond to running Winnow for this problem are not efficiently computable, and that there is no general construction that can run Winnow with kernels

Faster exponential-time algorithms in graphs of bounded average degree

We first show that the Traveling Salesman Problem in an n-vertex graph with average degree bounded by d can be solved in O*(2^{(1-\eps_d)n}) time and exponential space for a constant \eps_d depending only on d, where the O*-notation suppresses factors polynomial in the input size. Thus, we generalize the recent results of Bjorklund et al. [TALG 2012] on graphs of bounded degree. Then, we move to the problem of counting perfect matchings in a graph. We first present a simple algorithm for counting perfect matchings in an n-vertex graph in O*(2^{n/2}) time and polynomial space; our algorithm matches the complexity bounds of the algorithm of Bjorklund [SODA 2012], but relies on inclusion-exclusion principle instead of algebraic transformations. Building upon this result, we show that the number of perfect matchings in an n-vertex graph with average degree bounded by d can be computed in O*(2^{(1-\eps_{2d})n/2}) time and exponential space, where \eps_{2d} is the constant obtained by us for the Traveling Salesman Problem in graphs of average degree at most 2d. Moreover we obtain a simple algorithm that counts the number of perfect matchings in an n-vertex bipartite graph of average degree at most d in O*(2^{(1-1/(3.55d))n/2}) time, improving and simplifying the recent result of Izumi and Wadayama [FOCS 2012].Comment: 10 page

A composition theorem for the Fourier Entropy-Influence conjecture

The Fourier Entropy-Influence (FEI) conjecture of Friedgut and Kalai [FK96] seeks to relate two fundamental measures of Boolean function complexity: it states that $H[f] \leq C Inf[f]$ holds for every Boolean function $f$, where $H[f]$ denotes the spectral entropy of $f$, $Inf[f]$ is its total influence, and $C > 0$ is a universal constant. Despite significant interest in the conjecture it has only been shown to hold for a few classes of Boolean functions. Our main result is a composition theorem for the FEI conjecture. We show that if $g_1,...,g_k$ are functions over disjoint sets of variables satisfying the conjecture, and if the Fourier transform of $F$ taken with respect to the product distribution with biases $E[g_1],...,E[g_k]$ satisfies the conjecture, then their composition $F(g_1(x^1),...,g_k(x^k))$ satisfies the conjecture. As an application we show that the FEI conjecture holds for read-once formulas over arbitrary gates of bounded arity, extending a recent result [OWZ11] which proved it for read-once decision trees. Our techniques also yield an explicit function with the largest known ratio of $C \geq 6.278$ between $H[f]$ and $Inf[f]$, improving on the previous lower bound of 4.615

Evolution of a mating preference for a dual-utility trait used in intrasexual competition in genetically monogamous populations

The selection pressures by which mating preferences for ornamental traits can evolve in genetically monogamous mating systems remain understudied. Empirical evidence from several taxa supports the prevalence of dual-utility traits, defined as traits used both as armaments in intersexual selection and ornaments in intrasexual selection, as well as the importance of intrasexual resource competition for the evolution of female ornamentation. Here, we study whether mating preferences for traits used in intrasexual resource competition can evolve under genetic monogamy. We find that a mating preference for a competitive trait can evolve and affect the evolution of the trait. The preference is more likely to persist when the fecundity benefit for mates of successful competitors is large and the aversion to unornamented potential mates is strong. The preference can persist for long periods or potentially permanently even when it incurs slight costs. Our results suggest that, when females use ornaments as signals in intrasexual resource competition, males can evolve mating preferences for those ornaments, illuminating both the evolution of female ornamentation and the evolution of male preferences for female ornaments in monogamous species

Quantum Algorithms for Learning and Testing Juntas

In this article we develop quantum algorithms for learning and testing juntas, i.e. Boolean functions which depend only on an unknown set of k out of n input variables. Our aim is to develop efficient algorithms: - whose sample complexity has no dependence on n, the dimension of the domain the Boolean functions are defined over; - with no access to any classical or quantum membership ("black-box") queries. Instead, our algorithms use only classical examples generated uniformly at random and fixed quantum superpositions of such classical examples; - which require only a few quantum examples but possibly many classical random examples (which are considered quite "cheap" relative to quantum examples). Our quantum algorithms are based on a subroutine FS which enables sampling according to the Fourier spectrum of f; the FS subroutine was used in earlier work of Bshouty and Jackson on quantum learning. Our results are as follows: - We give an algorithm for testing k-juntas to accuracy $\epsilon$ that uses $O(k/\epsilon)$ quantum examples. This improves on the number of examples used by the best known classical algorithm. - We establish the following lower bound: any FS-based k-junta testing algorithm requires $\Omega(\sqrt{k})$ queries. - We give an algorithm for learning $k$-juntas to accuracy $\epsilon$ that uses $O(\epsilon^{-1} k\log k)$ quantum examples and $O(2^k \log(1/\epsilon))$ random examples. We show that this learning algorithms is close to optimal by giving a related lower bound.Comment: 15 pages, 1 figure. Uses synttree package. To appear in Quantum Information Processin

Families with infants: a general approach to solve hard partition problems

We introduce a general approach for solving partition problems where the goal is to represent a given set as a union (either disjoint or not) of subsets satisfying certain properties. Many NP-hard problems can be naturally stated as such partition problems. We show that if one can find a large enough system of so-called families with infants for a given problem, then this problem can be solved faster than by a straightforward algorithm. We use this approach to improve known bounds for several NP-hard problems as well as to simplify the proofs of several known results. For the chromatic number problem we present an algorithm with $O^*((2-\varepsilon(d))^n)$ time and exponential space for graphs of average degree $d$. This improves the algorithm by Bj\"{o}rklund et al. [Theory Comput. Syst. 2010] that works for graphs of bounded maximum (as opposed to average) degree and closes an open problem stated by Cygan and Pilipczuk [ICALP 2013]. For the traveling salesman problem we give an algorithm working in $O^*((2-\varepsilon(d))^n)$ time and polynomial space for graphs of average degree $d$. The previously known results of this kind is a polyspace algorithm by Bj\"{o}rklund et al. [ICALP 2008] for graphs of bounded maximum degree and an exponential space algorithm for bounded average degree by Cygan and Pilipczuk [ICALP 2013]. For counting perfect matching in graphs of average degree~$d$ we present an algorithm with running time $O^*((2-\varepsilon(d))^{n/2})$ and polynomial space. Recent algorithms of this kind due to Cygan, Pilipczuk [ICALP 2013] and Izumi, Wadayama [FOCS 2012] (for bipartite graphs only) use exponential space.Comment: 18 pages, a revised version of this paper is available at http://arxiv.org/abs/1410.220

Nearly optimal solutions for the Chow Parameters Problem and low-weight approximation of halfspaces

The \emph{Chow parameters} of a Boolean function $f: \{-1,1\}^n \to \{-1,1\}$ are its $n+1$ degree-0 and degree-1 Fourier coefficients. It has been known since 1961 (Chow, Tannenbaum) that the (exact values of the) Chow parameters of any linear threshold function $f$ uniquely specify $f$ within the space of all Boolean functions, but until recently (O'Donnell and Servedio) nothing was known about efficient algorithms for \emph{reconstructing} $f$ (exactly or approximately) from exact or approximate values of its Chow parameters. We refer to this reconstruction problem as the \emph{Chow Parameters Problem.} Our main result is a new algorithm for the Chow Parameters Problem which, given (sufficiently accurate approximations to) the Chow parameters of any linear threshold function $f$, runs in time \tilde{O}(n^2)\cdot (1/\eps)^{O(\log^2(1/\eps))} and with high probability outputs a representation of an LTF $f'$ that is \eps-close to $f$. The only previous algorithm (O'Donnell and Servedio) had running time \poly(n) \cdot 2^{2^{\tilde{O}(1/\eps^2)}}. As a byproduct of our approach, we show that for any linear threshold function $f$ over $\{-1,1\}^n$, there is a linear threshold function $f'$ which is \eps-close to $f$ and has all weights that are integers at most \sqrt{n} \cdot (1/\eps)^{O(\log^2(1/\eps))}. This significantly improves the best previous result of Diakonikolas and Servedio which gave a \poly(n) \cdot 2^{\tilde{O}(1/\eps^{2/3})} weight bound, and is close to the known lower bound of $\max\{\sqrt{n},$ (1/\eps)^{\Omega(\log \log (1/\eps))}\} (Goldberg, Servedio). Our techniques also yield improved algorithms for related problems in learning theory

Folksonomies and clustering in the collaborative system CiteULike

We analyze CiteULike, an online collaborative tagging system where users bookmark and annotate scientific papers. Such a system can be naturally represented as a tripartite graph whose nodes represent papers, users and tags connected by individual tag assignments. The semantics of tags is studied here, in order to uncover the hidden relationships between tags. We find that the clustering coefficient reflects the semantical patterns among tags, providing useful ideas for the designing of more efficient methods of data classification and spam detection.Comment: 9 pages, 5 figures, iop style; corrected typo