SelfieBoost: A Boosting Algorithm for Deep Learning

We describe and analyze a new boosting algorithm for deep learning called SelfieBoost. Unlike other boosting algorithms, like AdaBoost, which construct ensembles of classifiers, SelfieBoost boosts the accuracy of a single network. We prove a $\log(1/\epsilon)$ convergence rate for SelfieBoost under some "SGD success" assumption which seems to hold in practice

The Structure of Promises in Quantum Speedups

It has long been known that in the usual black-box model, one cannot get super-polynomial quantum speedups without some promise on the inputs. In this paper, we examine certain types of symmetric promises, and show that they also cannot give rise to super-polynomial quantum speedups. We conclude that exponential quantum speedups only occur given "structured" promises on the input. Specifically, we show that there is a polynomial relationship of degree $12$ between $D(f)$ and $Q(f)$ for any function $f$ defined on permutations (elements of $\{0,1,\dots, M-1\}^n$ in which each alphabet element occurs exactly once). We generalize this result to all functions $f$ defined on orbits of the symmetric group action $S_n$ (which acts on an element of $\{0,1,\dots, M-1\}^n$ by permuting its entries). We also show that when $M$ is constant, any function $f$ defined on a "symmetric set" - one invariant under $S_n$ - satisfies $R(f)=O(Q(f)^{12(M-1)})$.Comment: 15 page

Commutator maps, measure preservation, and T-systems

Let G be a finite simple group. We show that the commutator map $a : G \times G \to G$ is almost equidistributed as the order of G goes to infinity. This somewhat surprising result has many applications. It shows that for a subset X of G we have $a^{-1}(X)/|G|^2 = |X|/|G| + o(1)$, namely $a$ is almost measure preserving. From this we deduce that almost all elements $g \in G$ can be expressed as commutators $g = [x,y]$ where x,y generate G. This enables us to solve some open problems regarding T-systems and the Product Replacement Algorithm (PRA) graph. We show that the number of T-systems in G with two generators tends to infinity as the order of G goes to infinity. This settles a conjecture of Guralnick and Pak. A similar result follows for the number of connected components of the PRA graph of G with two generators. Some of our results apply for more general finite groups, and more general word maps. Our methods are based on representation theory, combining classical character theory with recent results on character degrees and values in finite simple groups. In particular the so called Witten zeta function plays a key role in the proofs.Comment: 28 pages. This article was submitted to the Transactions of the American Mathematical Society on 21 February 2007 and accepted on 24 June 200