591 research outputs found
An Introduction to Combinatorics via Cayley\u27s Theorem
In this paper, we explore some of the methods that are often used to solve combinatorial problems by proving Cayley’s theorem on trees in multiple ways. The intended audience of this paper is undergraduate and graduate mathematics students with little to no experience in combinatorics. This paper could also be used as a supplementary text for an undergraduate combinatorics course
Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
Despite remarkable performance on a variety of tasks, many properties of deep
neural networks are not yet theoretically understood. One such mystery is the
depth degeneracy phenomenon: the deeper you make your network, the closer your
network is to a constant function on initialization. In this paper, we examine
the evolution of the angle between two inputs to a ReLU neural network as a
function of the number of layers. By using combinatorial expansions, we find
precise formulas for how fast this angle goes to zero as depth increases. These
formulas capture microscopic fluctuations that are not visible in the popular
framework of infinite width limits, and leads to qualitatively different
predictions. We validate our theoretical results with Monte Carlo experiments
and show that our results accurately approximate finite network behaviour. The
formulas are given in terms of the mixed moments of correlated Gaussians passed
through the ReLU function. We also find a surprising combinatorial connection
between these mixed moments and the Bessel numbers that allows us to explicitly
evaluate these moments.Comment: Minor updates and exposition improved. 37 pages, comments welcom
Random surface growth and Karlin-McGregor polynomials
We consider consistent dynamics for non-intersecting birth and death chains,
originating from dualities of stochastic coalescing flows and one dimensional
orthogonal polynomials. As corollaries, we obtain unified and simple
probabilistic proofs of certain key intertwining relations between multivariate
Markov chains on the levels of some branching graphs. Special cases include the
dynamics on the Gelfand-Tsetlin graph considered by Borodin and Olshanski and
the ones on the BC-type graph recently studied by Cuenca. Moreover, we
introduce a general inhomogeneous random growth process with a wall that
includes as special cases the ones considered by Borodin and Kuan and Cerenzia,
that are related to the representation theory of classical groups and also the
Jacobi growth process more recently studied by Cerenzia and Kuan. Its most
important feature is that, this process retains the determinantal structure of
the ones studied previously and for the fully packed initial condition we are
able to calculate its correlation kernel explicitly in terms of a contour
integral involving orthogonal polynomials. At a certain scaling limit, at a
finite distance from the wall, one obtains for a single level discrete
determinantal ensembles associated to continuous orthogonal polynomials, that
were recently introduced by Borodin and Olshanski, and that depend on the
inhomogeneities.Comment: Published version: longer introduction, improved organization, minor
improvements throughou
- …