235,425 research outputs found
Eigenvalues of block structured asymmetric random matrices
We study the spectrum of an asymmetric random matrix with block structured
variances. The rows and columns of the random square matrix are divided into
partitions with arbitrary size (linear in ). The parameters of the model
are the variances of elements in each block, summarized in
. Using the Hermitization approach and by
studying the matrix-valued Stieltjes transform we show that these matrices have
a circularly symmetric spectrum, we give an explicit formula for their spectral
radius and a set of implicit equations for the full density function. We
discuss applications of this model to neural networks
A Random Matrix Approach to Echo-State Neural Networks
Abstract Recurrent neural networks, especially in their linear version, have provided many qualitative insights on their performance under different configurations. This article provides, through a novel random matrix framework, the quantitative counterpart of these performance results, specifically in the case of echo-state networks. Beyond mere insights, our approach conveys a deeper understanding on the core mechanism under play for both training and testing
Analysis and Approximate Inference of Large Random Kronecker Graphs
Random graph models are playing an increasingly important role in various
fields ranging from social networks, telecommunication systems, to physiologic
and biological networks. Within this landscape, the random Kronecker graph
model, emerges as a prominent framework for scrutinizing intricate real-world
networks. In this paper, we investigate large random Kronecker graphs, i.e.,
the number of graph vertices is large. Built upon recent advances in random
matrix theory (RMT) and high-dimensional statistics, we prove that the
adjacency of a large random Kronecker graph can be decomposed, in a spectral
norm sense, into two parts: a small-rank (of rank ) signal matrix
that is linear in the graph parameters and a zero-mean random noise matrix.
Based on this result, we propose a ``denoise-and-solve'' approach to infer the
key graph parameters, with significantly reduced computational complexity.
Experiments on both graph inference and classification are presented to
evaluate the our proposed method. In both tasks, the proposed approach yields
comparable or advantageous performance, than widely-used graph inference (e.g.,
KronFit) and graph neural net baselines, at a time cost that scales linearly as
the graph size .Comment: 27 pages, 5 figures, 2 table
Bayesian Deep Net GLM and GLMM
Deep feedforward neural networks (DFNNs) are a powerful tool for functional
approximation. We describe flexible versions of generalized linear and
generalized linear mixed models incorporating basis functions formed by a DFNN.
The consideration of neural networks with random effects is not widely used in
the literature, perhaps because of the computational challenges of
incorporating subject specific parameters into already complex models.
Efficient computational methods for high-dimensional Bayesian inference are
developed using Gaussian variational approximation, with a parsimonious but
flexible factor parametrization of the covariance matrix. We implement natural
gradient methods for the optimization, exploiting the factor structure of the
variational covariance matrix in computation of the natural gradient. Our
flexible DFNN models and Bayesian inference approach lead to a regression and
classification method that has a high prediction accuracy, and is able to
quantify the prediction uncertainty in a principled and convenient way. We also
describe how to perform variable selection in our deep learning method. The
proposed methods are illustrated in a wide range of simulated and real-data
examples, and the results compare favourably to a state of the art flexible
regression and classification method in the statistical literature, the
Bayesian additive regression trees (BART) method. User-friendly software
packages in Matlab, R and Python implementing the proposed methods are
available at https://github.com/VBayesLabComment: 35 pages, 7 figure, 10 table
- …