Spectra of large time-lagged correlation matrices from Random Matrix Theory

We analyze the spectral properties of large, time-lagged correlation matrices using the tools of random matrix theory. We compare predictions of the one-dimensional spectra, based on approaches already proposed in the literature. Employing the methods of free random variables and diagrammatic techniques, we solve a general random matrix problem, namely the spectrum of a matrix $\frac{1}{T}XAX^{\dagger}$, where $X$ is an $N\times T$ Gaussian random matrix and $A$ is \textit{any} $T\times T$, not necessarily symmetric (Hermitian) matrix. As a particular application, we present the spectral features of the large lagged correlation matrices as a function of the depth of the time-lag. We also analyze the properties of left and right eigenvector correlations for the time-lagged matrices. We positively verify our results by the numerical simulations.Comment: 44 pages, 11 figures; v2 typos corrected, final versio

Probing non-orthogonality of eigenvectors in non-Hermitian matrix models: diagrammatic approach

Using large $N$ arguments, we propose a scheme for calculating the two-point eigenvector correlation function for non-normal random matrices in the large $N$ limit. The setting generalizes the quaternionic extension of free probability to two-point functions. In the particular case of biunitarily invariant random matrices, we obtain a simple, general expression for the two-point eigenvector correlation function, which can be viewed as a further generalization of the single ring theorem. This construction has some striking similarities to the freeness of the second kind known for the Hermitian ensembles in large $N$. On the basis of several solved examples, we conjecture two kinds of microscopic universality of the eigenvectors - one in the bulk, and one at the rim. The form of the conjectured bulk universality agrees with the scaling limit found by Chalker and Mehlig [JT Chalker, B Mehlig, PRL, \textbf{81}, 3367 (1998)] in the case of the complex Ginibre ensemble.Comment: 20 pages + 4 pages of references, 12 figs; v2: typos corrected, refs added; v3: more explanator

Digital Vector Modulator with Diagnostic Circuit for Particle Accelerator

Modern particle accelerators[8] have very strict requirements for controlling signal. To generate signal controlling field in cavities there is a possibility to use digital vector modulator and analog upconverter[6]. Hardware circuit consists of FPGA device, digital to analog converter and low pass filter. For experiments there is also a need to design and develop measurement circuit. Same circuit could be used in final application as diagnostics module. This enables an option for constant long term measurement and fault detection. This paper presents construction of such module

Unveiling the significance of eigenvectors in diffusing non-hermitian matrices by identifying the underlying Burgers dynamics

Following our recent letter, we study in detail an entry-wise diffusion of non-hermitian complex matrices. We obtain an exact partial differential equation (valid for any matrix size $N$ and arbitrary initial conditions) for evolution of the averaged extended characteristic polynomial. The logarithm of this polynomial has an interpretation of a potential which generates a Burgers dynamics in quaternionic space. The dynamics of the ensemble in the large $N$ is completely determined by the coevolution of the spectral density and a certain eigenvector correlation function. This coevolution is best visible in an electrostatic potential of a quaternionic argument built of two complex variables, the first of which governs standard spectral properties while the second unravels the hidden dynamics of eigenvector correlation function. We obtain general large $N$ formulas for both spectral density and 1-point eigenvector correlation function valid for any initial conditions. We exemplify our studies by solving three examples, and we verify the analytic form of our solutions with numerical simulations.Comment: 24 pages, 11 figure

Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function

We demonstrate that in residual neural networks (ResNets) dynamical isometry is achievable irrespectively of the activation function used. We do that by deriving, with the help of Free Probability and Random Matrix Theories, a universal formula for the spectral density of the input-output Jacobian at initialization, in the large network width and depth limit. The resulting singular value spectrum depends on a single parameter, which we calculate for a variety of popular activation functions, by analyzing the signal propagation in the artificial neural network. We corroborate our results with numerical simulations of both random matrices and ResNets applied to the CIFAR-10 classification problem. Moreover, we study the consequence of this universal behavior for the initial and late phases of the learning processes. We conclude by drawing attention to the simple fact, that initialization acts as a confounding factor between the choice of activation function and the rate of learning. We propose that in ResNets this can be resolved based on our results, by ensuring the same level of dynamical isometry at initialization

Dysonian dynamics of the Ginibre ensemble

We study the time evolution of Ginibre matrices whose elements undergo Brownian motion. The non-Hermitian character of the Ginibre ensemble binds the dynamics of eigenvalues to the evolution of eigenvectors in a non-trivial way, leading to a system of coupled nonlinear equations resembling those for turbulent systems. We formulate a mathematical framework allowing simultaneous description of the flow of eigenvalues and eigenvectors, and we unravel a hidden dynamics as a function of new complex variable, which in the standard description is treated as a regulator only. We solve the evolution equations for large matrices and demonstrate that the non-analytic behavior of the Green's functions is associated with a shock wave stemming from a Burgers-like equation describing correlations of eigenvectors. We conjecture that the hidden dynamics, that we observe for the Ginibre ensemble, is a general feature of non-Hermitian random matrix models and is relevant to related physical applications.Comment: 5 pages, 2 figure

Liquid crystal hyperbolic metamaterial for wide-angle negative-positive refraction and reflection

We show that nanosphere dispersed liquid crystal (NDLC) metamaterial can be characterized in near IR spectral region as an indefinite medium whose real parts of effective ordinary and extraordinary permittivities are opposite in signs. Based on this fact we design a novel electrooptic effect: external electric field driven switch between normal refraction, negative refraction and reflection of TM incident electromagnetic wave from the boundary vacuum/NDLC. A detailed analysis of its functionality is given based on effective medium theory combined with a study of negative refraction in anisotropic metamaterials, and Finite Elements simulations

From synaptic interactions to collective dynamics in random neuronal networks models: critical role of eigenvectors and transient behavior

The study of neuronal interactions is currently at the center of several neuroscience big collaborative projects (including the Human Connectome, the Blue Brain, the Brainome, etc.) which attempt to obtain a detailed map of the entire brain matrix. Under certain constraints, mathematical theory can advance predictions of the expected neural dynamics based solely on the statistical properties of such synaptic interaction matrix. This work explores the application of free random variables (FRV) to the study of large synaptic interaction matrices. Besides recovering in a straightforward way known results on eigenspectra of neural networks, we extend them to heavy-tailed distributions of interactions. More importantly, we derive analytically the behavior of eigenvector overlaps, which determine stability of the spectra. We observe that upon imposing the neuronal excitation/inhibition balance, although the eigenvalues remain unchanged, their stability dramatically decreases due to strong non-orthogonality of associated eigenvectors. It leads us to the conclusion that the understanding of the temporal evolution of asymmetric neural networks requires considering the entangled dynamics of both eigenvectors and eigenvalues, which might bear consequences for learning and memory processes in these models. Considering the success of FRV analysis in a wide variety of branches disciplines, we hope that the results presented here foster additional application of these ideas in the area of brain sciences.Comment: 24 pages + 4 pages of refs, 8 figure

Non-orthogonal eigenvectors, fluctuation-dissipation relations and entropy production

Celebrated fluctuation-dissipation theorem (FDT) linking the response function to time dependent correlations of observables measured in the reference unperturbed state is one of the central results in equilibrium statistical mechanics. In this letter we discuss an extension of the standard FDT to the case when multidimensional matrix representing transition probabilities is strictly non-normal. This feature dramatically modifies the dynamics, by incorporating the effect of eigenvector non-orthogonality via the associated overlap matrix of Chalker-Mehlig type. In particular, the rate of entropy production per unit time is strongly enhanced by that matrix. We suggest, that this mechanism has an impact on the studies of collective phenomena in neural matrix models, leading, via transient behavior, to such phenomena as synchronisation and emergence of the memory. We also expect, that the described mechanism generating the entropy production is generic for wide class of phenomena, where dynamics is driven by non-normal operators. For the case of driving by a large Ginibre matrix the entropy production rate is evaluated analytically, as well as for the Rajan-Abbott model for neural networks.Comment: 3 figures, 8 pages. Important references added, calculation of entropy production rates for Rajan-Abbott model of neural networks and for Ginibre ensemble completed, title change