Learning to live with Dale's principle: ANNs with separate excitatory and inhibitory units

The units in artificial neural networks (ANNs) can be thought of as abstractions of biological neurons, and ANNs are increasingly used in neuroscience research. However, there are many important differences between ANN units and real neurons. One of the most notable is the absence of Dale's principle, which ensures that biological neurons are either exclusively excitatory or inhibitory. Dale's principle is typically left out of ANNs because its inclusion impairs learning. This is problematic, because one of the great advantages of ANNs for neuroscience research is their ability to learn complicated, realistic tasks. Here, by taking inspiration from feedforward inhibitory interneurons in the brain we show that we can develop ANNs with separate populations of excitatory and inhibitory units that learn just as well as standard ANNs. We call these networks Dale's ANNs (DANNs). We present two insights that enable DANNs to learn well: (1) DANNs are related to normalization schemes, and can be initialized such that the inhibition centres and standardizes the excitatory activity, (2) updates to inhibitory neuron parameters should be scaled using corrections based on the Fisher Information matrix. These results demonstrate how ANNs that respect Dale's principle can be built without sacrificing learning performance, which is important for future work using ANNs as models of the brain. The results may also have interesting implications for how inhibitory plasticity in the real brain operates

Macrostate Data Clustering

We develop an effective nonhierarchical data clustering method using an analogy to the dynamic coarse graining of a stochastic system. Analyzing the eigensystem of an interitem transition matrix identifies fuzzy clusters corresponding to the metastable macroscopic states (macrostates) of a diffusive system. A "minimum uncertainty criterion" determines the linear transformation from eigenvectors to cluster-defining window functions. Eigenspectrum gap and cluster certainty conditions identify the proper number of clusters. The physically motivated fuzzy representation and associated uncertainty analysis distinguishes macrostate clustering from spectral partitioning methods. Macrostate data clustering solves a variety of test cases that challenge other methods.Comment: keywords: cluster analysis, clustering, pattern recognition, spectral graph theory, dynamic eigenvectors, machine learning, macrostates, classificatio

Data clustering and noise undressing for correlation matrices

We discuss a new approach to data clustering. We find that maximum likelihood leads naturally to an Hamiltonian of Potts variables which depends on the correlation matrix and whose low temperature behavior describes the correlation structure of the data. For random, uncorrelated data sets no correlation structure emerges. On the other hand for data sets with a built-in cluster structure, the method is able to detect and recover efficiently that structure. Finally we apply the method to financial time series, where the low temperature behavior reveals a non trivial clustering.Comment: 8 pages, 5 figures, completely rewritten and enlarged version of cond-mat/0003241. Submitted to Phys. Rev.

Unified characterisations of resolution hardness measures

Various "hardness" measures have been studied for resolution, providing theoretical insight into the proof complexity of resolution and its fragments, as well as explanations for the hardness of instances in SAT solving. In this paper we aim at a unified view of a number of hardness measures, including different measures of width, space and size of resolution proofs. Our main contribution is a unified game-theoretic characterisation of these measures. As consequences we obtain new relations between the different hardness measures. In particular, we prove a generalised version of Atserias and Dalmau's result on the relation between resolution width and space from [5]

Preferencial growth: exact solution of the time dependent distributions

We consider a preferential growth model where particles are added one by one to the system consisting of clusters of particles. A new particle can either form a new cluster (with probability q) or join an already existing cluster with a probability proportional to the size thereof. We calculate exactly the probability \Pm_i(k,t) that the size of the i-th cluster at time t is k. We analyze the asymptotics, the scaling properties of the size distribution and of the mean size as well as the relation of our system to recent network models.Comment: 8 pages, 4 figure

Scaling of the distribution of fluctuations of financial market indices

We study the distribution of fluctuations over a time scale $\Delta t$ (i.e., the returns) of the S&P 500 index by analyzing three distinct databases. Database (i) contains approximately 1 million records sampled at 1 min intervals for the 13-year period 1984-1996, database (ii) contains 8686 daily records for the 35-year period 1962-1996, and database (iii) contains 852 monthly records for the 71-year period 1926-1996. We compute the probability distributions of returns over a time scale $\Delta t$, where $\Delta t$ varies approximately over a factor of 10^4 - from 1 min up to more than 1 month. We find that the distributions for $\Delta t \leq$ 4 days (1560 mins) are consistent with a power-law asymptotic behavior, characterized by an exponent $\alpha \approx 3$, well outside the stable L\'evy regime $0 < \alpha < 2$. To test the robustness of the S&P result, we perform a parallel analysis on two other financial market indices. Database (iv) contains 3560 daily records of the NIKKEI index for the 14-year period 1984-97, and database (v) contains 4649 daily records of the Hang-Seng index for the 18-year period 1980-97. We find estimates of $\alpha$ consistent with those describing the distribution of S&P 500 daily-returns. One possible reason for the scaling of these distributions is the long persistence of the autocorrelation function of the volatility. For time scales longer than $(\Delta t)_{\times} \approx 4$ days, our results are consistent with slow convergence to Gaussian behavior.Comment: 12 pages in multicol LaTeX format with 27 postscript figures (Submitted to PRE May 20, 1999). See http://polymer.bu.edu/~amaral/Professional.html for more of our work on this are

Topoisomerase 1 inhibition reversibly impairs synaptic function

Topoisomerases are enzymes that resolve DNA supercoiling during cell division and gene transcription. Inhibitors of these enzymes are used to treat multiple forms of cancer. Recently we found that topoisomerase inhibitors have profound effects on synaptic genes expressed in the brain. Here we examine the contribution of a clinically used topoisomerase inhibitor on the expression of synaptic proteins and synaptic transmission. We find that inhibition of topoisomerase 1 (TOP1) dampens excitatory and inhibitory synaptic transmission in cortical neurons. Additionally, these effects are fully reversible, because synaptic protein levels and synaptic transmission recover upon washout of the TOP1 inhibitor. These findings provide insights into how inhibition of TOP1 impacts synaptic function in neurons

Universal and non-universal properties of cross-correlations in financial time series

We use methods of random matrix theory to analyze the cross-correlation matrix C of price changes of the largest 1000 US stocks for the 2-year period 1994-95. We find that the statistics of most of the eigenvalues in the spectrum of C agree with the predictions of random matrix theory, but there are deviations for a few of the largest eigenvalues. We find that C has the universal properties of the Gaussian orthogonal ensemble of random matrices. Furthermore, we analyze the eigenvectors of C through their inverse participation ratio and find eigenvectors with large inverse participation ratios at both edges of the eigenvalue spectrum--a situation reminiscent of results in localization theory.Comment: 14 pages, 3 figures, Revte

Photo-antagonism of the GABAA receptor

Neurotransmitter receptor trafficking is fundamentally important for synaptic transmission and neural network activity. GABAA receptors and inhibitory synapses are vital components of brain function, yet much of our knowledge regarding receptor mobility and function at inhibitory synapses is derived indirectly from using recombinant receptors, antibody-tagged native receptors and pharmacological treatments. Here we describe the use of a set of research tools that can irreversibly bind to and affect the function of recombinant and neuronal GABAA receptors following ultraviolet photoactivation. These compounds are based on the competitive antagonist gabazine and incorporate a variety of photoactive groups. By using site-directed mutagenesis and ligand-docking studies, they reveal new areas of the GABA binding site at the interface between receptor β and α subunits. These compounds enable the selected inactivation of native GABAA receptor populations providing new insight into the function of inhibitory synapses and extrasynaptic receptors in controlling neuronal excitation