Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation

Early sensory systems in the brain rapidly adapt to fluctuating input statistics, which requires recurrent communication between neurons. Mechanistically, such recurrent communication is often indirect and mediated by local interneurons. In this work, we explore the computational benefits of mediating recurrent communication via interneurons compared with direct recurrent connections. To this end, we consider two mathematically tractable recurrent neural networks that statistically whiten their inputs -- one with direct recurrent connections and the other with interneurons that mediate recurrent communication. By analyzing the corresponding continuous synaptic dynamics and numerically simulating the networks, we show that the network with interneurons is more robust to initialization than the network with direct recurrent connections in the sense that the convergence time for the synaptic dynamics in the network with interneurons (resp. direct recurrent connections) scales logarithmically (resp. linearly) with the spectrum of their initialization. Our results suggest that interneurons are computationally useful for rapid adaptation to changing input statistics. Interestingly, the network with interneurons is an overparameterized solution of the whitening objective for the network with direct recurrent connections, so our results can be viewed as a recurrent neural network analogue of the implicit acceleration phenomenon observed in overparameterized feedforward linear networks.Comment: 16 pages, 5 figure

Adaptive whitening with fast gain modulation and slow synaptic plasticity

Neurons in early sensory areas rapidly adapt to changing sensory statistics, both by normalizing the variance of their individual responses and by reducing correlations between their responses. Together, these transformations may be viewed as an adaptive form of statistical whitening. Existing mechanistic models of adaptive whitening exclusively use either synaptic plasticity or gain modulation as the biological substrate for adaptation; however, on their own, each of these models has significant limitations. In this work, we unify these approaches in a normative multi-timescale mechanistic model that adaptively whitens its responses with complementary computational roles for synaptic plasticity and gain modulation. Gains are modified on a fast timescale to adapt to the current statistical context, whereas synapses are modified on a slow timescale to match structural properties of the input statistics that are invariant across contexts. Our model is derived from a novel multi-timescale whitening objective that factorizes the inverse whitening matrix into basis vectors, which correspond to synaptic weights, and a diagonal matrix, which corresponds to neuronal gains. We test our model on synthetic and natural datasets and find that the synapses learn optimal configurations over long timescales that enable adaptive whitening on short timescales using gain modulation.Comment: NeurIPS 2023 Spotlight; 18 pages, 8 figure

Adaptive whitening in neural populations with gain-modulating interneurons

Statistical whitening transformations play a fundamental role in many computational systems, and may also play an important role in biological sensory systems. Existing neural circuit models of adaptive whitening operate by modifying synaptic interactions; however, such modifications would seem both too slow and insufficiently reversible. Motivated by the extensive neuroscience literature on gain modulation, we propose an alternative model that adaptively whitens its responses by modulating the gains of individual neurons. Starting from a novel whitening objective, we derive an online algorithm that whitens its outputs by adjusting the marginal variances of an overcomplete set of projections. We map the algorithm onto a recurrent neural network with fixed synaptic weights and gain-modulating interneurons. We demonstrate numerically that sign-constraining the gains improves robustness of the network to ill-conditioned inputs, and a generalization of the circuit achieves a form of local whitening in convolutional populations, such as those found throughout the visual or auditory systems.Comment: 20 pages, 10 figures (incl. appendix). To appear in the Proceedings of the 40th International Conference on Machine Learnin

Effective affinities in microarray data

In the past couple of years several studies have shown that hybridization in Affymetrix DNA microarrays can be rather well understood on the basis of simple models of physical chemistry. In the majority of the cases a Langmuir isotherm was used to fit experimental data. Although there is a general consensus about this approach, some discrepancies between different studies are evident. For instance, some authors have fitted the hybridization affinities from the microarray fluorescent intensities, while others used affinities obtained from melting experiments in solution. The former approach yields fitted affinities that at first sight are only partially consistent with solution values. In this paper we show that this discrepancy exists only superficially: a sufficiently complete model provides effective affinities which are fully consistent with those fitted to experimental data. This link provides new insight on the relevant processes underlying the functioning of DNA microarrays.Comment: 8 pages, 6 figure

The Structure of the Big Bang from Higher-Dimensional Embeddings

We give relations for the embedding of spatially-flat Friedmann-Robertson-Walker cosmological models of Einstein's theory in flat manifolds of the type used in Kaluza-Klein theory. We present embedding diagrams that depict different 4D universes as hypersurfaces in a higher dimensional flat manifold. The morphology of the hypersurfaces is found to depend on the equation of state of the matter. The hypersurfaces possess a line-like curvature singularity infinitesimally close to the $t = 0^+$ 3-surface, where $t$ is the time expired since the big bang. The family of timelike comoving geodesics on any given hypersurface is found to have a caustic on the singular line, which we conclude is the 5D position of the point-like big bang.Comment: 11 pages, 5 figures, revtex4, accepted in Class. Quant. Gra

Highly Diastereo- and Enantioselective CuH-Catalyzed Synthesis of 2,3-Disubstituted Indolines

A diastereo- and enantioselective CuH-catalyzed method for the preparation of highly functionalized indolines is reported. The mild reaction conditions and high degree of functional group compatibility as demonstrated with substrates bearing heterocycles, olefins, and substituted aromatic groups, renders this technique highly valuable for the synthesis of a variety of cis-2,3-disubstituted indolines in high yield and enantioeselectivity.National Institutes of Health (U.S.) (Award GM46059)Danish Council for Independent Research (Postdoctoral Fellowship

Atropselective syntheses of (-) and (+) rugulotrosin A utilizing point-to-axial chirality transfer

Chiral, dimeric natural products containing complex structures and interesting biological properties have inspired chemists and biologists for decades. A seven-step total synthesis of the axially chiral, dimeric tetrahydroxanthone natural product rugulotrosin A is described. The synthesis employs a one-pot Suzuki coupling/dimerization to generate the requisite 2,2'-biaryl linkage. Highly selective point-to-axial chirality transfer was achieved using palladium catalysis with achiral phosphine ligands. Single X-ray crystal diffraction data were obtained to confirm both the atropisomeric configuration and absolute stereochemistry of rugulotrosin A. Computational studies are described to rationalize the atropselectivity observed in the key dimerization step. Comparison of the crude fungal extract with synthetic rugulotrosin A and its atropisomer verified that nature generates a single atropisomer of the natural product.P50 GM067041 - NIGMS NIH HHS; R01 GM099920 - NIGMS NIH HHS; GM-067041 - NIGMS NIH HHS; GM-099920 - NIGMS NIH HH

Preparation of anti-vicinal amino alcohols: asymmetric synthesis of D-erythro-Sphinganine, (+)-spisulosine and D-ribo-phytosphingosine

Two variations of the Overman rearrangement have been developed for the highly selective synthesis of anti-vicinal amino alcohol natural products. A MOM-ether directed palladium(II)-catalyzed rearrangement of an allylic trichloroacetimidate was used as the key step for the preparation of the protein kinase C inhibitor D-erythro-sphinganine and the antitumor agent (+)-spisulosine, while the Overman rearrangement of chiral allylic trichloroacetimidates generated by asymmetric reduction of an alpha,beta-unsaturated methyl ketone allowed rapid access to both D-ribo-phytosphingosine and L-arabino-phytosphingosine

Analysis of multiplex gene expression maps obtained by voxelation

BackgroundGene expression signatures in the mammalian brain hold the key to understanding neural development and neurological disease. Researchers have previously used voxelation in combination with microarrays for acquisition of genome-wide atlases of expression patterns in the mouse brain. On the other hand, some work has been performed on studying gene functions, without taking into account the location information of a gene's expression in a mouse brain. In this paper, we present an approach for identifying the relation between gene expression maps obtained by voxelation and gene functions.ResultsTo analyze the dataset, we chose typical genes as queries and aimed at discovering similar gene groups. Gene similarity was determined by using the wavelet features extracted from the left and right hemispheres averaged gene expression maps, and by the Euclidean distance between each pair of feature vectors. We also performed a multiple clustering approach on the gene expression maps, combined with hierarchical clustering. Among each group of similar genes and clusters, the gene function similarity was measured by calculating the average gene function distances in the gene ontology structure. By applying our methodology to find similar genes to certain target genes we were able to improve our understanding of gene expression patterns and gene functions. By applying the clustering analysis method, we obtained significant clusters, which have both very similar gene expression maps and very similar gene functions respectively to their corresponding gene ontologies. The cellular component ontology resulted in prominent clusters expressed in cortex and corpus callosum. The molecular function ontology gave prominent clusters in cortex, corpus callosum and hypothalamus. The biological process ontology resulted in clusters in cortex, hypothalamus and choroid plexus. Clusters from all three ontologies combined were most prominently expressed in cortex and corpus callosum.ConclusionThe experimental results confirm the hypothesis that genes with similar gene expression maps might have similar gene functions. The voxelation data takes into account the location information of gene expression level in mouse brain, which is novel in related research. The proposed approach can potentially be used to predict gene functions and provide helpful suggestions to biologists