Closing the loop between neural network simulators and the OpenAI Gym

Since the enormous breakthroughs in machine learning over the last decade, functional neural network models are of growing interest for many researchers in the field of computational neuroscience. One major branch of research is concerned with biologically plausible implementations of reinforcement learning, with a variety of different models developed over the recent years. However, most studies in this area are conducted with custom simulation scripts and manually implemented tasks. This makes it hard for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. This toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments of varying complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym

Minimum Participation Rules for the Provision of Public Goods

This paper considers the endogenous formation of an institution to provide a public good. If the institution governs only its members, players have an incentive to free ride on the institution formation of others and the social dilemma is simply shifted to a higher level. Addressing this second-order social dilemma, we study the effectiveness of three different minimum participation requirements: 1. full participation / unanimity rule; 2. partial participation; 3. unanimity first and in case of failure partial participation. While unanimity is most effective once established, one might suspect that a weaker minimum participation rule is preferable in practice as it might facilitate the formation of the institution. The data of our laboratory experiment do not support this latter view, though. In fact, weakening the participation requirement does not increase the number of implemented institutions. Thus, we conclude that the most effective participation requirement is the unanimity rule which leaves no room for free riding on either level of the social dilemma.public goods, coalition formation, endogenous institutions

Role of microRNAs in stem/progenitor cells and cardiovascular repair

MicroRNAs (miRNAs), small non-coding RNAs, play a critical role in differentiation and self-renewal of pluripotent stem cells, as well as in differentiation of cardiovascular lineage cells. Several miRNAs have been demonstrated to repress stemness factors such as Oct4, Nanog, Sox2 and Klf4 in embryonic stem cells, thereby promoting embryonic stem cell differentiation. Furthermore, targeting of different miRNAs promotes reprogramming towards induced pluripotent stem cells. MicroRNAs are critical for vascular smooth muscle cell differentiation and phenotype regulation, and miR-143 and miR-145 play a particularly important role in this respect. Notably, these miRNAs are down-regulated in several cardiovascular disease states, such as in atherosclerotic lesions and vascular neointima formation. MicroRNAs are critical regulators of endothelial cell differentiation and ischaemia-induced neovascularization. miR-126 is important for vascular integrity, endothelial cell proliferation and neovascularization. miR-1 and miR-133 are highly expressed in cardiomyocytes and their precursors and regulate cardiomyogenesis. In addition, miR-499 promotes differentiation of cardiomyocyte progenitor cells. Notably, miRNA expression is altered in cardiovascular disease states, and recent studies suggest that dysregulated miRNAs may limit cardiovascular repair responses. Dysregulation of miRNAs may lead to an altered function and differentiation of cardiovascular progenitor cells, which is also likely to represent a limitation of autologous cell-based treatment approaches in these patients. These findings suggest that targeting of specific miRNAs may represent an interesting novel opportunity to impact on endogenous cardiovascular repair responses, including effects on stem/progenitor cell differentiation and functions. This approach may also serve to optimize cell-based treatment approaches in patients with cardiovascular diseas

Signatures of criticality arise in simple neural population models with correlations

Large-scale recordings of neuronal activity make it possible to gain insights into the collective activity of neural ensembles. It has been hypothesized that neural populations might be optimized to operate at a 'thermodynamic critical point', and that this property has implications for information processing. Support for this notion has come from a series of studies which identified statistical signatures of criticality in the ensemble activity of retinal ganglion cells. What are the underlying mechanisms that give rise to these observations? Here we show that signatures of criticality arise even in simple feed-forward models of retinal population activity. In particular, they occur whenever neural population data exhibits correlations, and is randomly sub-sampled during data analysis. These results show that signatures of criticality are not necessarily indicative of an optimized coding strategy, and challenge the utility of analysis approaches based on equilibrium thermodynamics for understanding partially observed biological systems.Comment: 36 pages, LaTeX; added journal reference on page 1, added link to code repositor

Analyse von Bildmerkmalen zur Identifikation wichtiger Bildregionen

Eine zuverlässige Erkennung wichtiger Bildregionen ist die Grundlage für viele Verfahren im Bereich der Bildverarbeitung wie beispielsweise bei der Bildkompression, bei Verfahren zur Anpassung der Bildauflösung oder beim Einfügen digitaler Wasserzeichen in Bilder. Es wurde ein System entwickelt, das Merkmalspunkte in Bildern identifiziert und diese nutzt, um wichtige Bildbereiche zu identifizieren. Zur Berechnung der Merkmalspunkte wird das SURF-Verfahren (Speeded Up Robust Features) verwendet. Die gefundenen Merkmale werden in einem zweiten Schritt einzelnen Bildregionen zugeordnet. Die Qualität der ermittelten Regionen sowie das Laufzeitverhalten der verschiedenen Verfahren werden anhand einer umfangreichen Bilddatenbank analysiert

Resolving Horizon-Scale Dynamics of Sagittarius A*

Sagittarius A* (Sgr A*), the supermassive black hole at the heart of our galaxy, provides unique opportunities to study black hole accretion, jet formation, and gravitational physics. The rapid structural changes in Sgr A*'s emission pose a significant challenge for traditional imaging techniques. We present dynamic reconstructions of Sgr A* using Event Horizon Telescope (EHT) data from April 6th and 7th, 2017, analyzed with a one-minute temporal resolution with the Resolve framework. This Bayesian approach employs adaptive Gaussian Processes and Variational Inference for data-driven self-regularization. Our results not only fully confirm the initial findings by the EHT Collaboration for a time-averaged source but also reveal intricate details about the temporal dynamics within the black hole environment. We find an intriguing dynamic feature on April 6th that propagates in a clock-wise direction. Geometric modelling with ray-tracing, although not fully conclusive, indicates compatibility with high-inclination configurations of about $\theta_o = 160^\circ$, as seen in other studies

Statistical Analysis of Multi-Cell Recordings: Linking Population Coding Models to Experimental Data

Modern recording techniques such as multi-electrode arrays and two-photon imaging methods are capable of simultaneously monitoring the activity of large neuronal ensembles at single cell resolution. These methods finally give us the means to address some of the most crucial questions in systems neuroscience: what are the dynamics of neural population activity? How do populations of neurons perform computations? What is the functional organization of neural ensembles? While the wealth of new experimental data generated by these techniques provides exciting opportunities to test ideas about how neural ensembles operate, it also provides major challenges: multi-cell recordings necessarily yield data which is high-dimensional in nature. Understanding this kind of data requires powerful statistical techniques for capturing the structure of the neural population responses, as well as their relationship with external stimuli or behavioral observations. Furthermore, linking recorded neural population activity to the predictions of theoretical models of population coding has turned out not to be straightforward. These challenges motivated us to organize a workshop at the 2009 Computational Neuroscience Meeting in Berlin to discuss these issues. In order to collect some of the recent progress in this field, and to foster discussion on the most important directions and most pressing questions, we issued a call for papers for this Research Topic. We asked authors to address the following four questions: 1. What classes of statistical methods are most useful for modeling population activity? 2. What are the main limitations of current approaches, and what can be done to overcome them? 3. How can statistical methods be used to empirically test existing models of (probabilistic) population coding? 4. What role can statistical methods play in formulating novel hypotheses about the principles of information processing in neural populations? A total of 15 papers addressing questions related to these themes are now collected in this Research Topic. Three of these articles have resulted in “Focused reviews” in Frontiers in Neuroscience (Crumiller et al., 2011; Rosenbaum et al., 2011; Tchumatchenko et al., 2011), illustrating the great interest in the topic. Many of the articles are devoted to a better understanding of how correlations arise in neural circuits, and how they can be detected, modeled, and interpreted. For example, by modeling how pairwise correlations are transformed by spiking non-linearities in simple neural circuits, Tchumatchenko et al. (2010) show that pairwise correlation coefficients have to be interpreted with care, since their magnitude can depend strongly on the temporal statistics of their input-correlations. In a similar spirit, Rosenbaum et al. (2010) study how correlations can arise and accumulate in feed-forward circuits as a result of pooling of correlated inputs. Lyamzin et al. (2010) and Krumin et al. (2010) present methods for simulating correlated population activity and extend previous work to more general settings. The method of Lyamzin et al. (2010) allows one to generate synthetic spike trains which match commonly reported statistical properties, such as time varying firing rates as well signal and noise correlations. The Hawkes framework presented by Krumin et al. (2010) allows one to fit models of recurrent population activity to the correlation-structure of experimental data. Louis et al. (2010) present a novel method for generating surrogate spike trains which can be useful when trying to assess the significance and time-scale of correlations in neural spike trains. Finally, Pipa and Munk (2011) study spike synchronization in prefrontal cortex during working memory. A number of studies are also devoted to advancing our methodological toolkit for analyzing various aspects of population activity (Gerwinn et al., 2010; Machens, 2010; Staude et al., 2010; Yu et al., 2010). For example, Gerwinn et al. (2010) explain how full probabilistic inference can be performed in the popular model class of generalized linear models (GLMs), and study the effect of using prior distributions on the parameters of the stimulus and coupling filters. Staude et al. (2010) extend a method for detecting higher-order correlations between neurons via population spike counts to non-stationary settings. Yu et al. (2010) describe a new technique for estimating the information rate of a population of neurons using frequency-domain methods. Machens (2010) introduces a novel extension of principal component analysis for separating the variability of a neural response into different sources. Focusing less on the spike responses of neural populations but on aggregate signals of population activity, Boatman-Reich et al. (2010) and Hoerzer et al. (2010) describe methods for a quantitative analysis of field potential recordings. While Boatman-Reich et al. (2010) discuss a number of existing techniques in a unified framework and highlight the potential pitfalls associated with such approaches, Hoerzer et al. (2010) demonstrate how multivariate autoregressive models and the concept of Granger causality can be used to infer local functional connectivity in area V4 of behaving macaques. A final group of studies is devoted to understanding experimental data in light of computational models (Galán et al., 2010; Pandarinath et al., 2010; Shteingart et al., 2010). Pandarinath et al. (2010) present a novel mechanism that may explain how neural networks in the retina switch from one state to another by a change in gap junction coupling, and conjecture that this mechanism might also be found in other neural circuits. Galán et al. (2010) present a model of how hypoxia may change the network structure in the respiratory networks in the brainstem, and analyze neural correlations in multi-electrode recordings in light of this model. Finally, Shteingart et al. (2010) show that the spontaneous activation sequences they find in cultured networks cannot be explained by Zipf’s law, but rather require a wrestling model. The papers of this Research Topic thus span a wide range of topics in the statistical modeling of multi-cell recordings. Together with other recent advances, they provide us with a useful toolkit to tackle the challenges presented by the vast amount of data collected with modern recording techniques. The impact of novel statistical methods on the field and their potential to generate scientific progress, however, depends critically on how readily they can be adopted and applied by laboratories and researchers working with experimental data. An important step toward this goal is to also publish computer code along with the articles (Barnes, 2010) as a successful implementation of advanced methods also relies on many details which are hard to communicate in the article itself. In this way it becomes much more likely that other researchers can actually use the methods, and unnecessary re-implementations can be avoided. Some of the papers in this Research Topic already follow this goal (Gerwinn et al., 2010; Louis et al., 2010; Lyamzin et al., 2010). We hope that this practice becomes more and more common in the future and encourage authors and editors of Research Topics to make as much code available as possible, ideally in a format that can be easily integrated with existing software sharing initiatives (Herz et al., 2008; Goldberg et al., 2009)

Bose-Einstein condensate coupled to a nanomechanical resonator on an atom chip

We theoretically study the coupling of Bose-Einstein condensed atoms to the mechanical oscillations of a nanoscale cantilever with a magnetic tip. This is an experimentally viable hybrid quantum system which allows one to explore the interface of quantum optics and condensed matter physics. We propose an experiment where easily detectable atomic spin-flips are induced by the cantilever motion. This can be used to probe thermal oscillations of the cantilever with the atoms. At low cantilever temperatures, as realized in recent experiments, the backaction of the atoms onto the cantilever is significant and the system represents a mechanical analog of cavity quantum electrodynamics. With high but realistic cantilever quality factors, the strong coupling regime can be reached, either with single atoms or collectively with Bose-Einstein condensates. We discuss an implementation on an atom chip.Comment: published version (5 pages, 3 figures