Gradient-free activation maximization for identifying effective stimuli

A fundamental question for understanding brain function is what types of stimuli drive neurons to fire. In visual neuroscience, this question has also been posted as characterizing the receptive field of a neuron. The search for effective stimuli has traditionally been based on a combination of insights from previous studies, intuition, and luck. Recently, the same question has emerged in the study of units in convolutional neural networks (ConvNets), and together with this question a family of solutions were developed that are generally referred to as "feature visualization by activation maximization." We sought to bring in tools and techniques developed for studying ConvNets to the study of biological neural networks. However, one key difference that impedes direct translation of tools is that gradients can be obtained from ConvNets using backpropagation, but such gradients are not available from the brain. To circumvent this problem, we developed a method for gradient-free activation maximization by combining a generative neural network with a genetic algorithm. We termed this method XDream (EXtending DeepDream with real-time evolution for activation maximization), and we have shown that this method can reliably create strong stimuli for neurons in the macaque visual cortex (Ponce et al., 2019). In this paper, we describe extensive experiments characterizing the XDream method by using ConvNet units as in silico models of neurons. We show that XDream is applicable across network layers, architectures, and training sets; examine design choices in the algorithm; and provide practical guides for choosing hyperparameters in the algorithm. XDream is an efficient algorithm for uncovering neuronal tuning preferences in black-box networks using a vast and diverse stimulus space.Comment: 16 pages, 8 figures, 3 table

“Dust in the wind...”, deep learning application to wind energy time series forecasting

To balance electricity production and demand, it is required to use different prediction techniques extensively. Renewable energy, due to its intermittency, increases the complexity and uncertainty of forecasting, and the resulting accuracy impacts all the different players acting around the electricity systems around the world like generators, distributors, retailers, or consumers. Wind forecasting can be done under two major approaches, using meteorological numerical prediction models or based on pure time series input. Deep learning is appearing as a new method that can be used for wind energy prediction. This work develops several deep learning architectures and shows their performance when applied to wind time series. The models have been tested with the most extensive wind dataset available, the National Renewable Laboratory Wind Toolkit, a dataset with 126,692 wind points in North America. The architectures designed are based on different approaches, Multi-Layer Perceptron Networks (MLP), Convolutional Networks (CNN), and Recurrent Networks (RNN). These deep learning architectures have been tested to obtain predictions in a 12-h ahead horizon, and the accuracy is measured with the coefficient of determination, the R² method. The application of the models to wind sites evenly distributed in the North America geography allows us to infer several conclusions on the relationships between methods, terrain, and forecasting complexity. The results show differences between the models and confirm the superior capabilities on the use of deep learning techniques for wind speed forecasting from wind time series data.Peer ReviewedPostprint (published version

Striatal cholinergic interneurons generate beta and gamma oscillations in the corticostriatal circuit and produce motor deficits

Cortico-basal ganglia-thalamic (CBT) neural circuits are critical modulators of cognitive and motor function. When compromised, these circuits contribute to neurological and psychiatric disorders, such as Parkinson's disease (PD). In PD, motor deficits correlate with the emergence of exaggerated beta frequency (15-30 Hz) oscillations throughout the CBT network. However, little is known about how specific cell types within individual CBT brain regions support the generation, propagation, and interaction of oscillatory dynamics throughout the CBT circuit or how specific oscillatory dynamics are related to motor function. Here, we investigated the role of striatal cholinergic interneurons (SChIs) in generating beta and gamma oscillations in cortical-striatal circuits and in influencing movement behavior. We found that selective stimulation of SChIs via optogenetics in normal mice robustly and reversibly amplified beta and gamma oscillations that are supported by distinct mechanisms within striatal-cortical circuits. Whereas beta oscillations are supported robustly in the striatum and all layers of primary motor cortex (M1) through a muscarinic-receptor mediated mechanism, gamma oscillations are largely restricted to the striatum and the deeper layers of M1. Finally, SChI activation led to parkinsonian-like motor deficits in otherwise normal mice. These results highlight the important role of striatal cholinergic interneurons in supporting oscillations in the CBT network that are closely related to movement and parkinsonian motor symptoms.DP2 NS082126 - NINDS NIH HHS; R01 NS081716 - NINDS NIH HHS; R21 NS078660 - NINDS NIH HHShttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC4896681/Published versio