Training deep neural density estimators to identify mechanistic models of neural dynamics

Mechanistic modeling in neuroscience aims to explain observed phenomena in terms of underlying causes. However, determining which model parameters agree with complex and stochastic neural data presents a significant challenge. We address this challenge with a machine learning tool which uses deep neural density estimators-- trained using model simulations-- to carry out Bayesian inference and retrieve the full space of parameters compatible with raw data or selected data features. Our method is scalable in parameters and data features, and can rapidly analyze new data after initial training. We demonstrate the power and flexibility of our approach on receptive fields, ion channels, and Hodgkin-Huxley models. We also characterize the space of circuit configurations giving rise to rhythmic activity in the crustacean stomatogastric ganglion, and use these results to derive hypotheses for underlying compensation mechanisms. Our approach will help close the gap between data-driven and theory-driven models of neural dynamics

Parametric inference in the large data limit using maximally informative models

Motivated by data-rich experiments in transcriptional regulation and sensory neuroscience, we consider the following general problem in statistical inference. When exposed to a high-dimensional signal S, a system of interest computes a representation R of that signal which is then observed through a noisy measurement M. From a large number of signals and measurements, we wish to infer the "filter" that maps S to R. However, the standard method for solving such problems, likelihood-based inference, requires perfect a priori knowledge of the "noise function" mapping R to M. In practice such noise functions are usually known only approximately, if at all, and using an incorrect noise function will typically bias the inferred filter. Here we show that, in the large data limit, this need for a pre-characterized noise function can be circumvented by searching for filters that instead maximize the mutual information I[M;R] between observed measurements and predicted representations. Moreover, if the correct filter lies within the space of filters being explored, maximizing mutual information becomes equivalent to simultaneously maximizing every dependence measure that satisfies the Data Processing Inequality. It is important to note that maximizing mutual information will typically leave a small number of directions in parameter space unconstrained. We term these directions "diffeomorphic modes" and present an equation that allows these modes to be derived systematically. The presence of diffeomorphic modes reflects a fundamental and nontrivial substructure within parameter space, one that is obscured by standard likelihood-based inference.Comment: To appear in Neural Computatio

A Bayesian approach for inferring neuronal connectivity from calcium fluorescent imaging data

Deducing the structure of neural circuits is one of the central problems of modern neuroscience. Recently-introduced calcium fluorescent imaging methods permit experimentalists to observe network activity in large populations of neurons, but these techniques provide only indirect observations of neural spike trains, with limited time resolution and signal quality. In this work we present a Bayesian approach for inferring neural circuitry given this type of imaging data. We model the network activity in terms of a collection of coupled hidden Markov chains, with each chain corresponding to a single neuron in the network and the coupling between the chains reflecting the network's connectivity matrix. We derive a Monte Carlo Expectation--Maximization algorithm for fitting the model parameters; to obtain the sufficient statistics in a computationally-efficient manner, we introduce a specialized blockwise-Gibbs algorithm for sampling from the joint activity of all observed neurons given the observed fluorescence data. We perform large-scale simulations of randomly connected neuronal networks with biophysically realistic parameters and find that the proposed methods can accurately infer the connectivity in these networks given reasonable experimental and computational constraints. In addition, the estimation accuracy may be improved significantly by incorporating prior knowledge about the sparseness of connectivity in the network, via standard L$_1$ penalization methods.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS303 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Enhancing temporal series of Sentinel-2 and Sentinel-3 data products: from classical regression to deep learning approach

Treball de Final de Màster Universitari Erasmus Mundus en Tecnologia Geoespacial (Pla de 2013). Codi: SIW013. Curs acadèmic 2020-2021The free and open availability of satellite images covering global extent in recent days provides many novel opportunities for global monitoring of the earth’s surface. Sentinel-2 (S2) and Sentinel-3 (S3) satellite missions capture mid to high resolution imagery with frequent revisit and show data synergy as they both focus on land and ocean observational needs. Specifically, the high temporal resolution of S3 (1-2 day revisit) presents potential in filling the data gaps in S2 (5 day revisit) vegetation products. In this scenario, this study assesses the feasibility of using Sentinel-3 images for Sentinel-2 vegetation products estimation using machine learning (ML) and deep learning (DL) approaches. This study employs four state of the art ML regression algorithms, linear regression, ridge regression, Support Vector Regression (SVR) and Random Forest Regression (RFR) and two DL network architectures with different depth and complexities, Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) to predict the S2 NDVI and SAVI maps from the S3 spectral bands information. A paired S2/S3 dataset is prepared for the study area covering one S2 tile in Extremadura, Spain. The results demonstrate that all the DL architectures except pixel-wise MLP outperformed the ML models with the 3D CNN performing the best. The best performing 3D CNN architecture obtained remarkable mean squared error (MSE) of 0.00198 for NDVI and 0.00282 for SAVI while the best performing ML algorithms were patch-wise RFR with MSE of 0.0035 in case of NDVI and patchwise SVR with MSE of 0.00586 for SAVI. The models and the dataset prepared for this study will be useful for further research that focus on capitalizing the free and open availability of Sentinel-2 and Sentinel-3 imagery as well as new and advanced technologies to provide better vegetation monitoring capabilities for our planet

Fast non-negative deconvolution for spike train inference from population calcium imaging

Calcium imaging for observing spiking activity from large populations of neurons are quickly gaining popularity. While the raw data are fluorescence movies, the underlying spike trains are of interest. This work presents a fast non-negative deconvolution filter to infer the approximately most likely spike train for each neuron, given the fluorescence observations. This algorithm outperforms optimal linear deconvolution (Wiener filtering) on both simulated and biological data. The performance gains come from restricting the inferred spike trains to be positive (using an interior-point method), unlike the Wiener filter. The algorithm is fast enough that even when imaging over 100 neurons, inference can be performed on the set of all observed traces faster than real-time. Performing optimal spatial filtering on the images further refines the estimates. Importantly, all the parameters required to perform the inference can be estimated using only the fluorescence data, obviating the need to perform joint electrophysiological and imaging calibration experiments.Comment: 22 pages, 10 figure

An Emulator Toolbox to Approximate Radiative Transfer Models with Statistical Learning

Physically-based radiative transfer models (RTMs) help in understanding the processes occurring on the Earth’s surface and their interactions with vegetation and atmosphere. When it comes to studying vegetation properties, RTMs allows us to study light interception by plant canopies and are used in the retrieval of biophysical variables through model inversion. However, advanced RTMs can take a long computational time, which makes them unfeasible in many real applications. To overcome this problem, it has been proposed to substitute RTMs through so-called emulators. Emulators are statistical models that approximate the functioning of RTMs. Emulators are advantageous in real practice because of the computational efficiency and excellent accuracy and flexibility for extrapolation. We hereby present an “Emulator toolbox” that enables analysing multi-output machine learning regression algorithms (MO-MLRAs) on their ability to approximate an RTM. The toolbox is included in the free-access ARTMO’s MATLAB suite for parameter retrieval and model inversion and currently contains both linear and non-linear MO-MLRAs, namely partial least squares regression (PLSR), kernel ridge regression (KRR) and neural networks (NN). These MO-MLRAs have been evaluated on their precision and speed to approximate the soil vegetation atmosphere transfer model SCOPE (Soil Canopy Observation, Photochemistry and Energy balance). SCOPE generates, amongst others, sun-induced chlorophyll fluorescence as the output signal. KRR and NN were evaluated as capable of reconstructing fluorescence spectra with great precision. Relative errors fell below 0.5% when trained with 500 or more samples using cross-validation and principal component analysis to alleviate the underdetermination problem. Moreover, NN reconstructed fluorescence spectra about 50-times faster and KRR about 800-times faster than SCOPE. The Emulator toolbox is foreseen to open new opportunities in the use of advanced RTMs, in which both consistent physical assumptions and data-driven machine learning algorithms live together

The touch and zap method for in vivo whole-cell patch recording of intrinsic and visual responses of cortical neurons and Glial cells

Whole-cell patch recording is an essential tool for quantitatively establishing the biophysics of brain function, particularly in vivo. This method is of particular interest for studying the functional roles of cortical glial cells in the intact brain, which cannot be assessed with extracellular recordings. Nevertheless, a reasonable success rate remains a challenge because of stability, recording duration and electrical quality constraints, particularly for voltage clamp, dynamic clamp or conductance measurements. To address this, we describe "Touch and Zap", an alternative method for whole-cell patch clamp recordings, with the goal of being simpler, quicker and more gentle to brain tissue than previous approaches. Under current clamp mode with a continuous train of hyperpolarizing current pulses, seal formation is initiated immediately upon cell contact, thus the "Touch". By maintaining the current injection, whole-cell access is spontaneously achieved within seconds from the cell-attached configuration by a self-limited membrane electroporation, or "Zap", as seal resistance increases. We present examples of intrinsic and visual responses of neurons and putative glial cells obtained with the revised method from cat and rat cortices in vivo. Recording parameters and biophysical properties obtained with the Touch and Zap method compare favourably with those obtained with the traditional blind patch approach, demonstrating that the revised approach does not compromise the recorded cell. We find that the method is particularly well-suited for whole-cell patch recordings of cortical glial cells in vivo, targeting a wider population of this cell type than the standard method, with better access resistance. Overall, the gentler Touch and Zap method is promising for studying quantitative functional properties in the intact brain with minimal perturbation of the cell's intrinsic properties and local network. Because the Touch and Zap method is performed semi-automatically, this approach is more reproducible and less dependent on experimenter technique