Learning probabilistic neural representations with randomly connected circuits

The brain represents and reasons probabilistically about complex stimuli and motor actions using a noisy, spike-based neural code. A key building block for such neural computations, as well as the basis for supervised and unsupervised learning, is the ability to estimate the surprise or likelihood of incoming high-dimensional neural activity patterns. Despite progress in statistical modeling of neural responses and deep learning, current approaches either do not scale to large neural populations or cannot be implemented using biologically realistic mechanisms. Inspired by the sparse and random connectivity of real neuronal circuits, we present a model for neural codes that accurately estimates the likelihood of individual spiking patterns and has a straightforward, scalable, efficient, learnable, and realistic neural implementation. This model’s performance on simultaneously recorded spiking activity of >100 neurons in the monkey visual and prefrontal cortices is comparable with or better than that of state-of-the-art models. Importantly, the model can be learned using a small number of samples and using a local learning rule that utilizes noise intrinsic to neural circuits. Slower, structural changes in random connectivity, consistent with rewiring and pruning processes, further improve the efficiency and sparseness of the resulting neural representations. Our results merge insights from neuroanatomy, machine learning, and theoretical neuroscience to suggest random sparse connectivity as a key design principle for neuronal computation

maxent_toolbox: Maximum Entropy Toolbox for MATLAB, version 1.0.2

<p>Maximum entropy toolbox for MATLAB is a free, open-source toolbox for finding the maximum entropy distribution of training data, based on a set of constraints or observables over the data</p> <p>Maximum entropy models give the mathematically minimal probabilistic models of the states or configurations of a systems, given the mean values of some set of observed functions (Jaynes 1957). Since the entropy of a distribution measures the randomness or lack of interaction among different variables (Shannon 1949), the minimally structured distribution given a set of observables is the distribution with the maximal entropy that is consistent with these observables.</p> <p>Mathematically, in its discrete form, if $x_i$ are the elements of the system (here variables taking discrete values), then the maximum entropy model for $p(x_1,x_2 \ldots x_n)$ which is consistent with a set of observables of the form $\langle f_{i}(x_1,\ldots,x_n)\rangle_{p}$ has a unique solution in the form:</p> <p>$\hat{p}(x_1 \ldots x_n)=\frac{1}{Z} \exp[\sum_i \lambda_i f_i(x)]$</p> <p>where $\lambda_i$ are Lagrange multipliers and $Z=\sum_{x}\hat{p}(x)$. Since this problem is convex, the maximum entropy solution is unique and can be found numerically.</p> <p>This family of models thus offer the minimal model that is consistent with the constraints. This approach has been used as a way to approximate or explore the nature of correlations in systems of many variables ranging from systems of spins, populations of neurons, genes, pixels in images, words in language etc.</p> <p>The current toolbox allows for learning maximum entropy distributions of binary variables $x_i\in \{0,1\}$ and distributions of patterns of the form 1000110100. The toolbox takes as an input a set of samples of activity patterns and learns a model of the probability over all states, thus extrapolating to the entire distribution over all possible activity patterns.</p> <p>The user can choose between several variants of maximum entropy models, each relying on a different set of observables or constraints. The maximum entropy models currently supported by this package are:</p> <ul> <li>Independent model, which uses $\langle x_i \rangle_{data}$ as constraints</li> <li>Pairwise maximum entropy model: $\langle x_i \rangle_{data}$ and $\langle x_i x_j\rangle_{data}$</li> <li>K-Synchrony model: $\langle \sum_{i}x_i \rangle_{data}$</li> <li>K-Pairwise model</li> <li>Arbitrary set of high-order correlations</li> <li>Any combination of the above models</li> </ul> <p>The toolbox automatically switches between exhaustive solutions for small (<30) groups of variables and Markov Chain Monte Carlo (MCMC) methods for larger groups and can be used to learn distributions of up to several hundreds of binary variables. The software is provided as an installable toolbox for MATLAB, and most of the code is written in heavily optimized C++ precompiled for Windows (64 bit), OS X and Linux (CentOS).</p> <p>The project is hosted in GitHub: https://orimaoz.github.io/maxent_toolbox/</p

COVID-19 in Patients with Inflammatory Bowel Disease: The Israeli Experience

Background: Crohn’s disease (CD) and ulcerative colitis (UC) are chronic, immune-mediated inflammatory bowel diseases (IBD) affecting millions of people worldwide. IBD therapies, designed for continuous immune suppression, often render patients more susceptible to infections. The effect of the immune suppression on the risk of coronavirus disease-19 (COVID-19) is not fully determined yet. Objective: To describe COVID-19 characteristics and outcomes and to evaluate the association between IBD phenotypes, infection outcomes and immunomodulatory therapies. Methods: In this multi-center study, we prospectively followed IBD patients with proven COVID-19. De-identified data from medical charts were collected including age, gender, IBD type, IBD clinical activity, IBD treatments, comorbidities, symptoms and outcomes of COVID-19. A multivariable regression model was used to examine the effect of immunosuppressant drugs on the risk of infection by COVID-19 and the outcomes. Results: Of 144 IBD patients, 104 (72%) were CD and 40 (28%) were UC. Mean age was 32.2 ± 12.6 years. No mortalities were reported. In total, 94 patients (65.3%) received biologic therapy. Of them, 51 (54%) at escalated doses, 10 (11%) in combination with immunomodulators and 9 (10%) with concomitant corticosteroids. Disease location, behavior and activity did not correlate with the severity of COVID-19. Biologics as monotherapy or with immunomodulators or corticosteroids were not associated with more severe infection. On the contrary, patients receiving biologics had significantly milder infection course (p = 0.001) and were less likely to be hospitalized (p = 0.001). Treatment was postponed in 34.7% of patients until recovery from COVID-19, without consequent exacerbation. Conclusion: We did not witness aggravated COVID-19 outcomes in patients with IBD. Patients treated with biologics had a favorable outcome