How biased are maximum entropy models?

Maximum entropy models have become popular statistical models in neuroscience and other areas in biology, and can be useful tools for obtaining estimates of mutual information in biological systems. However, maximum entropy models fit to small data sets can be subject to sampling bias; i.e. the true entropy of the data can be severely underestimated. Here we study the sampling properties of estimates of the entropy obtained from maximum entropy models. We show that if the data is generated by a distribution that lies in the model class, the bias is equal to the number of parameters divided by twice the number of observations. However, in practice, the true distribution is usually outside the model class, and we show here that this misspecification can lead to much larger bias. We provide a perturbative approximation of the maximally expected bias when the true model is out of model class, and we illustrate our results using numerical simulations of an Ising model; i.e. the second-order maximum entropy distribution on binary data.

Total and partial cloud amount detection during summer 2005 at Westerland (Sylt, Germany)

The detection of cloudiness is investigated by means of partial and total cloud amount estimations from pyrgeometer radiation measurements and visible all-sky imager observations. The measurements have been performed in Westerland, a seaside resort on the North Sea island of Sylt, Germany, during summer 2005. An improvement to previous studies on this subject resulting in the first time partial cloud amounts (PCAs), defined as cloud amounts without high clouds calculated from longwave downward radiation (LDR) according to the APCADA algorithm (Dürr and Philipona, 2004), are validated against both human observations from the National Meteorological Servive DWD at the nearby airport of Sylt and digital all-sky imaging. The aim is to establish the APCADA scheme at a coastal midlatitude site for longterm observations of cloud cover and to quantify errors resulting from the different methods of detecting cloudiness. Differences between the resulting total cloud amounts (TCAs), defined as cloud amount for all-cloud situations, derived from the camera images and from human observations are within ±1 octa in 72% and within ±2 octa in 85% of the cases. Compared to human observations, PCA measurements, according to APCADA, underestimate the observed cloud cover in 47% of all cases and the differences are within ±1 octa in 60% and ±2 octa in 74% of all cases. Since high cirrus clouds can not be derived from LDR, separate comparisons for all cases without high clouds have been performed showing an agreement within ±1(2) octa in 73(90)% for PCA and also for camera-derived TCA. For this coastal mid-latitude site under investigation, we find similar though slightly smaller agreements to human observations as reported by Dürr and Philipona (2004). Though limited to daytime, the cloud cover retrievals from the sky imager are not really affected by cirrus clouds and provide a more reliable cloud climatology for all-cloud conditions than APCADA

Neural population coding: combining insights from microscopic and mass signals

Behavior relies on the distributed and coordinated activity of neural populations. Population activity can be measured using multi-neuron recordings and neuroimaging. Neural recordings reveal how the heterogeneity, sparseness, timing, and correlation of population activity shape information processing in local networks, whereas neuroimaging shows how long-range coupling and brain states impact on local activity and perception. To obtain an integrated perspective on neural information processing we need to combine knowledge from both levels of investigation. We review recent progress of how neural recordings, neuroimaging, and computational approaches begin to elucidate how interactions between local neural population activity and large-scale dynamics shape the structure and coding capacity of local information representations, make them state-dependent, and control distributed populations that collectively shape behavior

Bayesian Inference for Generalized Linear Models for Spiking Neurons

Generalized Linear Models (GLMs) are commonly used statistical methods for modelling the relationship between neural population activity and presented stimuli. When the dimension of the parameter space is large, strong regularization has to be used in order to fit GLMs to datasets of realistic size without overfitting. By imposing properly chosen priors over parameters, Bayesian inference provides an effective and principled approach for achieving regularization. Here we show how the posterior distribution over model parameters of GLMs can be approximated by a Gaussian using the Expectation Propagation algorithm. In this way, we obtain an estimate of the posterior mean and posterior covariance, allowing us to calculate Bayesian confidence intervals that characterize the uncertainty about the optimal solution. From the posterior we also obtain a different point estimate, namely the posterior mean as opposed to the commonly used maximum a posteriori estimate. We systematically compare the different inference techniques on simulated as well as on multi-electrode recordings of retinal ganglion cells, and explore the effects of the chosen prior and the performance measure used. We find that good performance can be achieved by choosing an Laplace prior together with the posterior mean estimate

Inferring decoding strategy from choice probabilities in the presence of noise correlations

The activity of cortical neurons in sensory areas covaries with perceptual decisions, a relationship often quantified by choice probabilities. While choice probabilities have been measured extensively, their interpretation has remained fraught with difficulty. Here, we derive the mathematical relationship between choice probabilities, read-out weights and noise correlations within the standard neural decision making model. Our solution allows us to prove and generalize earlier observations based on numerical simulations, and to derive novel predictions. Importantly, we show how the read-out weight profile, or decoding strategy, can be inferred from experimentally measurable quantities. Furthermore, we present a test to decide whether the decoding weights of individual neurons are optimal, even without knowing the underlying noise correlations. We confirm the practical feasibility of our approach using simulated data from a realistic population model. Our work thus provides the theoretical foundation for a growing body of experimental results on choice probabilities and correlations

Episode of unusual high solar ultraviolet radiation over central Europe due to dynamical reduced total ozone in May 2005

In late May 2005 unusual high levels of solar ultraviolet radiation were observed over central Europe. In Northern Germany the measured irradiance of erythemally effective radiation exceeded the climatological mean by more than about 20%. An extreme low ozone event for the season coincided with high solar elevation angles and high pressure induced clear sky conditions leading to the highest value of erythemal UV-radiation ever observed over this location in May since 1994. This hereafter called "ozone mini-hole" was caused by an elevation of tropopause height accompanied with a poleward advection of ozone-poor air from the tropics. The resultant increase in UV-radiation is of particular significance for human health. Dynamically induced low ozone episodes that happen in late spring can considerably enhance the solar UV-radiation in mid latitudes and therefore contribute to the UV-burden of people living in these regions

Signatures of criticality arise in simple neural population models with correlations

Large-scale recordings of neuronal activity make it possible to gain insights into the collective activity of neural ensembles. It has been hypothesized that neural populations might be optimized to operate at a 'thermodynamic critical point', and that this property has implications for information processing. Support for this notion has come from a series of studies which identified statistical signatures of criticality in the ensemble activity of retinal ganglion cells. What are the underlying mechanisms that give rise to these observations? Here we show that signatures of criticality arise even in simple feed-forward models of retinal population activity. In particular, they occur whenever neural population data exhibits correlations, and is randomly sub-sampled during data analysis. These results show that signatures of criticality are not necessarily indicative of an optimized coding strategy, and challenge the utility of analysis approaches based on equilibrium thermodynamics for understanding partially observed biological systems.Comment: 36 pages, LaTeX; added journal reference on page 1, added link to code repositor