Estimating the Maximum Expected Value: An Analysis of (Nested) Cross Validation and the Maximum Sample Average

We investigate the accuracy of the two most common estimators for the maximum expected value of a general set of random variables: a generalization of the maximum sample average, and cross validation. No unbiased estimator exists and we show that it is non-trivial to select a good estimator without knowledge about the distributions of the random variables. We investigate and bound the bias and variance of the aforementioned estimators and prove consistency. The variance of cross validation can be significantly reduced, but not without risking a large bias. The bias and variance of different variants of cross validation are shown to be very problem-dependent, and a wrong choice can lead to very inaccurate estimates

Bayesian Sampling Algorithms for the Sample Selection and Two-Part Models

This paper considers two models to deal with an outcome variable that contains a large fraction of zeros, such as individual expenditures on health care: a sample-selection model and a two-part model. The sample-selection model uses two possibly correlated processes to determine the outcome: a decision process and an outcome process; conditional on a favorable decision, the outcome is observed. The two-part model comprises uncorrelated decision and outcome processes. The paper addresses the issue of selecting between these two models. With a Gaussian specification of the likelihood, the models are nested and inference can focus on the correlation coefficient. Using a fully parametric Bayesian approach, I present sampling algorithms for the model parameters that are based on data augmentation. In addition to the sampler output of the correlation coefficient, a Bayes factor can be computed to distinguish between the models. The paper illustrates the methods and their potential pitfalls using simulated data setsSample Selection, Data Augmentation, Gibbs Sampling

Deep Reinforcement Learning with Double Q-learning

The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.Comment: AAAI 201

Individual differences in maternal care as a predictor for phenotypic variation later in life

Vroege levenservaringen hebben een grote invloed op hoe we later in het leven functioneren. Studies hebben aangetoond dat verstoringen van de leefomgeving in de jeugd de kans op het ontwikkelen van allerlei aandoeningen in volwassenheid vergroten. Met behulp van een model met ratten, waarbij wordt uitgegaan van een natuurlijke variatie in de hoeveelheid moederzorg die optreedt in een populatie, onderzocht Felisa van Hasselt wat het effect is van verschillen in ontvangen moederzorg op de karakteristieken van een dier later in zijn leven. De hoeveelheid moederzorg verschilt niet alleen sterk tussen verschillende nesten, maar vertoont ook een behoorlijke variatie tussen individuele pups binnen elk nest. Deze individuele verschillen in moederzorg correleren direct met structurele en functionele parameters in de volwassen hippocampus, een hersengebied dat betrokken is bij leren en geheugen en dat erg gevoelig is voor stress. Ook de expressie van bepaalde genen die bij deze processen betrokken zijn lijkt gerelateerd te zijn aan de hoeveelheid ontvangen moederzorg. Ten slotte bleek zowel speelgedrag als keuzegedrag, maar nauwelijks de prestatie in hippocampus-afhankelijke leertaken, samen te hangen met de hoeveelheid moederzorg die een dier vlak na de geboorte ontving. Voor sommige van bovenstaande parameters bleken de effecten op mannetjes en vrouwtjes verschillend

What doctors should look for in patients presenting with erectile dysfunction

Click on the link to view the commentary.S Afr Psychiatry Rev 2003;6:29-3

Studies on sleep patterns and sleep homeostasis in birds:An ecological approach

Sleep is a complex phenomenon that consists of two completely different and alternating states, slow-wave sleep (SWS) and rapid-eye-movement sleep (REM sleep). Each of these two states is thought to play an important role in supporting brain and bodily functions. Yet, how exactly sleep fulfills these functions is a topic of ongoing research and debate. Most of what is known about sleep is derived from studies that were done in mammals under strictly controlled laboratory conditions. However, sleep is not restricted to mammals but is thought to be present in all living animals. Moreover, studies in a laboratory setting may not provide a complete picture of the regulatory processes and functions of sleep under natural conditions. For that reason, I measured sleep in three bird species under both laboratory conditions and semi-natural conditions: the European jackdaw (Coloeus monedula), the European starling (Sturnus vulgaris) and the barnacle goose (Branta leucopsis). The results provide evidence for homeostatic regulation of SWS in birds similar to what has been reported for mammals, but also produced unexpected findings. For example, the geese only showed a rebound of SWS after brief sleep deprivation in summer but not in winter. Also, both geese and starlings displayed strong seasonal variation in the overall amount of sleep. The starling in particular slept 5h per day less in summer than they did in winter. Moreover, both geese and starlings slept about 2h less during full moon nights than new moon nights. Another intriguing finding was the strong variation in REM sleep between the 3 species, which ranged from hardly any REM sleep in starlings to a much higher, mammalian-like amount of REM sleep in jackdaws. Such findings are difficult to reconcile with current theories in the function of REM sleep that are largely based on studies in mammals. Together, these findings in birds indicate that sleep is highly sensitive to environmental factors and suggest a great deal of flexibility in the regulation of sleep under natural conditions