Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

A Framework for Generalising the Newton Method and Other Iterative Methods from Euclidean Space to Manifolds

The Newton iteration is a popular method for minimising a cost function on Euclidean space. Various generalisations to cost functions defined on manifolds appear in the literature. In each case, the convergence rate of the generalised Newton iteration needed establishing from first principles. The present paper presents a framework for generalising iterative methods from Euclidean space to manifolds that ensures local convergence rates are preserved. It applies to any (memoryless) iterative method computing a coordinate independent property of a function (such as a zero or a local minimum). All possible Newton methods on manifolds are believed to come under this framework. Changes of coordinates, and not any Riemannian structure, are shown to play a natural role in lifting the Newton method to a manifold. The framework also gives new insight into the design of Newton methods in general.Comment: 36 page

Thermodynamics as a theory of decision-making with information processing costs

Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we propose an information-theoretic formalization of bounded rational decision-making where decision-makers trade off expected utility and information processing costs. Such bounded rational decision-makers can be thought of as thermodynamic machines that undergo physical state changes when they compute. Their behavior is governed by a free energy functional that trades off changes in internal energy-as a proxy for utility-and entropic changes representing computational costs induced by changing states. As a result, the bounded rational decision-making problem can be rephrased in terms of well-known concepts from statistical physics. In the limit when computational costs are ignored, the maximum expected utility principle is recovered. We discuss the relation to satisficing decision-making procedures as well as links to existing theoretical frameworks and human decision-making experiments that describe deviations from expected utility theory. Since most of the mathematical machinery can be borrowed from statistical physics, the main contribution is to axiomatically derive and interpret the thermodynamic free energy as a model of bounded rational decision-making.Comment: 26 pages, 5 figures, (under revision since February 2012

Dyslexia detection from EEG signals using SSA component correlation and Convolutional Neural Networks

Objective dyslexia diagnosis is not a straighforward task since it is traditionally performed by means of the intepretation of different behavioural tests. Moreover, these tests are only applicable to readers. This way, early diagnosis requires the use of specific tasks not only related to reading. Thus, the use of Electroencephalography (EEG) constitutes an alternative for an objective and early diagnosis that can be used with pre-readers. In this way, the extraction of relevant features in EEG signals results crucial for classification. However, the identification of the most relevant features is not straighforward, and predefined statistics in the time or frequency domain are not always discriminant enough. On the other hand, classical processing of EEG signals based on extracting EEG bands frequency descriptors, usually make some assumptions on the raw signals that could cause indormation loosing. In this work we propose an alternative for analysis in the frequency domain based on Singluar Spectrum Analysis (SSA) to split the raw signal into components representing different oscillatory modes. Moreover, correlation matrices obtained for each component among EEG channels are classfied using a Convolutional Neural network.Comment: 11 pages, 7 figures. Submitted to conferenc

Identifying Variability in Process Performance Indicators

The performance perspective of business processes is concerned with the definition of performance requirements usually specified as a set of Process Performance Indicators (PPIs). Like other business process perspectives such as control-flow or data, there are cases in which PPIs are subject to variability. However, although the modelling of business process variability (BPV) has evolved significantly, there are very few contributions addressing the variability in the performance perspective of business processes. Modelling PPI variants with tools and techniques non-suitable for variability may generate redundant models, thus making it difficult its maintenance and future adaptations, also increasing possibility of errors in its managing. In this paper we present different cases of PPI variability detected as result of the analysis of several processes where BPV is present. Based on an existent metamodel used for defining PPIs over BPs, we propose its formal extension that allows the definition of PPI variability according to the cases identified.Ministerio de Economía y Competitividad TIN2015-70560-RJunta de Andalucía P12-TIC-1867Junta de Andalucía P10-TIC-590

Neurogenesis Drives Stimulus Decorrelation in a Model of the Olfactory Bulb

The reshaping and decorrelation of similar activity patterns by neuronal networks can enhance their discriminability, storage, and retrieval. How can such networks learn to decorrelate new complex patterns, as they arise in the olfactory system? Using a computational network model for the dominant neural populations of the olfactory bulb we show that fundamental aspects of the adult neurogenesis observed in the olfactory bulb -- the persistent addition of new inhibitory granule cells to the network, their activity-dependent survival, and the reciprocal character of their synapses with the principal mitral cells -- are sufficient to restructure the network and to alter its encoding of odor stimuli adaptively so as to reduce the correlations between the bulbar representations of similar stimuli. The decorrelation is quite robust with respect to various types of perturbations of the reciprocity. The model parsimoniously captures the experimentally observed role of neurogenesis in perceptual learning and the enhanced response of young granule cells to novel stimuli. Moreover, it makes specific predictions for the type of odor enrichment that should be effective in enhancing the ability of animals to discriminate similar odor mixtures

Extracellular Hsp72 concentration relates to a minimum endogenous criteria during acute exercise-heat exposure

Extracellular heat-shock protein 72 (eHsp72) concentration increases during exercise-heat stress when conditions elicit physiological strain. Differences in severity of environmental and exercise stimuli have elicited varied response to stress. The present study aimed to quantify the extent of increased eHsp72 with increased exogenous heat stress, and determine related endogenous markers of strain in an exercise-heat model. Ten males cycled for 90 min at 50% O2peak in three conditions (TEMP, 20°C/63% RH; HOT, 30.2°C/51%RH; VHOT, 40.0°C/37%RH). Plasma was analysed for eHsp72 pre, immediately post and 24-h post each trial utilising a commercially available ELISA. Increased eHsp72 concentration was observed post VHOT trial (+172.4%) (P<0.05), but not TEMP (-1.9%) or HOT (+25.7%) conditions. eHsp72 returned to baseline values within 24hrs in all conditions. Changes were observed in rectal temperature (Trec), rate of Trec increase, area under the curve for Trec of 38.5°C and 39.0°C, duration Trec ≥ 38.5°C and ≥ 39.0°C, and change in muscle temperature, between VHOT, and TEMP and HOT, but not between TEMP and HOT. Each condition also elicited significantly increasing physiological strain, described by sweat rate, heart rate, physiological strain index, rating of perceived exertion and thermal sensation. Stepwise multiple regression reported rate of Trec increase and change in Trec to be predictors of increased eHsp72 concentration. Data suggests eHsp72 concentration increases once systemic temperature and sympathetic activity exceeds a minimum endogenous criteria elicited during VHOT conditions and is likely to be modulated by large, rapid changes in core temperature

Spatial heterogeneity of habitat suitability for Rift Valley fever occurrence in Tanzania: an ecological niche modelling approach

Despite the long history of Rift Valley fever (RVF) in Tanzania, extent of its suitable habitat in the country remains unclear. In this study we investigated potential effects of temperature, precipitation, elevation, soil type, livestock density, rainfall pattern, proximity to wild animals, protected areas and forest on the habitat suitability for RVF occurrence in Tanzania. Presence-only records of 193 RVF outbreak locations from 1930 to 2007 together with potential predictor variables were used to model and map the suitable habitats for RVF occurrence using ecological niche modelling. Ground-truthing of the model outputs was conducted by comparing the levels of RVF virus specific antibodies in cattle, sheep and goats sampled from locations in Tanzania that presented different predicted habitat suitability values. Habitat suitability values for RVF occurrence were higher in the northern and central-eastern regions of Tanzania than the rest of the regions in the country. Soil type and precipitation of the wettest quarter contributed equally to habitat suitability (32.4% each), followed by livestock density (25.9%) and rainfall pattern (9.3%). Ground-truthing of model outputs revealed that the odds of an animal being seropositive for RVFV when sampled from areas predicted to be most suitable for RVF occurrence were twice the odds of an animal sampled from areas least suitable for RVF occurrence (95% CI: 1.43, 2.76, p < 0.001). The regions in the northern and central-eastern Tanzania were more suitable for RVF occurrence than the rest of the regions in the country. The modelled suitable habitat is characterised by impermeable soils, moderate precipitation in the wettest quarter, high livestock density and a bimodal rainfall pattern. The findings of this study should provide guidance for the design of appropriate RVF surveillance, prevention and control strategies which target areas with these characteristics