Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

Uncovering tumour heterogeneity through PKR and nc886 analysis in metastatic colon cancer patients treated with 5-FU-based chemotherapy

Colorectal cancer treatment has advanced over the past decade. The drug 5-fluorouracil is still used with a wide percentage of patients who do not respond. Therefore, a challenge is the identification of predictive biomarkers. The protein kinase R (PKR also called EIF2AK2) and its regulator, the non-coding pre-mir-nc886, have multiple effects on cells in response to numerous types of stress, including chemotherapy. In this work, we performed an ambispective study with 197 metastatic colon cancer patients with unresectable metastases to determine the relative expression levels of both nc886 and PKR by qPCR, as well as the location of PKR by immunohistochemistry in tumour samples and healthy tissues (plasma and colon epithelium). As primary end point, the expression levels were related to the objective response to first-line chemotherapy following the response evaluation criteria in solid tumours (RECIST) and, as the second end point, with survival at 18 and 36 months. Hierarchical agglomerative clustering was performed to accommodate the heterogeneity and complexity of oncological patients\u27 data. High expression levels of nc886 were related to the response to treatment and allowed to identify clusters of patients. Although the PKR mRNA expression was not associated with chemotherapy response, the absence of PKR location in the nucleolus was correlated with first-line chemotherapy response. Moreover, a relationship between survival and the expression of both PKR and nc886 in healthy tissues was found. Therefore, this work evaluated the best way to analyse the potential biomarkers PKR and nc886 in order to establish clusters of patients depending on the cancer outcomes using algorithms for complex and heterogeneous data

Weight gain and dietary intake during pregnancy in industrialized countries - a systematic review of observational studies

Background: Gestational weight gain (GWG) above the recently recommended ranges is likely to be related to adverse pregnancy outcomes and therefore a challenge in industrialized countries. Aims: We conducted a systematic review on observational studies in order to gain more evidence on whether diets with lower caloric/protein content or other diets might be associated with lower GWG. Methods: We searched in MEDLINE and EMBASE for observational studies written in English or German reporting associations between diet and GWG in singleton pregnancies of healthy women in industrialized countries. Results: We identified 12 studies which met the inclusion criteria. Five studies suggested significant positive associations between energy intake and GWG, whereas three found no significant association. Further significant positive associations of GWG were reported with respect to protein intake, animal lipids, energy density and a number of different food servings per day, whereas intake of carbohydrates and vegetarian diet were associated with less GWG. Conclusions: We suggest that GWG might be reduced by lower energy intake in pregnancy

A Novel Methodology for Prediction Urban Water Demand by Wavelet Denoising and Adaptive Neuro-Fuzzy Inference System Approach

Accurate and reliable urban water demand prediction is imperative for providing the basis to design, operate, and manage water system, especially under the scarcity of the natural water resources. A new methodology combining discrete wavelet transform (DWT) with an adaptive neuro-fuzzy inference system (ANFIS) is proposed to predict monthly urban water demand based on several intervals of historical water consumption. This ANFIS model is evaluated against a hybrid crow search algorithm and artificial neural network (CSA-ANN), since these methods have been successfully used recently to tackle a range of engineering optimization problems. The study outcomes reveal that 1) data preprocessing is essential for denoising raw time series and choosing the model inputs to render the highest model performance; 2) both methodologies, ANFIS and CSA-ANN, are statistically equivalent and capable of accurately predicting monthly urban water demand with high accuracy based on several statistical metric measures such as coefficient of efficiency (0.974, 0.971, respectively). This study could help policymakers to manage extensions of urban water system in response to the increasing demand with low risk related to a decision

Urban Water Demand Prediction for a City that Suffers from Climate Change and Population Growth: Gauteng Province case study

The proper management of municipal water system is essential to sustain cities and support water security of societies. Urban water estimating has always been a challenging task for managers of water utilities and policymakers. This paper applies a novel methodology that includes data pre-processing and Artificial Neural Network (ANN) optimized with Backtracking Search Algorithm (BSA-ANN) to estimate monthly water demand in relation to previous water consumption. Historical data of monthly water consumption in the Gauteng Province, South Africa, for the period 2007–2016, were selected for the creation and evaluation of the methodology. Data pre-processing techniques played a crucial role in the enhancing of the quality of the data before creating the prediction model. The BSA-ANN model yielded the best result with a root mean square error and a coefficient of efficiency of 0.0099 mega liters and 0.979, respectively. Also, it proved more efficient and reliable than the Crow Search Algorithm (CSA-ANN), based on the scale of error. Overall, this paper presents a new application for the hybrid model BSA-ANN that can be successfully used to predict water demand with high accuracy, in a city that heavily suffers from the impact of climate change and population growth

Utilizing an enhanced cellular automata model for data mining

Data mining deals with clustering and classifying large amounts of data, in order to discover new knowledge from the existent data by identifying correlations and relationships between various data-sets. Cellular automata have been used before for classification purposes. This paper presents a cellular automata enhanced classification algorithm for data mining. Experimental results show that the proposed enhancement gives better performance in terms of accuracy and execution time than previous work using cellular automata