Performance Enhancement of Deep Reinforcement Learning Networks using Feature Extraction

The combination of Deep Learning and Reinforcement Learning, termed Deep Reinforcement Learning Networks (DRLN), offers the possibility of using a Deep Learning Neural Network to produce an approximate Reinforcement Learning value table that allows extraction of features from neurons in the hidden layers of the network. This paper presents a two stage technique for training a DRLN on features extracted from a DRLN trained on a identical problem, via the implementation of the Q-Learning algorithm, using TensorFlow. The results show that the extraction of features from the hidden layers of the Deep Q-Network improves the learning process of the agent (4.58 times faster and better) and proves the existence of encoded information about the environment which can be used to select the best action. The research contributes preliminary work in an ongoing research project in modeling features extracted from DRLNs

Towards the spectral mapping of plastic debris on beaches

Floating and washed ashore marine plastic debris (MPD) is a growing environmental challenge. It has become evident that secluded locations including the Arctic, Antarctic, and remote islands are being impacted by plastic pollution generated thousands of kilometers away. Optical remote sensing of MPD is an emerging field that can aid in monitoring remote environments where in-person observation and data collection is not always feasible. Here we evaluate MPD spectral features in the visible to shortwave infrared regions for detecting varying quantities of MPD that have accumulated on beaches using a spectroradiometer. Measurements were taken from a range of in situ MPD accumulations ranging from 0.08% to 7.94% surface coverage. Our results suggest that spectral absorption features at 1215 nm and 1732 nm are useful for detecting varying abundance levels of MPD in a complex natural environment, however other absorption features at 931 nm, 1045 nm and 2046 nm could not detect in situ MPD. The reflectance of some in situ MPD accumulations was statistically different from samples that only contained organic debris and sand between 1.56% and 7.94% surface cover; however other samples with similar surface cover did not have reflectance that was statistically different from samples containing no MPD. Despite MPD being detectable against a background of sand and organic beach debris, a clear relationship between the surface cover of MPD and the strength of key absorption features could not be established. Additional research is needed to advance our understanding of the factors, such as type of MPD assemblage, that contribute to the bulk reflectance of MPD contaminated landscapes

Language and social/emotional problems identified at a universal developmental assessment at 30 months

Non peer reviewedPublisher PD

A Typology of Digital Sharing Business Models: A Design Science Research Approach

The digitally enabled sharing economy, also called the “digital sharing economy” (DSE), has changed patterns of consumption by introducing new choices and channels for provision and receipt of services. The DSE encompasses sharing systems whose business models may vary distinctly from platform to platform. Although business models in the context of the sharing economy have been studied so far, we have observed that the current literature does not provide an approach that covers all the possible business models (in the broadest sense of the term) that (potentially) exist within the scope of the DSE. The present paper, therefore, aims to propose a typology of business models in the DSE that covers a wide space of models – even those which may not involve “business” in the commercial sense. This is achieved through an iterative inductive process based on a design science research approach. The typology can assist in positioning the current and future sharing systems in the DSE by systematically classifying their business models. It is intended to serve as a guiding tool for the sustainability assessment of platforms from both resource and socio-economic perspectives. The present study can also enable researchers and practitioners to capture and systematically analyse digital sharing business models based on a structured, actionable approach

Active learning and optimal climate policy

This paper develops a climate-economy model with uncertainty, irreversibility, and active learning. Whereas previous papers assume learning from one observation per period, or experiment with control variables to gain additional information, this paper considers active learning from investment in monitoring, specifically in improved observations of the global mean temperature. We find that the decision maker invests a significant amount of money in climate research, far more than the current level, in order to increase the rate of learning about climate change. This helps the decision maker make improved decisions. The level of uncertainty decreases more rapidly in the active learning model than in the passive learning model with only temperature observations. As the uncertainty about climate change is smaller, active learning reduces the optimal carbon tax. The greater the risk, the larger is the effect of learning. The method proposed here is applicable to any dynamic control problem where the quality of monitoring is a choice variable, for instance, the precision at which we observe GDP, unemployment, or the quality of education

ATHENA: A knowledge-based hybrid backpropagation-grammatical evolution neural network algorithm for discovering epistasis among quantitative trait Loci

<p>Abstract</p> <p>Background</p> <p>Growing interest and burgeoning technology for discovering genetic mechanisms that influence disease processes have ushered in a flood of genetic association studies over the last decade, yet little heritability in highly studied complex traits has been explained by genetic variation. Non-additive gene-gene interactions, which are not often explored, are thought to be one source of this "missing" heritability.</p> <p>Methods</p> <p>Stochastic methods employing evolutionary algorithms have demonstrated promise in being able to detect and model gene-gene and gene-environment interactions that influence human traits. Here we demonstrate modifications to a neural network algorithm in ATHENA (the Analysis Tool for Heritable and Environmental Network Associations) resulting in clear performance improvements for discovering gene-gene interactions that influence human traits. We employed an alternative tree-based crossover, backpropagation for locally fitting neural network weights, and incorporation of domain knowledge obtainable from publicly accessible biological databases for initializing the search for gene-gene interactions. We tested these modifications <it>in silico </it>using simulated datasets.</p> <p>Results</p> <p>We show that the alternative tree-based crossover modification resulted in a modest increase in the sensitivity of the ATHENA algorithm for discovering gene-gene interactions. The performance increase was highly statistically significant when backpropagation was used to locally fit NN weights. We also demonstrate that using domain knowledge to initialize the search for gene-gene interactions results in a large performance increase, especially when the search space is larger than the search coverage.</p> <p>Conclusions</p> <p>We show that a hybrid optimization procedure, alternative crossover strategies, and incorporation of domain knowledge from publicly available biological databases can result in marked increases in sensitivity and performance of the ATHENA algorithm for detecting and modelling gene-gene interactions that influence a complex human trait.</p

What Is True Halving in the Payoff Matrix of Game Theory?

In game theory, there are two social interpretations of rewards (payoffs) for decision-making strategies: (1) the interpretation based on the utility criterion derived from expected utility theory and (2) the interpretation based on the quantitative criterion (amount of gain) derived from validity in the empirical context. A dynamic decision theory has recently been developed in which dynamic utility is a conditional (state) variable that is a function of the current wealth of a decision maker. We applied dynamic utility to the equal division in dove-dove contests in the hawk-dove game. Our results indicate that under the utility criterion, the half-share of utility becomes proportional to a player’s current wealth. Our results are consistent with studies of the sense of fairness in animals, which indicate that the quantitative criterion has greater validity than the utility criterion. We also find that traditional analyses of repeated games must be reevaluated

Large-scale unit commitment under uncertainty: an updated literature survey

The Unit Commitment problem in energy management aims at finding the optimal production schedule of a set of generation units, while meeting various system-wide constraints. It has always been a large-scale, non-convex, difficult problem, especially in view of the fact that, due to operational requirements, it has to be solved in an unreasonably small time for its size. Recently, growing renewable energy shares have strongly increased the level of uncertainty in the system, making the (ideal) Unit Commitment model a large-scale, non-convex and uncertain (stochastic, robust, chance-constrained) program. We provide a survey of the literature on methods for the Uncertain Unit Commitment problem, in all its variants. We start with a review of the main contributions on solution methods for the deterministic versions of the problem, focussing on those based on mathematical programming techniques that are more relevant for the uncertain versions of the problem. We then present and categorize the approaches to the latter, while providing entry points to the relevant literature on optimization under uncertainty. This is an updated version of the paper "Large-scale Unit Commitment under uncertainty: a literature survey" that appeared in 4OR 13(2), 115--171 (2015); this version has over 170 more citations, most of which appeared in the last three years, proving how fast the literature on uncertain Unit Commitment evolves, and therefore the interest in this subject