Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

Modelling Transmission of Vector-Borne Pathogens Shows Complex Dynamics When Vector Feeding Sites Are Limited

The relationship between species richness and the prevalence of vector-borne disease has been widely studied with a range of outcomes. Increasing the number of host species for a pathogen may decrease infection prevalence (dilution effect), increase it (amplification), or have no effect. We derive a general model, and a specific implementation, which show that when the number of vector feeding sites on each host is limiting, the effects on pathogen dynamics of host population size are more complex than previously thought. The model examines vector-borne disease in the presence of different host species that are either competent or incompetent (i.e. that cannot transmit the pathogen to vectors) as reservoirs for the pathogen. With a single host species present, the basic reproduction ratio R0 is a non-monotonic function of the population size of host individuals (H), i.e. a value exists that maximises R0. Surprisingly, if a reduction in host population size may actually increase R0. Extending this model to a two-host species system, incompetent individuals from the second host species can alter the value of which may reverse the effect on pathogen prevalence of host population reduction. We argue that when vector-feeding sites on hosts are limiting, the net effect of increasing host diversity might not be correctly predicted using simple frequency-dependent epidemiological models

Pathogenesis of Scleroderma

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/65784/1/j.1365-4362.1984.tb05678.x.pd

Utilisation of an operative difficulty grading scale for laparoscopic cholecystectomy

Background A reliable system for grading operative difficulty of laparoscopic cholecystectomy would standardise description of findings and reporting of outcomes. The aim of this study was to validate a difficulty grading system (Nassar scale), testing its applicability and consistency in two large prospective datasets. Methods Patient and disease-related variables and 30-day outcomes were identified in two prospective cholecystectomy databases: the multi-centre prospective cohort of 8820 patients from the recent CholeS Study and the single-surgeon series containing 4089 patients. Operative data and patient outcomes were correlated with Nassar operative difficultly scale, using Kendall’s tau for dichotomous variables, or Jonckheere–Terpstra tests for continuous variables. A ROC curve analysis was performed, to quantify the predictive accuracy of the scale for each outcome, with continuous outcomes dichotomised, prior to analysis. Results A higher operative difficulty grade was consistently associated with worse outcomes for the patients in both the reference and CholeS cohorts. The median length of stay increased from 0 to 4 days, and the 30-day complication rate from 7.6 to 24.4% as the difficulty grade increased from 1 to 4/5 (both p < 0.001). In the CholeS cohort, a higher difficulty grade was found to be most strongly associated with conversion to open and 30-day mortality (AUROC = 0.903, 0.822, respectively). On multivariable analysis, the Nassar operative difficultly scale was found to be a significant independent predictor of operative duration, conversion to open surgery, 30-day complications and 30-day reintervention (all p < 0.001). Conclusion We have shown that an operative difficulty scale can standardise the description of operative findings by multiple grades of surgeons to facilitate audit, training assessment and research. It provides a tool for reporting operative findings, disease severity and technical difficulty and can be utilised in future research to reliably compare outcomes according to case mix and intra-operative difficulty

The acute mania of King George III: A computational linguistic analysis.

We used a computational linguistic approach, exploiting machine learning techniques, to examine the letters written by King George III during mentally healthy and apparently mentally ill periods of his life. The aims of the study were: first, to establish the existence of alterations in the King's written language at the onset of his first manic episode; and secondly to identify salient sources of variation contributing to the changes. Effects on language were sought in two control conditions (politically stressful vs. politically tranquil periods and seasonal variation). We found clear differences in the letter corpus, across a range of different features, in association with the onset of mental derangement, which were driven by a combination of linguistic and information theory features that appeared to be specific to the contrast between acute mania and mental stability. The paucity of existing data relevant to changes in written language in the presence of acute mania suggests that lexical, syntactic and stylometric descriptions of written discourse produced by a cohort of patients with a diagnosis of acute mania will be necessary to support the diagnosis independently and to look for other periods of mental illness of the course of the King's life, and in other historically significant figures with similarly large archives of handwritten documents

Ce-Duox1/BLI-3 Generated Reactive Oxygen Species Trigger Protective SKN-1 Activity via p38 MAPK Signaling during Infection in C. elegans

Infected animals will produce reactive oxygen species (ROS) and other inflammatory molecules that help fight pathogens, but can inadvertently damage host tissue. Therefore specific responses, which protect and repair against the collateral damage caused by the immune response, are critical for successfully surviving pathogen attack. We previously demonstrated that ROS are generated during infection in the model host Caenorhabditis elegans by the dual oxidase Ce-Duox1/BLI-3. Herein, an important connection between ROS generation by Ce-Duox1/BLI-3 and upregulation of a protective transcriptional response by SKN-1 is established in the context of infection. SKN-1 is an ortholog of the mammalian Nrf transcription factors and has previously been documented to promote survival, following oxidative stress, by upregulating genes involved in the detoxification of ROS and other reactive compounds. Using qRT-PCR, transcriptional reporter fusions, and a translational fusion, SKN-1 is shown to become highly active in the C. elegans intestine upon exposure to the human bacterial pathogens, Enterococcus faecalis and Pseudomonas aeruginosa. Activation is dependent on the overall pathogenicity of the bacterium, demonstrated by a weakened response observed in attenuated mutants of these pathogens. Previous work demonstrated a role for p38 MAPK signaling both in pathogen resistance and in activating SKN-1 upon exposure to chemically induced oxidative stress. We show that NSY-1, SEK-1 and PMK-1 are also required for SKN-1 activity during infection. Evidence is also presented that the ROS produced by Ce-Duox1/BLI-3 is the source of SKN-1 activation via p38 MAPK signaling during infection. Finally, for the first time, SKN-1 activity is shown to be protective during infection; loss of skn-1 decreases resistance, whereas increasing SKN-1 activity augments resistance to pathogen. Overall, a model is presented in which ROS generation by Ce-Duox1/BLI-3 activates a protective SKN-1 response via p38 MAPK signaling

An assessment of monitoring requirements and costs of 'Reduced Emissions from Deforestation and Degradation'

<p>Abstract</p> <p>Background</p> <p>Negotiations on a future climate policy framework addressing Reduced Emissions from Deforestation and Degradation (REDD) are ongoing. Regardless of how such a framework will be designed, many technical solutions of estimating forest cover and forest carbon stock change exist to support policy in monitoring and accounting. These technologies typically combine remotely sensed data with ground-based inventories. In this article we assess the costs of monitoring REDD based on available technologies and requirements associated with key elements of REDD policy.</p> <p>Results</p> <p>We find that the design of a REDD policy framework (and specifically its rules) can have a significant impact on monitoring costs. Costs may vary from 0.5 to 550 US$ per square kilometre depending on the required precision of carbon stock and area change detection. Moreover, they follow economies of scale, i.e. single country or project solutions will face relatively higher monitoring costs.</p> <p>Conclusion</p> <p>Although monitoring costs are relatively small compared to other cost items within a REDD system, they should be shared not only among countries but also among sectors, because an integrated monitoring system would have multiple benefits for non-REDD management. Overcoming initialization costs and unequal access to monitoring technologies is crucial for implementation of an integrated monitoring system, and demands for international cooperation.</p

Analysis of the effects of exposure to acute hypoxia on oxidative lesions and tumour progression in a transgenic mouse breast cancer model

<p>Abstract</p> <p>Background</p> <p>Tumour hypoxia is known to be a poor prognostic indicator, predictive of increased risk of metastatic disease and reduced survival. Genomic instability has been proposed as one of the potential mechanisms for hypoxic tumour progression. Both of these features are commonly found in many cancer types, but their relationship and association with tumour progression has not been examined in the same model.</p> <p>Methods</p> <p>To address this issue, we determined the effects of 6 week <it>in vivo </it>acute hypoxic exposure on the levels of mutagenic lipid peroxidation product, malondialdehyde, and 8-oxo-7,8-dihydro-2'-deoxyguanosine DNA (8-oxo-dG) lesions in the transgenic polyomavirus middle T (PyMT) breast cancer mouse model.</p> <p>Results</p> <p>We observed significantly increased plasma lipid peroxidation and 8-oxo-dG lesion levels in the hypoxia-exposed mice. Consumption of malondialdehyde also induced a significant increase in the PyMT tumour DNA lesion levels, however, these increases did not translate into enhanced tumour progression. We further showed that the <it>in vivo </it>exposure to acute hypoxia induced accumulation of F4/80 positive tumour-associated macrophages (TAMs), demonstrating a relationship between hypoxia and macrophages in an experimental model.</p> <p>Conclusion</p> <p>These data suggest that although exposure to acute hypoxia causes an increase in 8-oxo-dG lesions and TAMs in the PyMT tumours, these increases do not translate into significant changes in tumour progression at the primary or metastatic levels in this strong viral oncogene-driven breast cancer model.</p