Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

A screen to identify drug resistant variants to target-directed anti-cancer agents

The discovery of oncogenes and signal transduction pathways important for mitogenesis has triggered the development of target-specific small molecule anti-cancer compounds. As exemplified by imatinib (Gleevec), a specific inhibitor of the Chronic Myeloid Leukemia (CML)-associated Bcr-Abl kinase, these agents promise impressive activity in clinical trials, with low levels of clinical toxicity. However, such therapy is susceptible to the emergence of drug resistance due to amino acid substitutions in the target protein. Defining the spectrum of such mutations is important for patient monitoring and the design of next-generation inhibitors. Using imatinib and BCR/ABL as a paradigm for a drug-target pair, we recently reported a retroviral vector-based screening strategy to identify the spectrum of resistance-conferring mutations. Here we provide a detailed methodology for the screen, which can be generally applied to any drug-target pair

Genetic dissection of quantitative and qualitative traits using a minimum set of barley Recombinant Chromosome Substitution Lines

BACKGROUND:Exploring the natural occurring genetic variation of the wild barley genepool has become a major target of barley crop breeding programmes aiming to increase crop productivity and sustainability in global climate change scenarios. However this diversity remains unexploited and effective approaches are required to investigate the benefits that unadapted genomes could bring to crop improved resilience. In the present study, a set of Recombinant Chromosome Substitution Lines (RCSLs) derived from an elite barley cultivar 'Harrington' as the recurrent parent, and a wild barley accession from the Fertile Crescent 'Caesarea 26-24', as the donor parent (Matus et al. Genome 46:1010-23, 2003) have been utilised in field and controlled conditions to examine the contribution of wild barley genome as a source of novel allelic variation for the cultivated barley genepool. METHODS:Twenty-eight RCSLs which were selected to represent the entire genome of the wild barley accession, were genotyped using the 9 K iSelect SNP markers (Comadran et al. Nat Genet 44:1388-92, 2012) and phenotyped for a range of morphological, developmental and agronomic traits in 2 years using a rain-out shelter with four replicates and three water treatments. Data were analysed for marker traits associations using a mixed model approach. RESULTS:We identified lines that differ significantly from the elite parent for both qualitative and quantitative traits across growing seasons and water regimes. The detailed genotypic characterisation of the lines for over 1800 polymorphic SNP markers and the design of a mixed model analysis identified chromosomal regions associated with yield related traits where the wild barley allele had a positive response increasing grain weight and size. In addition, variation for qualitative characters, such as the presence of cuticle waxes on the developing spikes, was associated with the wild barley introgressions. Despite the coarse location of the QTLs, interesting candidate genes for the major marker-trait associations were identified using the recently released barley genome assembly. CONCLUSION:This study has highlighted the role of exotic germplasm to contribute novel allelic variation by using an optimised experimental approach focused on an exotic genetic library. The results obtained constitute a step forward to the development of more tolerant and resilient varieties.Carla De la Fuente Cantó, Joanne Russell, Christine A. Hackett, Allan Booth, Siobhan Dancey, Timothy S. George and Robbie Waug

Footprints- In the Footprints of Squier and Davis: Archeological Fieldwork in Ross County, Ohio

TABLE OF CONTENTS Acknowledgements....................................................................iList of Tables...................................................................vii List of Figures...................................................................ix Contributors .........................................................................xiii Chapter 1 In the Footprints of Squier and Davis : Archeological Fieldwork in Ross County, Ohio Mark J. Lynott................................................................................1 Previous Studies in Ohio ...................................................1 Hopewell Studies Today ................................................................4 Recent Field Research ..........................................................................6 Goal of this Volume ................................................................................12 Chapter 2 In Non-mound Space at the Hopewell Mound Group Jennifer Pederson Weinberger .........................................13 Geophysical Survey .......................................................15Western Village Area.....................................................................16 Near the East Village .....................................................................18 Central Area ............................................................................19 Discussion .......................................................................................19 Chapter 3 Field Studies of the Octagon and Great Circle, High Bank Earthworks Ross County, Ohio N’omi B. Greber and Orrin C. Shane III ..............................................23 Excavations at the Octagon ............................................................25 Geophysics at the Great Circle ............................................................30 Excavations at the Great Circle ..........................................................33 Radiocarbon Assays ...........................................................................41 Comments ....................................................................................44 Addendum ..........................................................................................46 Chapter 4 Spruce Hill Earthworks: The 1995-1996 National Park Service Investigations Bret J. Ruby .............................................................................................49 Background .......................................................................................49 The Spruce Hill Earthworks .......................................................................49 Spruce Hill Revisited ......................................................................53 The 1995-1996 National Park Service Investigations ............................53 Discussion and Conclusions ..............................................................61 Chapter 5 Falling Through a Crack in the Core: The Surprise and Demise of Anderson Earthwork William H. Pickard and Jeffrey W. Weinberger ........................................ 67 History and Setting ..................................................................................68 1993 Excavations ..................................................................................70 Discussion ..............................................................................................72 Conclusion ............................................................................................74 Chapter 6 Middle Woodland and Other Settlement Remains in the Overly Tract Near The Hopeton Earthwork, Ross County, Ohio William S. Dancey .....................................................................................................77 Research Design.............................................................................77 Artifact Categories and their Distributions..............................................80 Discussion............................................................................................92 Chapter 7 Hopewell Occupation at the Hopeton Earthworks: Large Scale Surface Survey Using GPS Technology Jarrod Burks and Dawn Walter Gagliano................................................97 The Survey Area ................................................................................98 Survey Methodology .............................................................................99 Survey Results ...................................................................................99 The Surface Data: A Siteless Approach .................................................103 Discussion and Conclusion .........................................................................106 Notes ...................................................................................................107 Chapter 8 Hopewellian Centers in Context: Investigations In and Around the Hopeton Earthworks Bret J. Ruby And Mark J. Lynott .................................................................109 Surface Survey ......................................................................................110 Redwing Site ...........................................................................................111 Comparisons .........................................................................................118 Conclusions ......................................................................................122 Chapter 9 Searching for Hopewell Settlements: The Triangle Site at the Hopeton Earthworks Mark Lynott ...................................................................................... 125 Field Investigations ............................................................................127 Geophysical Survey ............................................................................128 1998 Season ....................................................................................128 Features, Artifacts and Radiocarbon Dating ........................................130 Animal Remains ...............................................................................137 Plant Remains ................................................................................138 Interpretations ..............................................................................139 Chapter 10 Geophysical Investigations at the Hopeton Earthworks John Weymouth, Bruce Bevan, and Rinita Dalan ...............................145 The Cesium Gradiometer Survey ...........................................................146 Cesium Gradiometer Results ...................................................................146 Geoscan Instrument Surveys ..............................................................148 Comparison of Cesium Magnetic and Resistance Data ........................148 Small Circles...........................................................................................149 Trench Excavations ...................................................................................149 Discussion ................................................................................................152 Conclusions ......................................................................................157 Chapter 11 Archeological and Geoarcheological Study of the Rectangular Enclosure at the Hopeton Works Mark J. Lynott and Rolfe D. Mandel ......................................................159 The Study of the Rectangular Earthwork ................................................161 Geophysical Survey ..............................................................................163 Trench Excavations .............................................................................164 Chronology ........................................................................................170 Geoarcheological Analysis of Trench 1 .................................................172 Interpretations ................................................................................174 Chapter 12 Ohio Hopewell Ritual Craft Production Katherine A. Spielmann ............................................................ 179Raw Material Procurement ....................................................................180 Hopewell Crafting ................................................................................181 Deposition ...........................................................................................186 Conclusions ....................................................................................188 References Cited .....................................................................................................18

Shared mental models and intra-team psychophysiological patterns: A test of the juggling paradigm

We explored implicit coordination mechanisms underlying the conceptual notion of "shared mental models" (SMM) through physiological (i.e., breathing and heart rates) and affective-cognitive (i.e., arousal, pleasantness, attention, self-efficacy, other's efficacy) monitoring of two professional jugglers performing a real-time interactive task of increasing difficulty. There were two experimental conditions: "individual" (i.e., solo task) and "interactive" (i.e., two jugglers established a cooperative interaction by juggling sets of balls with each other). In both conditions, there were two task difficulties: “easy” and “hard”. Descriptive analyses revealed that engaging in a dyadic cooperative motor task (interactive condition) required greater physiological effort (Median Cohen’s d = 2.13) than performing a solo motor task (individual condition) of similar difficulty. Our results indicated a strong positive correlation between the jugglers’ heart rate for the easy (r = .87) and hard tasks (r = .77). The relationship between the jugglers’ breathing rate was significant for the easy task (r = .73) but non-significant for the hard task. The findings are interpreted based on research on SMM and Theory of Mind. Practitioners should advance the notion of “shared-regulation” in the context of team coordination through the use of biofeedback training

A systematic review on the relationship between self-esteem and interrogative suggestibility

Some factors, such as age, learning disability and mental health difficulties, have been identified as making police suspects more vulnerable to suggestibility and false confessions during interview. However, there has been no systematic review on the association between self-esteem and suggestibility. Seven electronic bibliographic databases and reference lists of previous literature reviews of suggestibility in children were searched. Selected studies were quality assessed using pre-defined criteria before data were extracted. Electronic searches yielded 1914 hits. Of these, 685 duplicates, 1181 irrelevant references and 39 references that did not meet the inclusion criteria were removed. Nine publications were included in the review. Significant correlations between self-esteem and suggestibility, most notably on the Yield 1 subscale of the GSS, were found but four of the nine studies found no significant correlation. The prevalent use of self-report measures and lack of clarity in defining self-esteem limit the validity of those studies

Using guided individualised feedback to review self-reported quality of life in health and its importance

© 2014 Taylor & Francis. This pilot study investigated the effects of providing guided, individualised feedback on subjective quality of life (QoL), using results from the multidimensional WHOQOL-BREF profile. Participants (n=129; 85 chronically ill) were recruited in the community, and primary care. They were randomised to receive written or verbal guidance on interpreting a new graphical summary profile, which simultaneously presented (a) their individual self-ratings of QoL and (b) the importance attributed to each QoL dimension. Before and after feedback, participants completed health status, subjective QoL, QoL Importance, goal-oriented QoL and mood measures. Receiving individualised feedback was associated with increased psychological QoL, with modest effect size. No effects were found for physical, social or environmental QoL or QoL importance, health status, mood or goal-oriented QoL. There were no differences between modes of delivering guidance, indicating equal effectiveness. Chronic illness participants reported poorer QoL, moved more slowly towards their QoL goals, and had larger differences between core QoL and QoL Importance than healthy participants. Guided individualised empirical feedback about QoL judgements could be used to promote psychological well-being. Although professional interpretation of feedback is unnecessary, if shared, patients’ profiled WHOQOL information could support self-monitoring, self-management and clinical decision-making

Arginine Deprivation With Pegylated Arginine Deiminase in Patients With Argininosuccinate Synthetase 1-Deficient Malignant Pleural Mesothelioma A Randomized Clinical Trial

IMPORTANCE: Preclinical studies show that arginine deprivation is synthetically lethal in argininosuccinate synthetase 1 (ASS1)-negative cancers, including mesothelioma. The role of the arginine-lowering agent pegylated arginine deiminase (ADI-PEG20) has not been evaluated in a randomized and biomarker-driven study among patients with cancer. OBJECTIVE: To assess the clinical impact of arginine depletion in patients with ASS1-deficient malignant pleural mesothelioma. DESIGN, SETTING, AND PARTICIPANTS: A multicenter phase 2 randomized clinical trial, the Arginine Deiminase and Mesothelioma (ADAM) study, was conducted between March 2, 2011, and May 21, 2013, at 8 academic cancer centers. Immunohistochemical screening of 201 patients (2011-2013) identified 68 with advanced ASS1-deficient malignant pleural mesothelioma. INTERVENTIONS: Randomization 2:1 to arginine deprivation (ADI-PEG20, 36.8 mg/m2, weekly intramuscular) plus best supportive care (BSC) or BSC alone. MAIN OUTCOMES AND MEASURES: The primary end point was progression-free survival (PFS) assessed by modified Response Evaluation Criteria in Solid Tumors (RECIST) (target hazard ratio, 0.60). Secondary end points were overall survival (OS), tumor response rate, safety, and quality of life, analyzed by intention to treat. We measured plasma arginine and citrulline levels, anti–ADI-PEG20 antibody titer, ASS1 methylation status, and metabolic response by 18F-fluorodeoxyglucose positron-emission tomography. RESULTS: Median (range) follow-up in 68 adults (median [range] age, 66 [48-83] years; 19% female) was 38 (2.5-39) months. The PFS hazard ratio was 0.56 (95% CI, 0.33-0.96), with a median of 3.2 months in the ADI-PEG20 group vs 2.0 months in the BSC group (P = .03) (absolute risk, 18% vs 0% at 6 months). Best response at 4 months (modified RECIST) was stable disease: 12 of 23 (52%) in the ADI-PEG20 group vs 2 of 9 (22%) in the BSC group (P = .23). The OS curves crossed, so life expectancy was used: 15.7 months in the ADI-PEG20 group vs 12.1 months in the BSC group (difference of 3.6 [95% CI, −1.0 to 8.1] months; P = .13). The incidence of symptomatic adverse events of grade at least 3 was 11 of 44 (25%) in the ADI-PEG20 group vs 4 of 24 (17%) in the BSC group (P = .43), the most common being immune related, nonfebrile neutropenia, gastrointestinal events, and fatigue. Differential ASS1 gene-body methylation correlated with ASS1 immunohistochemistry, and longer arginine deprivation correlated with improved PFS. CONCLUSIONS AND RELEVANCE: In this trial, arginine deprivation with ADI-PEG20 improved PFS in patients with ASS1-deficient mesothelioma. Targeting arginine is safe and warrants further clinical investigation in arginine-dependent cancers