Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

Combination of RAD001 (everolimus) and docetaxel reduces prostate and breast cancer cell VEGF production and tumour vascularisation independently of sphingosine-kinase-1

Resistance to docetaxel is a key problem in current prostate and breast cancer management. We have recently discovered a new molecular mechanism of prostate cancer docetaxel chemoresistance mediated by the mammalian target of rapamycin (mTOR)/sphingosine-kinase-1 (SK1) pathway. Here we investigated the influence of this pathway on vascular endothelial growth factor (VEGF) production and tumour vascularisation in hormone resistant prostate and breast cancer models. Immunofluorescent staining of tumour sections from human oestrogen receptor (ER)-negative breast cancer patients showed a strong correlation between phosphorylated P70S6 kinase (mTOR downstream target), VEGF and SK1 protein expression. In hormone-insensitive prostate (PC3) and breast (MDA-MB-231 and BT-549) cancer cell lines the mTOR inhibitor RAD001 (everolimus) has significantly inhibited SK1 and VEGF expression, while low dose (5 nM) docetaxel had no significant effect. In these cell lines, SK1 overexpression slightly increased the basal levels of VEGF, but did not block the inhibitory effect of RAD001 on VEGF. In a human prostate xenograft model established in nude mice, RAD001 alone or in combination with docetaxel has suppressed tumour growth, VEGF expression and decreased tumour vasculature. Overall, our data demonstrate a new mechanism of an independent regulation of SK1 and VEGF by mTOR in hormone-insensitive prostate and breast cancers

Arginine Deprivation With Pegylated Arginine Deiminase in Patients With Argininosuccinate Synthetase 1-Deficient Malignant Pleural Mesothelioma A Randomized Clinical Trial

IMPORTANCE: Preclinical studies show that arginine deprivation is synthetically lethal in argininosuccinate synthetase 1 (ASS1)-negative cancers, including mesothelioma. The role of the arginine-lowering agent pegylated arginine deiminase (ADI-PEG20) has not been evaluated in a randomized and biomarker-driven study among patients with cancer. OBJECTIVE: To assess the clinical impact of arginine depletion in patients with ASS1-deficient malignant pleural mesothelioma. DESIGN, SETTING, AND PARTICIPANTS: A multicenter phase 2 randomized clinical trial, the Arginine Deiminase and Mesothelioma (ADAM) study, was conducted between March 2, 2011, and May 21, 2013, at 8 academic cancer centers. Immunohistochemical screening of 201 patients (2011-2013) identified 68 with advanced ASS1-deficient malignant pleural mesothelioma. INTERVENTIONS: Randomization 2:1 to arginine deprivation (ADI-PEG20, 36.8 mg/m2, weekly intramuscular) plus best supportive care (BSC) or BSC alone. MAIN OUTCOMES AND MEASURES: The primary end point was progression-free survival (PFS) assessed by modified Response Evaluation Criteria in Solid Tumors (RECIST) (target hazard ratio, 0.60). Secondary end points were overall survival (OS), tumor response rate, safety, and quality of life, analyzed by intention to treat. We measured plasma arginine and citrulline levels, anti–ADI-PEG20 antibody titer, ASS1 methylation status, and metabolic response by 18F-fluorodeoxyglucose positron-emission tomography. RESULTS: Median (range) follow-up in 68 adults (median [range] age, 66 [48-83] years; 19% female) was 38 (2.5-39) months. The PFS hazard ratio was 0.56 (95% CI, 0.33-0.96), with a median of 3.2 months in the ADI-PEG20 group vs 2.0 months in the BSC group (P = .03) (absolute risk, 18% vs 0% at 6 months). Best response at 4 months (modified RECIST) was stable disease: 12 of 23 (52%) in the ADI-PEG20 group vs 2 of 9 (22%) in the BSC group (P = .23). The OS curves crossed, so life expectancy was used: 15.7 months in the ADI-PEG20 group vs 12.1 months in the BSC group (difference of 3.6 [95% CI, −1.0 to 8.1] months; P = .13). The incidence of symptomatic adverse events of grade at least 3 was 11 of 44 (25%) in the ADI-PEG20 group vs 4 of 24 (17%) in the BSC group (P = .43), the most common being immune related, nonfebrile neutropenia, gastrointestinal events, and fatigue. Differential ASS1 gene-body methylation correlated with ASS1 immunohistochemistry, and longer arginine deprivation correlated with improved PFS. CONCLUSIONS AND RELEVANCE: In this trial, arginine deprivation with ADI-PEG20 improved PFS in patients with ASS1-deficient mesothelioma. Targeting arginine is safe and warrants further clinical investigation in arginine-dependent cancers

Porosity estimation of (Moso bamboo) by computed tomography and backscattered electron imaging

This study aims to investigate and quantify the porosity in the cross section of Phyllostachys edulis (Moso bamboo) culm wall. The porosity results are expected to be utilised in numerical study of heat and moisture transfer. Computed tomography (CT) and backscattered electron (BSE) imaging methods are utilised in this study because these two methods allow measurements of the anisotropic features of bamboo specimens. The results of these two methods can be represented as the function of the real dimension rather than the pore size distribution of the specimen. The specimens are obtained from eight different locations along the Moso bamboo culms. Both internodes and nodes specimens are measured in this study. The average porosity, standard deviation (SD) and coefficient of variation (COV) are calculated for BSE and CT results. Pearson product-moment correlation coefficient (PPMCC) is also calculated in this study to analyse the correlation between the BSE results and CT results. Typical porosity results from 400 sampling points and 10 portions average porosity are analysed in this study. The CT scanning results show similar trend with BSE results. The correlation relationship between BSE and CT results approaches moderate correlation level to strong correlation level. The average porosity of internode specimens is from 43.9 to 58.8 % by BSE measurement and from 44.9 to 63.4 % by CT measurement. The average porosity of node specimens is from 37.4 to 56.6 % by BSE measurement and from 32.1 to 62.2 % by CT measurement

Porosity estimation of Phyllostachys edulis (Moso bamboo) by computed tomography and backscattered electron imaging

This study aims to investigate and quantify the porosity in the cross section of Phyllostachys edulis (Moso bamboo) culm wall. The porosity results are expected to be utilised in numerical study of heat and moisture transfer. Computed tomography (CT) and backscattered electron (BSE) imaging methods are utilised in this study because these two methods allow measurements of the anisotropic features of bamboo specimens. The results of these two methods can be represented as the function of the real dimension rather than the pore size distribution of the specimen. The specimens are obtained from eight different locations along the Moso bamboo culms. Both internodes and nodes specimens are measured in this study. The average porosity, standard deviation (SD) and coefficient of variation (COV) are calculated for BSE and CT results. Pearson product-moment correlation coefficient (PPMCC) is also calculated in this study to analyse the correlation between the BSE results and CT results. Typical porosity results from 400 sampling points and 10 portions average porosity are analysed in this study. The CT scanning results show similar trend with BSE results. The correlation relationship between BSE and CT results approaches moderate correlation level to strong correlation level. The average porosity of internode specimens is from 43.9 to 58.8 % by BSE measurement and from 44.9 to 63.4 % by CT measurement. The average porosity of node specimens is from 37.4 to 56.6 % by BSE measurement and from 32.1 to 62.2 % by CT measurement

Neutrophils in cancer: neutral no more

Neutrophils are indispensable antagonists of microbial infection and facilitators of wound healing. In the cancer setting, a newfound appreciation for neutrophils has come into view. The traditionally held belief that neutrophils are inert bystanders is being challenged by the recent literature. Emerging evidence indicates that tumours manipulate neutrophils, sometimes early in their differentiation process, to create diverse phenotypic and functional polarization states able to alter tumour behaviour. In this Review, we discuss the involvement of neutrophils in cancer initiation and progression, and their potential as clinical biomarkers and therapeutic targets

Shared mental models and intra-team psychophysiological patterns: A test of the juggling paradigm

We explored implicit coordination mechanisms underlying the conceptual notion of "shared mental models" (SMM) through physiological (i.e., breathing and heart rates) and affective-cognitive (i.e., arousal, pleasantness, attention, self-efficacy, other's efficacy) monitoring of two professional jugglers performing a real-time interactive task of increasing difficulty. There were two experimental conditions: "individual" (i.e., solo task) and "interactive" (i.e., two jugglers established a cooperative interaction by juggling sets of balls with each other). In both conditions, there were two task difficulties: “easy” and “hard”. Descriptive analyses revealed that engaging in a dyadic cooperative motor task (interactive condition) required greater physiological effort (Median Cohen’s d = 2.13) than performing a solo motor task (individual condition) of similar difficulty. Our results indicated a strong positive correlation between the jugglers’ heart rate for the easy (r = .87) and hard tasks (r = .77). The relationship between the jugglers’ breathing rate was significant for the easy task (r = .73) but non-significant for the hard task. The findings are interpreted based on research on SMM and Theory of Mind. Practitioners should advance the notion of “shared-regulation” in the context of team coordination through the use of biofeedback training

A systematic review on the relationship between self-esteem and interrogative suggestibility

Some factors, such as age, learning disability and mental health difficulties, have been identified as making police suspects more vulnerable to suggestibility and false confessions during interview. However, there has been no systematic review on the association between self-esteem and suggestibility. Seven electronic bibliographic databases and reference lists of previous literature reviews of suggestibility in children were searched. Selected studies were quality assessed using pre-defined criteria before data were extracted. Electronic searches yielded 1914 hits. Of these, 685 duplicates, 1181 irrelevant references and 39 references that did not meet the inclusion criteria were removed. Nine publications were included in the review. Significant correlations between self-esteem and suggestibility, most notably on the Yield 1 subscale of the GSS, were found but four of the nine studies found no significant correlation. The prevalent use of self-report measures and lack of clarity in defining self-esteem limit the validity of those studies

A systematic quality assessment of Environmental Impact Statements in the oil and gas industry

The global economy relies heavily on oil and gas resources. However, hydrocarbon exploitation projects can cause significant impacts on the environment. But despite the production of numerous Environmental Impact Statements (EISs) to identify/mitigate such impacts, no study has specifically assessed the quality of EISs for both onshore and offshore oil and gas projects, with tested hypotheses. To address this research gap, our paper, for the first time, develops a modified Lee and Colley evaluation model to assess the quality of 19 sampled oil and gas project EISs produced from 1998 to 2008 in Nigeria. Our findings show that Project Description and Communication of Results are the main areas of strength. However, Environmental Impact Prediction, and Project Decommissioning, were among the key areas requiring attention. A key finding, though, is that Mann-Whitney tests suggest that there is no evidence that the quality of EISs for the latter period (2004–2008) is higher than that of the earlier period (1998–2004). We suggest that periodic systematic review of the quality of submitted/approved EISs (c. every 3–5 years) should be established to monitor trends in EIS quality and identify strong and weak areas. This would help to drive continual improvement in both the EIA processes and the resultant EISs of technical engineering projects. Such reviews have the potential to illuminate some of the underlying problems of, and solutions to, oil and gas exploration, production and transportation, and their related environmental impacts. This suggested change would also be useful internationally, including for the burgeoning exploration and production of unconventional hydrocarbon resources.Publisher Statement: NOTICE: this is the author’s version of a work that was accepted for publication in Science of The Total Environment. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Science of The Total Environment, [572, (2016)] DOI: 10.1016/j.scitotenv.2016.07.083© 2016, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0

Reproducibility of shear wave elastography measuresof the Achilles tendon.

OBJECTIVE To assess the reproducibility of shear wave elastography (SWE) measures in the Achilles tendon (AT) in vivo. MATERIALS AND METHODS Shear wave velocity (SWV) of 14 healthy volunteers [7 males, 7 females; mean age 26.5 ± 3.8 years, mean height 171.6 ± 10.9 cm, mean Victorian Institute of Sports Assessment Achilles questionnaire (VISA-A) score 99.4 ± 1.2] was measured with the foot relaxed and fixed at 90°. Data were collected over five consecutive measures and 5 consecutive days. RESULTS Mean SWV values ranged from 7.91 m/s-9.56 m/s ± 0.27-0.50 m/s. Coefficient of variation (CV), correlations and intra-class correlation coefficient (ICC) scores ranged from 2.9%-6.3%, 0.4-0.7 and 0.54-0.85 respectively. No significant differences were noted for longitudinal or transverse data with respect to protocol or time and no significant differences were noted for foot position in transverse data. Significant differences in SWV values were noted between foot positions for longitudinal scanning (p = <0.05), with a relaxed foot position providing SWV values on average 0.47 m/s faster than a fixed position. Increased reproducibility was obtained with the foot relaxed. ICC between operators was 0.70 for transverse and 0.80 for longitudinal scanning. CONCLUSIONS Reproducible SWE measures were obtained over a 1-h period as well as a period of 5 consecutive days with more reliable measures obtained from a longitudinal plane using a relaxed foot position. SWE also has a high level of agreement between operators making SWE a reproducible technique for quantitatively assessing the mechanical properties of the human AT in vivo