Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

A Versatile Orthotopic Nude Mouse Model for Study of Esophageal Squamous Cell Carcinoma

Increasing evidence indicates tumor-stromal interactions play a crucial role in cancer. An in vivo esophageal squamous cell carcinoma (ESCC) orthotopic animal model was developed with bioluminescence imaging established with a real-time monitoring platform for functional and signaling investigation of tumor-stromal interactions. The model was produced by injection of luciferase-labelled ESCC cells into the intraesophageal wall of nude mice. Histological examination indicates this orthotopic model is highly reproducible with 100% tumorigenesis among the four ESCC cell lines tested. This new model recapitulates many clinical and pathological properties of human ESCC, including esophageal luminal stricture by squamous cell carcinoma with nodular tumor growth, adventitia invasion, lymphovascular invasion, and perineural infiltration. It was tested using an AKT shRNA knockdown of ESCC cell lines and the in vivo tumor suppressive effects of AKT knockdown were observed. In conclusion, this ESCC orthotopic mouse model allows investigation of gene functions of cancer cells in a more natural tumor microenvironment and has advantages over previous established models. It provides a versatile platform with potential application for metastasis and therapeutic regimen testing.published_or_final_versio

Partial Covering Arrays: Algorithms and Asymptotics

A covering array $\mathsf{CA}(N;t,k,v)$ is an $N\times k$ array with entries in $\{1, 2, \ldots , v\}$, for which every $N\times t$ subarray contains each $t$-tuple of $\{1, 2, \ldots , v\}^t$ among its rows. Covering arrays find application in interaction testing, including software and hardware testing, advanced materials development, and biological systems. A central question is to determine or bound $\mathsf{CAN}(t,k,v)$, the minimum number $N$ of rows of a $\mathsf{CA}(N;t,k,v)$. The well known bound $\mathsf{CAN}(t,k,v)=O((t-1)v^t\log k)$ is not too far from being asymptotically optimal. Sensible relaxations of the covering requirement arise when (1) the set $\{1, 2, \ldots , v\}^t$ need only be contained among the rows of at least $(1-\epsilon)\binom{k}{t}$ of the $N\times t$ subarrays and (2) the rows of every $N\times t$ subarray need only contain a (large) subset of $\{1, 2, \ldots , v\}^t$. In this paper, using probabilistic methods, significant improvements on the covering array upper bound are established for both relaxations, and for the conjunction of the two. In each case, a randomized algorithm constructs such arrays in expected polynomial time

Identifying critically important vascular access outcomes for trials in haemodialysis : an international survey with patients, caregivers and health professionals

BACKGROUND: Vascular access outcomes reported across haemodialysis (HD) trials are numerous, heterogeneous and not always relevant to patients and clinicians. This study aimed to identify critically important vascular access outcomes. METHOD: Outcomes derived from a systematic review, multi-disciplinary expert panel and patient input were included in a multilanguage online survey. Participants rated the absolute importance of outcomes using a 9-point Likert scale (7-9 being critically important). The relative importance was determined by a best-worst scale using multinomial logistic regression. Open text responses were analysed thematically. RESULTS: The survey was completed by 873 participants [224 (26%) patients/caregivers and 649 (74%) health professionals] from 58 countries. Vascular access function was considered the most important outcome (mean score 7.8 for patients and caregivers/8.5 for health professionals, with 85%/95% rating it critically important, and top ranked on best-worst scale), followed by infection (mean 7.4/8.2, 79%/92% rating it critically important, second rank on best-worst scale). Health professionals rated all outcomes of equal or higher importance than patients/caregivers, except for aneurysms. We identified six themes: necessity for HD, applicability across vascular access types, frequency and severity of debilitation, minimizing the risk of hospitalization and death, optimizing technical competence and adherence to best practice and direct impact on appearance and lifestyle. CONCLUSIONS: Vascular access function was the most critically important outcome among patients/caregivers and health professionals. Consistent reporting of this outcome across trials in HD will strengthen their value in supporting vascular access practice and shared decision making in patients requiring HD

Pulmonary Nodule Classification Based on Heterogeneous Features Learning

IEEE Pulmonary cancer is one of the most dangerous cancers with a high incidence and mortality. An early accurate diagnosis and treatment of pulmonary cancer can observably increase the survival rates, where computer-aided diagnosis systems can largely improve the efficiency of radiologists. In this paper, we propose a deep automated lung nodule diagnosis system based on three-dimensional convolutional neural network (3D-CNN) and support vector machine (SVM) with multiple kernel learning (MKL) algorithms. The system not only explores the computed tomography (CT) scans, but also the clinical information of patients like age, smoking history and cancer history. To extract deeper image features, a 34-layers 3D Residual Network (3D-ResNet) is employed. Heterogeneous features including the extracted image features and the clinical data are learned with MKL. The experimental results prove the effectiveness of the proposed image feature extractor and the combination of heterogeneous features in the task of lung nodule diagnosis

A Rydberg Quantum Simulator

Following Feynman and as elaborated on by Lloyd, a universal quantum simulator (QS) is a controlled quantum device which reproduces the dynamics of any other many particle quantum system with short range interactions. This dynamics can refer to both coherent Hamiltonian and dissipative open system evolution. We investigate how laser excited Rydberg atoms in large spacing optical or magnetic lattices can provide an efficient implementation of a universal QS for spin models involving (high order) n-body interactions. This includes the simulation of Hamiltonians of exotic spin models involving n-particle constraints such as the Kitaev toric code, color code, and lattice gauge theories with spin liquid phases. In addition, it provides the ingredients for dissipative preparation of entangled states based on engineering n-particle reservoir couplings. The key basic building blocks of our architecture are efficient and high-fidelity n-qubit entangling gates via auxiliary Rydberg atoms, including a possible dissipative time step via optical pumping. This allows to mimic the time evolution of the system by a sequence of fast, parallel and high-fidelity n-particle coherent and dissipative Rydberg gates.Comment: 8 pages, 4 figure

A stitch in time: Efficient computation of genomic DNA melting bubbles

Background: It is of biological interest to make genome-wide predictions of the locations of DNA melting bubbles using statistical mechanics models. Computationally, this poses the challenge that a generic search through all combinations of bubble starts and ends is quadratic. Results: An efficient algorithm is described, which shows that the time complexity of the task is O(NlogN) rather than quadratic. The algorithm exploits that bubble lengths may be limited, but without a prior assumption of a maximal bubble length. No approximations, such as windowing, have been introduced to reduce the time complexity. More than just finding the bubbles, the algorithm produces a stitch profile, which is a probabilistic graphical model of bubbles and helical regions. The algorithm applies a probability peak finding method based on a hierarchical analysis of the energy barriers in the Poland-Scheraga model. Conclusions: Exact and fast computation of genomic stitch profiles is thus feasible. Sequences of several megabases have been computed, only limited by computer memory. Possible applications are the genome-wide comparisons of bubbles with promotors, TSS, viral integration sites, and other melting-related regions.Comment: 16 pages, 10 figure

Privacy-preserving using homomorphic encryption in Mobile IoT systems

The data privacy concerns are increasingly affecting the Internet of things (IoT) and artificial intelligence (AI) applications, in which it is very challenging to protect the privacy of the underlying data. In recent, the advancements in the performances of homomorphic encryption (HE) make it possible to help protect sensitive and personal data in IoT applications using homomorphic encryption based schemes. This paper proposed a practical homomorphic encryption scheme that can enable data users in IoT systems to securely operate data over encrypted data, which can effectively protect the privacy of key data in the system. The experimental results demonstrated the effectiveness proposed scheme

Patterns of Tobacco-Use Behavior Among Chinese Smokers with Medical Conditions

Understanding the characteristics of Chinese American smokers with medical conditions and factors associated with their tobacco-use behaviors will guide effective cessation programs. In 2008, the authors described socio-demographic profiles of Chinese smokers with medical conditions treated during the period 2002–2006, documented their tobacco-use behaviors (i.e., average daily cigarette use, nicotine dependence, and number of past-year quit attempts), and drew comparisons between subjects recruited from hospitals (IP) and ambulatory settings (OP). Compared to OP, IP were significantly older, less educated, less acculturated, and more likely to be retired. Of the two groups, IP had poorer disease profiles, smoked less (4.4 vs. 11.9 cigarettes per day), and had lower nicotine-addiction scores (5.5 vs. 6.7). There was no difference between groups in past-year quit attempts. After adjustments, the data revealed that being employed and OP was associated with higher average daily cigarette use; IP were less nicotine dependent than OP; and for both groups, years of smoking was negatively associated with past-year quit attempts. Our study suggests that, more than acculturation level, health status influences the Chinese smoker’s level of cigarette use and nicotine addiction. Given the severity of their disease profiles, IP should be aggressively targeted for intervention, as they are more likely to be light smokers and to be less nicotine dependent than OP. Future tobacco treatment studies should pay attention to health status among smokers in health-care settings in order to provide a more accurate assessment of treatment needs and of barriers to successful smoking cessation