Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and unstable behaviour have been long standing issues in DQNs. The unstable behaviour is often characterised by overestimation in the $Q$-values, commonly referred to as the overestimation bias. To address the overestimation bias and the divergent behaviour, a number of heuristic extensions have been proposed. Notably, multi-step updates have been shown to drastically reduce unstable behaviour while improving agent's training performance. However, agents are often highly sensitive to the selection of the multi-step update horizon ($n$), and our empirical experiments show that a poorly chosen static value for $n$ can in many cases lead to worse performance than single-step DQN. Inspired by the success of $n$-step DQN and the effects that multi-step updates have on overestimation bias, this paper proposes a new algorithm that we call `Elastic Step DQN' (ES-DQN). It dynamically varies the step size horizon in multi-step updates based on the similarity of states visited. Our empirical evaluation shows that ES-DQN out-performs $n$-step with fixed $n$ updates, Double DQN and Average DQN in several OpenAI Gym environments while at the same time alleviating the overestimation bias

Dark-adapted red flash ERGs in healthy adults

Purpose: The x-wave of the dark-adapted (DA) ERG to a red flash reflects DA cone function. This exploratory study of healthy adults aimed to investigate changes in the DA red ERG with flash strength and during dark adaptation to optimise visualisation and therefore quantification of the x-wave. Methods: The effect of altering red flash strength was investigated in four subjects by recording ERGs after 20 minutes dark adaptation to red flashes (0.2–2.0 cd s m-2) using skin electrodes and natural pupils. The effect of dark adaptation duration was investigated in 16 subjects during 20 minutes in the dark, by recording DA 1.5 red ERGs at 1, 2, 3, 4, 5, 10, 15 and 20 minutes. Results: For a dark adaption period of 20 minutes, the x-wave was more clearly visualised to weaker (< 0.6 cd s m-2) red flash strengths: to stronger flashes it became obscured by the b-wave. For red flashes of 1.5 cd s m-2, the x-wave was most prominent in ERGs recorded after 1–5 minutes of dark adaptation: with longer dark-adaptation, it was subsumed into the b-wave’s rising edge. Conclusions: This small study suggests that x-wave visibility in healthy subjects after 20 minutes dark adaptation is improved by using flashes weaker than around 0.6 cd s m-2; for flash strengths of 1.5 cd s m-2, x-wave visibility is enhanced by recording after only around 5 minutes of dark adaptation. No evidence was found that interim red flash ERGs affecting the dark-adapted state of the normal retina

CDK4/6 inhibitors induce replication stress to cause long-term cell cycle withdrawal

CDK4/6 inhibitors arrest the cell cycle in G1‐phase. They are approved to treat breast cancer and are also undergoing clinical trials against a range of other tumour types. To facilitate these efforts, it is important to understand why a cytostatic arrest in G1 causes long‐lasting effects on tumour growth. Here, we demonstrate that a prolonged G1 arrest following CDK4/6 inhibition downregulates replisome components and impairs origin licencing. Upon release from that arrest, many cells fail to complete DNA replication and exit the cell cycle in a p53‐dependent manner. If cells fail to withdraw from the cell cycle following DNA replication problems, they enter mitosis and missegregate chromosomes causing excessive DNA damage, which further limits their proliferative potential. These effects are observed in a range of tumour types, including breast cancer, implying that genotoxic stress is a common outcome of CDK4/6 inhibition. This unanticipated ability of CDK4/6 inhibitors to induce DNA damage now provides a rationale to better predict responsive tumour types and effective combination therapies, as demonstrated by the fact that CDK4/6 inhibition induces sensitivity to chemotherapeutics that also cause replication stress

Epigenetic Profiling and Molecular Characterisation of Non-melanoma Skin Cancer

PhDNon-melanoma skin (NMSC) cancer is the most common human malignancy. Cutaneous squamous cell carcinoma (cSCC) and its precursor, actinic keratosis (AK) affect tens of thousands of people each year in the UK. Merkel cell carcinoma is a rare, yet aggressive type of NMSC recently linked with Merkel Cell Polyomavirus (MCPyV). In spite of the clinical burden of NMSC, key molecular regulatory patterns remain largely unknown. The aims of this thesis were to investigate genome-wide genetic, epigenetic and transcriptional changes in AK and cSCC, and assess the prevalence of MCPyV and its effect on methylation in NMSC. Copy-number analysis revealed that AK harbours significantly more genomic aberrations compared to skin, the majority of which occurs on chromosomes 8 and 9. Transcriptional profiling has found 292 and 308 genes as differentially expressed in AK compared to non-sunexposed and sun-exposed skin, respectively, and gene-set enrichment analysis (GSEA) revealed dysregulation of PPAR pathway in this lesion. Expression profiling of cSCC and AK has revealed 346 differentially expressed genes, and GSEA detected dysregulation in several canonical pathways including TGF-β and MAPK pathway. Aberrant methylation in cSCC cell lines occurs in the promoters of many developmental genes. A total of 1085 hyper- and 833 hypomethylated genes were detected in cSCCs, and GSEA revealed dysregulation of critical signalling pathways (WNT, MAPK signalling pathways). Methylation analysis of AK revealed a total of 4194 differentially methylated genes, and implicated FOXF2, PITX2, RUNX1 and SMAD3 transcription factors in this lesions. MiRNA profiling of cSCC and normal skin revealed significant dysregulation of 38 miRNAs including several of viral origin. MCPyV was shown to be common in NMSC, yet MCPyV nor human papillomavirus does not affect cSCC methylation. Taken together, this work provides novel insight into molecular regulation of cSCC oncogenesis, and identifies potential epigenetic targets for functional evaluation in this malignancy.British Skin Foundation and the Barts and the London Charity research grant

Validity of self-reported weight, height, and body mass index among university students in Thailand: Implications for population studies of obesity in developing countries

<p>Abstract</p> <p>Background</p> <p>Large-scale epidemiological studies commonly use self-reported weights and heights to determine weight status. Validity of such self-reported data has been assessed primarily in Western populations in developed countries, although its use is widespread in developing countries. We examine the validity of obesity based on self-reported data in an Asian developing country, and derive improved obesity prevalence estimates using the "reduced BMI threshold" method.</p> <p>Methods</p> <p>Self-reported and measured heights and weights were obtained from 741 students attending an open university in Thailand (mean age 34 years). Receiver operator characteristic techniques were applied to derive "reduced BMI thresholds."</p> <p>Results</p> <p>Height was over-reported by a mean of 1.54 cm (SD 2.23) in men and 1.33 cm (1.84) in women. Weight was under-reported by 0.93 kg (3.47) in men and 0.62 kg (2.14) in women. Sensitivity and specificity for determining obesity (Thai BMI threshold 25 kg/m²) using self-reported data were 74.2% and 97.3%, respectively, for men and 71.9% and 100% for women. For men, reducing the BMI threshold to 24.5 kg/m²increased the estimated obesity prevalence based on self-reports from 29.1% to 33.8% (true prevalence was 36.9%). For women, using a BMI threshold of 24.4 kg/m², the improvement was from 12.0% to 15.9% (true prevalence 16.7%).</p> <p>Conclusion</p> <p>Young educated Thais under-report weight and over-report height in ways similar to their counterparts in developed countries. Simple adjustments to BMI thresholds will overcome these reporting biases for estimation of obesity prevalence. Our study suggests that self-reported weights and heights can provide economical and valid measures of weight status in high school-educated populations in developing countries.</p

Decomposing socioeconomic inequality for binary health outcomes: an improved estimation that does not vary by choice of reference group

BACKGROUND Decomposition of concentration indices yields useful information regarding the relative importance of various determinants of inequitable health outcomes. But the two estimation approaches to decomposition in current use are not suitable for binary outcomes. FINDINGS The paper compares three estimation approaches for decomposition of inequality concentration indices: Ordinary Least Squares (OLS), probit, and the Generalized Linear Model (GLM) binomial distribution and identity link. Data are from the Thai Health and Welfare Survey 2003. The OLS estimates do not take into account the binary nature of the outcome and the probit estimates depend on the choice of reference groups, whereas the GLM binomial identity approach has neither of these problems. CONCLUSIONS The GLM with binomial distribution and identity link allows the inequality decomposition model to hold, and produces valid estimates of determinants that do not vary according to choice of reference groups. This GLM approach is readily available in standard statistical packages.The study was conducted under the auspices of the overarching project "The Thai Health-Risk Transition: a National Cohort Study", funded by the Wellcome Trust UK (GR071587 MA) and the Australian National Health and Medical Research Council (268055)

Aberration-corrected electron microscopy of nanoparticles

The early history of scanning transmission electron microscopy (STEM) is reviewed as a way to frame the technical issues that make aberration correction an essential upgrade for the study of nanoparticles using STEM. The principles of aberration correction are explained, and the use of aberration-corrected microscopy in the study of nanostructures is exemplified in order to remark the features and challenges in the use of this measuring techniqu

Polo-like kinase 1 (PLK1) inhibition suppresses cell growth and enhances radiation sensitivity in medulloblastoma cells

<p>Abstract</p> <p>Background</p> <p>Medulloblastoma is the most common malignant brain tumor in children and remains a therapeutic challenge due to its significant therapy-related morbidity. Polo-like kinase 1 (<it>PLK1</it>) is highly expressed in many cancers and regulates critical steps in mitotic progression. Recent studies suggest that targeting PLK1 with small molecule inhibitors is a promising approach to tumor therapy.</p> <p>Methods</p> <p>We examined the expression of <it>PLK1 </it>mRNA in medulloblastoma tumor samples using microarray analysis. The impact of PLK1 on cell proliferation was evaluated by depleting expression with RNA interference (RNAi) or by inhibiting function with the small molecule inhibitor BI 2536. Colony formation studies were performed to examine the impact of BI 2536 on medulloblastoma cell radiosensitivity. In addition, the impact of depleting <it>PLK1 </it>mRNA on tumor-initiating cells was evaluated using tumor sphere assays.</p> <p>Results</p> <p>Analysis of gene expression in two independent cohorts revealed that <it>PLK1 </it>mRNA is overexpressed in some, but not all, medulloblastoma patient samples when compared to normal cerebellum. Inhibition of PLK1 by RNAi significantly decreased medulloblastoma cell proliferation and clonogenic potential and increased cell apoptosis. Similarly, a low nanomolar concentration of BI 2536, a small molecule inhibitor of PLK1, potently inhibited cell growth, strongly suppressed the colony-forming ability, and increased cellular apoptosis of medulloblastoma cells. Furthermore, BI 2536 pretreatment sensitized medulloblastoma cells to ionizing radiation. Inhibition of PLK1 impaired tumor sphere formation of medulloblastoma cells and decreased the expression of SRY (sex determining region Y)-box 2 (<it>SOX2</it>) mRNA in tumor spheres indicating a possible role in targeting tumor inititiating cells.</p> <p>Conclusions</p> <p>Our data suggest that targeting PLK1 with small molecule inhibitors, in combination with radiation therapy, is a novel strategy in the treatment of medulloblastoma that warrants further investigation.</p

Identification of Synaptic Targets of Drosophila Pumilio

Drosophila Pumilio (Pum) protein is a translational regulator involved in embryonic patterning and germline development. Recent findings demonstrate that Pum also plays an important role in the nervous system, both at the neuromuscular junction (NMJ) and in long-term memory formation. In neurons, Pum appears to play a role in homeostatic control of excitability via down regulation of para, a voltage gated sodium channel, and may more generally modulate local protein synthesis in neurons via translational repression of eIF-4E. Aside from these, the biologically relevant targets of Pum in the nervous system remain largely unknown. We hypothesized that Pum might play a role in regulating the local translation underlying synapse-specific modifications during memory formation. To identify relevant translational targets, we used an informatics approach to predict Pum targets among mRNAs whose products have synaptic localization. We then used both in vitro binding and two in vivo assays to functionally confirm the fidelity of this informatics screening method. We find that Pum strongly and specifically binds to RNA sequences in the 3′UTR of four of the predicted target genes, demonstrating the validity of our method. We then demonstrate that one of these predicted target sequences, in the 3′UTR of discs large (dlg1), the Drosophila PSD95 ortholog, can functionally substitute for a canonical NRE (Nanos response element) in vivo in a heterologous functional assay. Finally, we show that the endogenous dlg1 mRNA can be regulated by Pumilio in a neuronal context, the adult mushroom bodies (MB), which is an anatomical site of memory storage