Automata guided hierarchical reinforcement learning for zero-shot skill composition

An obstacle that prevents the wide adoption of (deep) reinforcement learning (RL) in control systems is its need for a large amount of interactions with the environment in order to master a skill. The learned skill usually generalizes poorly across domains and re-training is often necessary when presented with a new task. We present a framework that combines methods in formal methods with hierarchical reinforcement learning (HRL). The set of techniques we provide allows for convenient specification of tasks with complex logic, learn hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards using any RL methods and is able to construct new skills from existing ones without additional learning. We evaluate the proposed methods in a simple grid world simulation as well as simulation on a Baxter robot

Post-Transcriptional Regulation of BCL2 mRNA by the RNA-Binding Protein ZFP36L1 in Malignant B Cells

The human ZFP36 zinc finger protein family consists of ZFP36, ZFP36L1, and ZFP36L2. These proteins regulate various cellular processes, including cell apoptosis, by binding to adenine uridine rich elements in the 3′ untranslated regions of sets of target mRNAs to promote their degradation. The pro-apoptotic and other functions of ZFP36 family members have been implicated in the pathogenesis of lymphoid malignancies. To identify candidate mRNAs that are targeted in the pro-apoptotic response by ZFP36L1, we reverse-engineered a gene regulatory network for all three ZFP36 family members using the ‘maximum information coefficient’ (MIC) for target gene inference on a large microarray gene expression dataset representing cells of diverse histological origin. Of the three inferred ZFP36L1 mRNA targets that were identified, we focussed on experimental validation of mRNA for the pro-survival protein, BCL2, as a target for ZFP36L1. RNA electrophoretic mobility shift assay experiments revealed that ZFP36L1 interacted with the BCL2 adenine uridine rich element. In murine BCL1 leukemia cells stably transduced with a ZFP36L1 ShRNA lentiviral construct, BCL2 mRNA degradation was significantly delayed compared to control lentiviral expressing cells and ZFP36L1 knockdown in different cell types (BCL1, ACHN, Ramos), resulted in increased levels of BCL2 mRNA levels compared to control cells. 3′ untranslated region luciferase reporter assays in HEK293T cells showed that wild type but not zinc finger mutant ZFP36L1 protein was able to downregulate a BCL2 construct containing the BCL2 adenine uridine rich element and removal of the adenine uridine rich core from the BCL2 3′ untranslated region in the reporter construct significantly reduced the ability of ZFP36L1 to mediate this effect. Taken together, our data are consistent with ZFP36L1 interacting with and mediating degradation of BCL2 mRNA as an important target through which ZFP36L1 mediates its pro-apoptotic effects in malignant B-cells

An increased fraction of circulating miR-363 and miR-16 is particle bound in patients with chronic lymphocytic leukaemia as compared to normal subjects.

In vitro culture studies have shown that miR-363 is enriched in extracellular vesicles from chronic lymphocytic leukaemia cells. We wondered whether miR-363 was detectable in plasma, which is an essential precondition for further studies to assess its usefulness as a biomarker. Using samples from two clinical trials: one enrolling patients with advanced disease and the other asymptomatic patients with early stage disease, we determined plasma miR-363 levels and secondly investigated the distribution of this miRNA between plasma and particle bound fractions in patients and normal subjects.Advanced disease (n = 95) was associated with higher levels of miR-363 than early stage disease (n = 45) or normal subjects (n = 11) but there was no association with markers of prognosis. The distribution of specific miRNA between particle bound and plasma protein fractions was investigated using size exclusion chromatography on plasma from patients (n = 4) and normal subjects (n = 3). ~ 20% of total miR-16 and miR-363 is particle bound in patients while there was no detectable particle bound material in normal subjects. Our work demonstrates that miR-363 levels are raised in chronic lymphocytic leukaemia patients and raises the possibility that distribution of circulating miRNA between plasma fractions differs in health and disease

Modulation of MicroRNA-194 and cell migration by HER2-targeting trastuzumab in breast cancer

Conceived and designed the experiments: XFL GAC RCB. Performed the experiments: XFL MIA WM RS MSN SZ. Analyzed the data: XFL SR. Contributed reagents/materials/analysis tools: YW GAC. Wrote the paper: XFL RCB.Trastuzumab, a humanized monoclonal antibody directed against the extracellular domain of the HER2 oncoprotein, can effectively target HER2-positive breast cancer through several mechanisms. Although the effects of trastuzumab on cancer cell proliferation, angiogenesis and apoptosis have been investigated in depth, the effect of trastuzumab on microRNA (miRNA) has not been extensively studied. We have performed miRNA microarray profiling before and after trastuzumab treatment in SKBr3 and BT474 human breast cancer cells that overexpress HER2. We found that trastuzumab treatment of SKBr3 cells significantly decreased five miRNAs and increased three others, whereas treatment of BT474 cells significantly decreased two miRNAs and increased nine. The only change in miRNA expression observed in both cell lines following trastuzumab treatment was upregulation of miRNA-194 (miR-194) that was further validated in vitro and in vivo. Forced expression of miR-194 in breast cancer cells that overexpress HER2 produced no effect on apoptosis, modest inhibition of proliferation, significant inhibition of cell migration/invasion in vitro and significant inhibition of xenograft growth in vivo. Conversely, knockdown of miR-194 promoted cell migration. Increased miR-194 expression markedly reduced levels of the cytoskeletal protein talin2 and specifically inhibited luciferase reporter activity of a talin2 wild-type 39-untranslated region, but not that of a mutant reporter, indicating that talin2 is a direct downstream target of miR-194. Trastuzumab treatment inhibited breast cancer cell migration and reduced talin2 expression in vitro and in vivo. Knockdown of talin2 inhibited cell migration/invasion. Knockdown of trastuzumab-induced miR-194 expression with a miR-194 inhibitor compromised trastuzumab-inhibited cell migration in HER2-overexpressing breast cancer cells. Consequently, trastuzumab treatment upregulates miR-194 expression and may exert its cell migration-inhibitory effect through miR-194-mediated downregulation of cytoskeleton protein talin2 in HER2-overexpressing human breast cancer cells.This work was supported by the Anne and Henry Zarrow Foundation, kind gifts from Stuart and Gaye Lynn Zarrow and from Mrs. Delores Wilkenfeld, the Laura and John Arnold Foundation, the RGK Foundation, and the MD Anderson NCI CCSG P30 CA16672. G.A.C. is supported as a Fellow at the University of Texas MD Anderson Research Trust, as a University of Texas System Regents Research Scholar and by the CLL Global Research Foundation

MYC-microRNA-9-metastasis connection in breast cancer

[Excerpt] Metastasis accounts for more than 90% of cancer patients’ mortality. The metastatic process involves multiple steps [1]. Initially, cancer cells from the primary tumor invade adjacent stroma. To acquire this capacity, cells undergo a process called epithelial-mesenchymal transition (EMT), in which cells in re-sponse to signals from the surrounding stroma, undergo a switch between cell phenotypes and acquire mesenchymal properties and show reduced intercel-lular adhesion, allowing cells to be-come motile. Then cells enter systemic circulation, either through the blood or lymph, and finally extravasate into the parenchyma of distant tissues, where they form micrometastasis and prolifer-ate to form secondary tumors [2]. [...

Epidermal growth factor receptor-regulated miR-125a-5p – a metastatic inhibitor of lung cancer

Both the epidermal growth factor receptor signaling pathway and microRNA (miRNA) play an important role in lung cancer development and progression. To address the potential role of miRNA in epidermal growth factor receptor signaling, we identified miR-125a-5p as a downstream target, using an miRNA array. We further demonstrated that miR-125a-5p inhibited migration and invasion of lung cancer cells. Moreover, miR-125a-5p regulated the expression of several downstream genes of epidermal growth factor receptor signaling. Importantly, examination of lung cancer samples revealed a significant correlation of miR-125a-5p repression with lung carcinogenesis. Taken together, our results provide compelling evidence that miR-125a-5p, an epidermal growth factor-signaling-regulated miRNA, may function as a metastatic suppressor

Fluorescence activated cell sorting followed by small RNA sequencing reveals stable microRNA expression during cell cycle progression.

BACKGROUND: Previously, drug-based synchronization procedures were used for characterizing the cell cycle dependent transcriptional program. However, these synchronization methods result in growth imbalance and alteration of the cell cycle machinery. DNA content-based fluorescence activated cell sorting (FACS) is able to sort the different cell cycle phases without perturbing the cell cycle. MiRNAs are key transcriptional regulators of the cell cycle, however, their expression dynamics during cell cycle has not been explored. METHODS: Following an optimized FACS, a complex initiative of high throughput platforms (microarray, Taqman Low Density Array, small RNA sequencing) were performed to study gene and miRNA expression profiles of cell cycle sorted human cells originating from different tissues. Validation of high throughput data was performed using quantitative real time PCR. Protein expression was detected by Western blot. Complex statistics and pathway analysis were also applied. RESULTS: Beyond confirming the previously described cell cycle transcriptional program, cell cycle dependently expressed genes showed a higher expression independently from the cell cycle phase and a lower amplitude of dynamic changes in cancer cells as compared to untransformed fibroblasts. Contrary to mRNA changes, miRNA expression was stable throughout the cell cycle. CONCLUSIONS: Cell cycle sorting is a synchronization-free method for the proper analysis of cell cycle dynamics. Altered dynamic expression of universal cell cycle genes in cancer cells reflects the transformed cell cycle machinery. Stable miRNA expression during cell cycle progression may suggest that dynamical miRNA-dependent regulation may be of less importance in short term regulations during the cell cycle

Protein phosphatase 2A as a therapeutic target in acute myeloid leukemia

Acute myeloid leukemia (AML) is a heterogeneous malignant disorder of hematopoietic progenitor cells in which several genetic and epigenetic aberrations have been described. Despite progressive advances in our understanding of the molecular biology of this disease, the outcome for most patients is poor. It is therefore necessary to develop more effective treatment strategies. Genetic aberrations affecting kinases have been widely studied in AML; however, the role of phosphatases remains underexplored. Inactivation of the tumor suppressor protein phosphatase 2A (PP2A) is frequent in AML patients, making it a promising target for therapy. There are several PP2A inactivating mechanisms reported in this disease. Deregulation or specific post-translational modifications of PP2A subunits have been identified as a cause of PP2A malfunction, which lead to deregulation of proliferation or apoptosis pathways, depending on the subunit affected. Likewise, overexpression of either SET or CIP2A, endogenous inhibitors of PP2A, is a recurrent event in AML that impairs PP2A activity, contributing to leukemogenesis progression. Interestingly, the anticancer activity of several PP2A-activating drugs (PADs) depends on interaction/sequestration of SET. Preclinical studies show that pharmacological restoration of PP2A activity by PADs effectively antagonizes leukemogenesis, and that these drugs have synergistic cytotoxic effects with conventional chemotherapy and kinase inhibitors, opening new possibilities for personalized treatment in AML patients, especially in cases with SET-dependent inactivation of PP2A. Here, we review the role of PP2A as a druggable tumor suppressor in AML

NeuronBank: A Tool for Cataloging Neuronal Circuitry

The basic unit of any nervous system is the neuron. Therefore, understanding the operation of nervous systems ultimately requires an inventory of their constituent neurons and synaptic connectivity, which form neural circuits. The presence of uniquely identifiable neurons or classes of neurons in many invertebrates has facilitated the construction of cellular-level connectivity diagrams that can be generalized across individuals within a species. Homologous neurons can also be recognized across species. Here we describe NeuronBank.org, a web-based tool that we are developing for cataloging, searching, and analyzing neuronal circuitry within and across species. Information from a single species is represented in an individual branch of NeuronBank. Users can search within a branch or perform queries across branches to look for similarities in neuronal circuits across species. The branches allow for an extensible ontology so that additional characteristics can be added as knowledge grows. Each entry in NeuronBank generates a unique accession ID, allowing it to be easily cited. There is also an automatic link to a Wiki page allowing an encyclopedic explanation of the entry. All of the 44 previously published neurons plus one previously unpublished neuron from the mollusc, Tritonia diomedea, have been entered into a branch of NeuronBank as have 4 previously published neurons from the mollusc, Melibe leonina. The ability to organize information about neuronal circuits will make this information more accessible, ultimately aiding research on these important models