Omega-Regular Objectives in Model-Free Reinforcement Learning

We provide the first solution for model-free reinforcement learning of ω-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of ω-regular bjectives to an almost-sure reachability problem, and extend this technique to learning how to control an unknown model so that the chance of satisfying the objective is maximized. We compile ω-regular properties into limit-deterministic B¨uchi automata instead of the traditional Rabin automata; this choice sidesteps difficulties that have marred previous proposals. Our approach allows us to apply model-free, off-the-shelf reinforcement learning algorithms to compute optimal strategies from the observations of the MDP. We present an experimental evaluation of our technique on benchmark learning problems

The Human Minor Histocompatibility Antigen1 Is a RhoGAP

The human minor Histocompatibility Antigen HMHA-1 is a major target of immune responses after allogeneic stem cell transplantation applied for the treatment of leukemia and solid tumors. The restriction of its expression to hematopoietic cells and many solid tumors raised questions regarding its cellular functions. Sequence analysis of the HMHA-1 encoding HMHA1 protein revealed the presence of a possible C-terminal RhoGTPase Activating Protein (GAP) domain and an N-terminal BAR domain. Rho-family GTPases, including Rac1, Cdc42, and RhoA are key regulators of the actin cytoskeleton and control cell spreading and migration. RhoGTPase activity is under tight control as aberrant signaling can lead to pathology, including inflammation and cancer. Whereas Guanine nucleotide Exchange Factors (GEFs) mediate the exchange of GDP for GTP resulting in RhoGTPase activation, GAPs catalyze the low intrinsic GTPase activity of active RhoGTPases, resulting in inactivation. Here we identify the HMHA1 protein as a novel RhoGAP. We show that HMHA1 constructs, lacking the N-terminal region, negatively regulate the actin cytoskeleton as well as cell spreading. Furthermore, we show that HMHA1 regulates RhoGTPase activity in vitro and in vivo. Finally, we demonstrate that the HMHA1 N-terminal BAR domain is auto-inhibitory as HMHA1 mutants lacking this region, but not full-length HMHA1, showed GAP activity towards RhoGTPases. In conclusion, this study shows that HMHA1 acts as a RhoGAP to regulate GTPase activity, cytoskeletal remodeling and cell spreading, which are crucial functions in normal hematopoietic and cancer cells

Phylogenetic Detection of Recombination with a Bayesian Prior on the Distance between Trees

Genomic regions participating in recombination events may support distinct topologies, and phylogenetic analyses should incorporate this heterogeneity. Existing phylogenetic methods for recombination detection are challenged by the enormous number of possible topologies, even for a moderate number of taxa. If, however, the detection analysis is conducted independently between each putative recombinant sequence and a set of reference parentals, potential recombinations between the recombinants are neglected. In this context, a recombination hotspot can be inferred in phylogenetic analyses if we observe several consecutive breakpoints. We developed a distance measure between unrooted topologies that closely resembles the number of recombinations. By introducing a prior distribution on these recombination distances, a Bayesian hierarchical model was devised to detect phylogenetic inconsistencies occurring due to recombinations. This model relaxes the assumption of known parental sequences, still common in HIV analysis, allowing the entire dataset to be analyzed at once. On simulated datasets with up to 16 taxa, our method correctly detected recombination breakpoints and the number of recombination events for each breakpoint. The procedure is robust to rate and transition∶transversion heterogeneities for simulations with and without recombination. This recombination distance is related to recombination hotspots. Applying this procedure to a genomic HIV-1 dataset, we found evidence for hotspots and de novo recombination

Local therapy of cancer with free IL-2

This is a position paper about the therapeutic effects of locally applied free IL-2 in the treatment of cancer. Local therapy: IL-2 therapy of cancer was originally introduced as a systemic therapy. This therapy led to about 20% objective responses. Systemic therapy however was very toxic due to the vascular leakage syndrome. Nevertheless, this treatment was a break-through in cancer immunotherapy and stimulated some interesting questions: Supposing that the mechanism of IL-2 treatment is both proliferation and tumoricidal activity of the tumor infiltrating cells, then locally applied IL-2 should result in a much higher local IL-2 concentration than systemic IL-2 application. Consequently a greater beneficial effect could be expected after local IL-2 application (peritumoral = juxtatumoral, intratumoral, intra-arterial, intracavitary, or intratracheal = inhalation). Free IL-2: Many groups have tried to prepare a more effective IL-2 formulation than free IL-2. Examples are slow release systems, insertion of the IL-2 gene into a tumor cell causing prolonged IL-2 release. However, logistically free IL-2 is much easier to apply; hence we concentrated in this review and in most of our experiments on the use of free IL-2. Local therapy with free IL-2 may be effective against transplanted tumors in experimental animals, and against various spontaneous carcinomas, sarcomas, and melanoma in veterinary and human cancer patients. It may induce rejection of very large, metastasized tumor loads, for instance advanced clinical tumors. The effects of even a single IL-2 application may be impressive. Not each tumor or tumor type is sensitive to local IL-2 application. For instance transplanted EL4 lymphoma or TLX9 lymphoma were not sensitive in our hands. Also the extent of sensitivity differs: In Bovine Ocular Squamous Cell Carcinoma (BOSCC) often a complete regression is obtained, whereas with the Bovine Vulval Papilloma and Carcinoma Complex (BVPCC) mainly stable disease is attained. Analysis of the results of local IL-2 therapy in 288 cases of cancer in human patients shows that there were 27% Complete Regressions (CR), 23% Partial Regressions (PR), 18% Stable Disease (SD), and 32% Progressive Disease (PD). In all tumors analyzed, local IL-2 therapy was more effective than systemic IL-2 treatment. Intratumoral IL-2 applications are more effective than peritumoral application or application at a distant site. Tumor regression induced by intratumoral IL-2 application may be a fast process (requiring about a week) in the case of a highly vascular tumor since IL-2 induces vascular leakage/edema and consequently massive tumor necrosis. The latter then stimulates an immune response. In less vascular tumors or less vascular tumor sites, regression may require 9–20 months; this regression is mainly caused by a cytotoxic leukocyte reaction. Hence the disadvantageous vascular leakage syndrome complicating systemic treatment is however advantageous in local treatment, since local edema may initiate tumor necrosis. Thus the therapeutic effect of local IL-2 treatment is not primarily based on tumor immunity, but tumor immunity seems to be useful as a secondary component of the IL-2 induced local processes. If local IL-2 is combined with surgery, radiotherapy or local chemotherapy the therapeutic effect is usually greater than with either therapy alone. Hence local free IL-2 application can be recommended as an addition to standard treatment protocols. Local treatment with free IL-2 is straightforward and can readily be applied even during surgical interventions. Local IL-2 treatment is usually without serious side effects and besides minor complaints it is generally well supported. Only small quantities of IL-2 are required. Hence the therapy is relatively cheap. A single IL-2 application of 4.5 million U IL-2 costs about 70 Euros. Thus combined local treatment may offer an alternative in those circumstances when more expensive forms of treatment are not available, for instance in resource poor countries

Evenness mediates the global relationship between forest productivity and richness

1. Biodiversity is an important component of natural ecosystems, with higher species richness often correlating with an increase in ecosystem productivity. Yet, this relationship varies substantially across environments, typically becoming less pronounced at high levels of species richness. However, species richness alone cannot reflect all important properties of a community, including community evenness, which may mediate the relationship between biodiversity and productivity. If the evenness of a community correlates negatively with richness across forests globally, then a greater number of species may not always increase overall diversity and productivity of the system. Theoretical work and local empirical studies have shown that the effect of evenness on ecosystem functioning may be especially strong at high richness levels, yet the consistency of this remains untested at a global scale. 2. Here, we used a dataset of forests from across the globe, which includes composition, biomass accumulation and net primary productivity, to explore whether productivity correlates with community evenness and richness in a way that evenness appears to buffer the effect of richness. Specifically, we evaluated whether low levels of evenness in speciose communities correlate with the attenuation of the richness–productivity relationship. 3. We found that tree species richness and evenness are negatively correlated across forests globally, with highly speciose forests typically comprising a few dominant and many rare species. Furthermore, we found that the correlation between diversity and productivity changes with evenness: at low richness, uneven communities are more productive, while at high richness, even communities are more productive. 4. Synthesis. Collectively, these results demonstrate that evenness is an integral component of the relationship between biodiversity and productivity, and that the attenuating effect of richness on forest productivity might be partly explained by low evenness in speciose communities. Productivity generally increases with species richness, until reduced evenness limits the overall increases in community diversity. Our research suggests that evenness is a fundamental component of biodiversity–ecosystem function relationships, and is of critical importance for guiding conservation and sustainable ecosystem management decisions