DeepSynth: Automata synthesis for automatic task segmentation in deep reinforcement learning

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of compact automata to uncover this sequential structure automatically. We synthesise a humaninterpretable automaton from trace data collected by exploring the environment. The state space of the environment is then enriched with the synthesised automaton so that the generation of a control policy by deep RL is guided by the discovered structure encoded in the automaton. The proposed approach is able to cope with both high-dimensional, low-level features and unknown sparse non-Markovian rewards. We have evaluated DeepSynth’s performance in a set of experiments that includes the Atari game Montezuma’s Revenge. Compared to existing approaches, we obtain a reduction of two orders of magnitude in the number of iterations required for policy synthesis, and also a significant improvement in scalability

Buffalo law review

Abstract. We present a SAT-based algorithm for assisting users of Symbolic Trajectory Evaluation (STE) in manual abstraction refinement. As a case study, we demonstrate the usefulness of the algorithm by showing how to refine and verify an STE specification of a CAM.

Combining higher order abstract syntax with tactical theorem proving and (co)induction

Abstract. Combining Higher Order Abstract Syntax (HOAS) and induction is well known to be problematic. We have implemented a tool called Hybrid, within Isabelle HOL, which does allow object logics to be represented using HOAS, and reasoned about using tactical theorem proving in general and principles of (co)induction in particular. In this paper we describe Hybrid, and illustrate its use with case studies. We also provide some theoretical adequacy results which underpin our practical work. 1 Introduction Many people are concerned with the development of computing systems which can be used to reason about and prove properties of programming languages. However, developing such systems is not easy. Difficulties abound in both practical implementation and underpinning theory. Our paper makes both a theoretical and practical contribution to this research area. More precisely, this paper concerns how to reason about object level logics with syntax involving variable binding--note that a programming language can be presented as an example of such an object logic. Our contribution is the provision of a mechanized tool, Hybrid, which has been coded within Isabelle HOL, and- provides a form of logical framework within which the syntax of an objec

Applications of Polytypism in Theorem Proving

Abstract. Polytypic functions have mainly been studied in the context of functional programming languages. In that setting, applications of polytypism include elegant treatments of polymorphic equality, prettyprinting, and the encoding and decoding of high-level datatypes to and from low-level binary formats. In this paper, we discuss how polytypism supports some aspects of theorem proving: automated termination proofs of recursive functions, incorporation of the results of metalanguage evaluation, and equivalence-preserving translation to a low-level format suitable for propositional methods. The approach is based on an interpretation of higher order logic types as terms, and easily deals with mutual and nested recursive types.

Open-Label Phase II Prospective, Randomized, Controlled Study of Romyelocel-L Myeloid Progenitor Cells to Reduce Infection During Induction Chemotherapy for Acute Myeloid Leukemia.

PurposeStandard cytotoxic induction chemotherapy for acute myeloid leukemia (AML) results in prolonged neutropenia and risk of infection. Romyelocel-L is a universal, allogeneic myeloid progenitor cell product being studied to reduce infection during induction chemotherapy.Patients and methodsOne hundred sixty-three patients with de novo AML (age ≥ 55 years) receiving induction chemotherapy were randomly assigned on day 0 (d0), of whom 120 were evaluable. Subjects received either romyelocel-L infusion on d9 with granulocyte colony-stimulating factor (G-CSF) starting daily d14 (treatment group) or G-CSF daily alone on d14 (control) until absolute neutrophil count recovery to 500/µL. End points included days in febrile episode, microbiologically defined infections, clinically diagnosed infection, and days in hospital.ResultsMean days in febrile episode was shorter in the treatment arm from d15 through d28 (2.36 v 3.90; P = .02). Similarly, a trend toward decreased microbiologically defined infections and clinically diagnosed infection in the treatment arm was observed from d9 to d28 (35.6% v 47.5%; P = .09), reaching a statistically significant difference from d15 to d28 (6.8% v 27.9%; P = .002). Because of this, antibacterial or antifungal use for treatment of an infection was significantly less in the treatment group (d9-d28: 44.1% v 63.9%; P = .01). Significantly fewer patients in the treatment arm received empiric antifungals from d9 tod28 (42.4% v 63.9%; P = .02) and d15-d28 (42.4% v 62.3%; P = .02). Patients in the treatment arm also had 3.2 fewer hospital days compared with control (25.5 v 28.7; P = .001). Remission rates and days to absolute neutrophil count recovery were similar in the two groups. No patients in the romyelocel-L plus G-CSF group died because of infection compared with two patients in the control arm. No graft-versus-host disease was observed.ConclusionSubjects receiving romyelocel-L showed a decreased incidence of infections, antimicrobial use, and hospitalization, suggesting that romyelocel-L may provide a new option to reduce infections in patients with AML undergoing induction therapy

Open-Label Phase II Prospective, Randomized, Controlled Study of Romyelocel-L Myeloid Progenitor Cells to Reduce Infection During Induction Chemotherapy for Acute Myeloid Leukemia.

PURPOSE: Standard cytotoxic induction chemotherapy for acute myeloid leukemia (AML) results in prolonged neutropenia and risk of infection. Romyelocel-L is a universal, allogeneic myeloid progenitor cell product being studied to reduce infection during induction chemotherapy. PATIENTS AND METHODS: One hundred sixty-three patients with de novo AML (age ≥ 55 years) receiving induction chemotherapy were randomly assigned on day 0 (d0), of whom 120 were evaluable. Subjects received either romyelocel-L infusion on d9 with granulocyte colony-stimulating factor (G-CSF) starting daily d14 (treatment group) or G-CSF daily alone on d14 (control) until absolute neutrophil count recovery to 500/µL. End points included days in febrile episode, microbiologically defined infections, clinically diagnosed infection, and days in hospital. RESULTS: Mean days in febrile episode was shorter in the treatment arm from d15 through d28 (2.36 CONCLUSION: Subjects receiving romyelocel-L showed a decreased incidence of infections, antimicrobial use, and hospitalization, suggesting that romyelocel-L may provide a new option to reduce infections in patients with AML undergoing induction therapy