A Certified Procedure for RL Verification

International audienceProving programs correct is hard. During the last decades computer scientists developed various logics dedicated to program verification. One such effort is Reachability Logic (RL): a language-parametric generalisation of Hoare Logic. Recently, based on RL, an automatic verification procedure was given and proved sound. In this paper we generalise this procedure and prove its soundness formally in the Coq proof assistant. For the formalisation we had to deal with all the minutiae that were neglected in the paper proof (i.e., an insufficient assumption, implicit hypotheses, and a missing case in the paper proof). The Coq formalisation provides us with a certified program-verification procedure

La preuve formelle de la correction pour un demonstrateur RL

Proving programs correct is one of the major challenges that computer scientists have been struggling with during the last decades.For this purpose, Reachability Logic (RL) was proposed as a language-parametric generalisation of Hoare Logic. Recently, based on RL, an automatic verification procedure was given and proved sound. In this paper we generalise this procedure and prove its soundness formally in the Coq proof assistant. For the formalisation we had to deal with all the minutiae that were neglected in the paper proof. The trickiest one was appropriate renaming of free variables which, we discovered, was handled in the paper proof using an insufficient assumption.We also discovered a missing case in the paper proof, and we clarified some implicit and hidden hypotheses.Last but not least, the Coq formalisation provides us with a certified program-verification procedure

Certified Reinforcement Learning with Logic Guidance

This paper proposes the first model-free Reinforcement Learning (RL) framework to synthesise policies for unknown, and continuous-state Markov Decision Processes (MDPs), such that a given linear temporal property is satisfied. We convert the given property into a Limit Deterministic Buchi Automaton (LDBA), namely a finite-state machine expressing the property. Exploiting the structure of the LDBA, we shape a synchronous reward function on-the-fly, so that an RL algorithm can synthesise a policy resulting in traces that probabilistically satisfy the linear temporal property. This probability (certificate) is also calculated in parallel with policy learning when the state space of the MDP is finite: as such, the RL algorithm produces a policy that is certified with respect to the property. Under the assumption of finite state space, theoretical guarantees are provided on the convergence of the RL algorithm to an optimal policy, maximising the above probability. We also show that our method produces ''best available'' control policies when the logical property cannot be satisfied. In the general case of a continuous state space, we propose a neural network architecture for RL and we empirically show that the algorithm finds satisfying policies, if there exist such policies. The performance of the proposed framework is evaluated via a set of numerical examples and benchmarks, where we observe an improvement of one order of magnitude in the number of iterations required for the policy synthesis, compared to existing approaches whenever available.Comment: This article draws from arXiv:1801.08099, arXiv:1809.0782

Shuttle orbiter Ku-band radar/communications system design evaluation

Tasks performed in an examination and critique of a Ku-band radar communications system for the shuttle orbiter are reported. Topics cover: (1) Ku-band high gain antenna/widebeam horn design evaluation; (2) evaluation of the Ku-band SPA and EA-1 LRU software; (3) system test evaluation; (4) critical design review and development test evaluation; (5) Ku-band bent pipe channel performance evaluation; (6) Ku-band LRU interchangeability analysis; and (7) deliverable test equipment evaluation. Where discrepancies were found, modifications and improvements to the Ku-band system and the associated test procedures are suggested

Data management study, volume 5. Appendix F - Contractor data package test /TE/ and mission operations /MP/ Final report

Test and mission operations contractor data package for Voyager spacecraf

Performance criteria guideline for three explosion protection methods of electrical equipment rated up to 15,000 volts AC

The Bureau of Mines, U.S. Department of the Interior, is reviewing explosion protection methods for use in gassy coal mines. This performance criteria guideline is an evaluation of three explosion protection methods of machines electrically powered with voltages up to 15,000 volts ac. A sufficient amount of basic research has been accomplished to verify that the explosion proof and pressurized enclosure methods can provide adequate explosion protection with the present state of the art up to 15,000 volts ac. This routine application of the potted enclosure as a stand alone protection method requires further investigation or development in order to clarify performance criteria and verification certification requirements. An extensive literature search, a series of high voltage tests, and a design evaluation of the three explosion protection methods indicate that the explosion proof, pressurized, and potted enclosures can all be used to enclose up to 15,000 volts ac