Monitoring with uncertainty

We discuss the problem of runtime verification of an instrumented program that misses to emit and to monitor some events. These gaps can occur when a monitoring overhead control mechanism is introduced to disable the monitor of an application with real-time constraints. We show how to use statistical models to learn the application behavior and to "fill in" the introduced gaps. Finally, we present and discuss some techniques developed in the last three years to estimate the probability that a property of interest is violated in the presence of an incomplete trace.Comment: In Proceedings HAS 2013, arXiv:1308.490

Real-Time Recurrent Reinforcement Learning

In this paper we propose real-time recurrent reinforcement learning (RTRRL), a biologically plausible approach to solving discrete and continuous control tasks in partially-observable markov decision processes (POMDPs). RTRRL consists of three parts: (1) a Meta-RL RNN architecture, implementing on its own an actor-critic algorithm; (2) an outer reinforcement learning algorithm, exploiting temporal difference learning and dutch eligibility traces to train the Meta-RL network; and (3) random-feedback local-online (RFLO) learning, an online automatic differentiation algorithm for computing the gradients with respect to parameters of the network.Our experimental results show that by replacing the optimization algorithm in RTRRL with the biologically implausible back propagation through time (BPTT), or real-time recurrent learning (RTRL), one does not improve returns, while matching the computational complexity for BPTT, and even increasing complexity for RTRL. RTRRL thus serves as a model of learning in biological neural networks, mimicking reward pathways in the basal ganglia.Comment: 14 pages, 9 figures, includes Appendi

Shared Variables Interaction Diagrams

Scenario-based specifications offer an intuitive and visual way of describing design requirements of distributed software systems. For the communication paradigm based on messages, message sequence charts (MSC) offer a standardized and formal notation amenable to formal analysis. In this paper, we define shared variables interaction diagrams (SVID) as the counterpart of MSCs when processes communicate via shared variables. After formally defining SVIDs, we develop an intuitive as well as formal definition of refinement for SVIDs. This notion provides a basis for systematically adding details to SVID requirements