System Design for Digital Experimentation and Explanation Generation

Experimentation increasingly drives everyday decisions in modern life, as it is considered by some to be the gold standard for determining cause and effect within any system. Digital experiments have expanded the scope and frequency of experiments, which can range in complexity from classic A/B tests to contextual bandits experiments, which share features with reinforcement learning. Although there exists a large body of prior work on estimating treatment effects using experiments, this prior work did not anticipate the new challenges and opportu- nities introduced by digital experimentation. Novel errors and threats to validity arise at the intersection of software and experimentation, especially when experimentation is in service of understanding humans behavior or autonomous black-box agents. We present several novel tools for automating aspects of the experimentation- analysis pipeline. We propose new methods for evaluating online field experimentation, automatically generating corresponding analyses of treatment effects. We then draw the connection between software testing and experimental design and argue that applying software testing techniques to a kind of autonomous agent—a deep reinforcement learning agent—to demonstrate the need for novel testing paradigms when a software stack uses learned components that may have emergent behavior. We show how our system may be used to evaluate claims made about the behavior of autonomous agents and find that some claims do not hold up under test. Finally, we show how to produce explanations of the behavior of black-box software-defined agents interacting with white-box environments via automated experimentation. We show how an automated system can be used for exploratory data analysis, with a human in the loop, to investigate a large space of possible counterfactual explanations

Structural Testing of Active DataBases

Active databases (ADBs) are databases that include active components or agents that can execute actions. The rise of active databases in the picture of software development has a great impact on software systems and in the discipline of software engineering. However, we still lack the foundations that are needed to adequately support this new tool. These foundations are needed in order to properly apply known software engineering techniques to ADBs and systems that use them. Among the methods and techniques used to improve quality, we count systematic testing. In this work, we generalize structural testing techniques to ADB systems. We introduce a model of active databases, called dbgraph, suitable for testing. We show that dbgraphs can be used to generalize structural testing techniques for ADBs. Moreover, we introduce several new structural criteria aimed at find errors in a set of rules for an ADB. We also compare the strength of the coverage criteria presented in this work.Supported in part by UBACyT grant EX186.Sociedad Argentina de Informática e Investigación Operativ

GNSS Signal spoofing detection

This thesis elaborates on the implementation of spoofing detection techniques for GPS L1 C/A signals, topic which is up to the minute in the GNSS community. The interest of this topic has its origin on the fact that, currently, there is a large number of applications relying on GNSS communications. Moreover, the public character of the communication details and specifications have exposed the communications to spoofing agents, which, with a relatively cheap equipment, are capable of controlling the tracking loops of a victim receiver and, as a result, manipulate the its timing or navigation solution. In front of this issue, this project aims to contribute on the spoofing detection community by implementing, in the recognized Borre¿s GNSS receiver software, and testing some techniques. To do so, the project is organized in three sections; the preliminary study of the state of the art and the software that will be considered as the starting point, the spoofing signal analysis and the implementation of the selected spoofing detection techniques, and the result¿s evaluation

Modelling of Multi-Agent Systems: Experiences with Membrane Computing and Future Challenges

Formal modelling of Multi-Agent Systems (MAS) is a challenging task due to high complexity, interaction, parallelism and continuous change of roles and organisation between agents. In this paper we record our research experience on formal modelling of MAS. We review our research throughout the last decade, by describing the problems we have encountered and the decisions we have made towards resolving them and providing solutions. Much of this work involved membrane computing and classes of P Systems, such as Tissue and Population P Systems, targeted to the modelling of MAS whose dynamic structure is a prominent characteristic. More particularly, social insects (such as colonies of ants, bees, etc.), biology inspired swarms and systems with emergent behaviour are indicative examples for which we developed formal MAS models. Here, we aim to review our work and disseminate our findings to fellow researchers who might face similar challenges and, furthermore, to discuss important issues for advancing research on the application of membrane computing in MAS modelling.Comment: In Proceedings AMCA-POP 2010, arXiv:1008.314

Model checking multi-agent systems

A multi-agent system (MAS) is usually understood as a system composed of interacting autonomous agents. In this sense, MAS have been employed successfully as a modelling paradigm in a number of scenarios, especially in Computer Science. However, the process of modelling complex and heterogeneous systems is intrinsically prone to errors: for this reason, computer scientists are typically concerned with the issue of verifying that a system actually behaves as it is supposed to, especially when a system is complex. Techniques have been developed to perform this task: testing is the most common technique, but in many circumstances a formal proof of correctness is needed. Techniques for formal verification include theorem proving and model checking. Model checking techniques, in particular, have been successfully employed in the formal verification of distributed systems, including hardware components, communication protocols, security protocols. In contrast to traditional distributed systems, formal verification techniques for MAS are still in their infancy, due to the more complex nature of agents, their autonomy, and the richer language used in the specification of properties. This thesis aims at making a contribution in the formal verification of properties of MAS via model checking. In particular, the following points are addressed: • Theoretical results about model checking methodologies for MAS, obtained by extending traditional methodologies based on Ordered Binary Decision Diagrams (OBDDS) for temporal logics to multi-modal logics for time, knowledge, correct behaviour, and strategies of agents. Complexity results for model checking these logics (and their symbolic representations). • Development of a software tool (MCMAS) that permits the specification and verification of MAS described in the formalism of interpreted systems. • Examples of application of MCMAS to various MAS scenarios (communication, anonymity, games, hardware diagnosability), including experimental results, and comparison with other tools available

Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration

Testing in Continuous Integration (CI) involves test case prioritization, selection, and execution at each cycle. Selecting the most promising test cases to detect bugs is hard if there are uncertainties on the impact of committed code changes or, if traceability links between code and tests are not available. This paper introduces Retecs, a new method for automatically learning test case selection and prioritization in CI with the goal to minimize the round-trip time between code commits and developer feedback on failed test cases. The Retecs method uses reinforcement learning to select and prioritize test cases according to their duration, previous last execution and failure history. In a constantly changing environment, where new test cases are created and obsolete test cases are deleted, the Retecs method learns to prioritize error-prone test cases higher under guidance of a reward function and by observing previous CI cycles. By applying Retecs on data extracted from three industrial case studies, we show for the first time that reinforcement learning enables fruitful automatic adaptive test case selection and prioritization in CI and regression testing.Comment: Spieker, H., Gotlieb, A., Marijan, D., & Mossige, M. (2017). Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration. In Proceedings of 26th International Symposium on Software Testing and Analysis (ISSTA'17) (pp. 12--22). AC

Practical applications of multi-agent systems in electric power systems

The transformation of energy networks from passive to active systems requires the embedding of intelligence within the network. One suitable approach to integrating distributed intelligent systems is multi-agent systems technology, where components of functionality run as autonomous agents capable of interaction through messaging. This provides loose coupling between components that can benefit the complex systems envisioned for the smart grid. This paper reviews the key milestones of demonstrated agent systems in the power industry and considers which aspects of agent design must still be addressed for widespread application of agent technology to occur