Tester versus Bug: A Generic Framework for Model-Based Testing via Games

We propose a generic game-based approach for test case generation. We set up a game between the tester and the System Under Test, in such a way that test cases correspond to game strategies, and the conformance relation ioco corresponds to alternating refinement. We show that different test assumptions from the literature can be easily incorporated, by slightly varying the moves in the games and their outcomes. In this way, our framework allows a wide plethora of game-theoretic techniques to be deployed for model based testing.Comment: In Proceedings GandALF 2018, arXiv:1809.0241

Automated unique input output sequence generation for conformance testing of FSMs

This paper describes a method for automatically generating unique input output (UIO) sequences for FSM conformance testing. UIOs are used in conformance testing to verify the end state of a transition sequence. UIO sequence generation is represented as a search problem and genetic algorithms are used to search this space. Empirical evidence indicates that the proposed method yields considerably better (up to 62% better) results compared with random UIO sequence generation

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Adaptive Intelligent Tutoring System for learning Computer Theory

In this paper, we present an intelligent tutoring system developed to help students in learning Computer Theory. The Intelligent tutoring system was built using ITSB authoring tool. The system helps students to learn finite automata, pushdown automata, Turing machines and examines the relationship between these automata and formal languages, deterministic and nondeterministic machines, regular expressions, context free grammars, undecidability, and complexity. During the process the intelligent tutoring system gives assistance and feedback of many types in an intelligent manner according to the behavior of the student. An evaluation of the intelligent tutoring system has revealed reasonably acceptable results in terms of its usability and learning abilities are concerned

State Identification for Labeled Transition Systems with Inputs and Outputs

For Finite State Machines (FSMs) a rich testing theory has been developed to discover aspects of their behavior and ensure their correct functioning. Although this theory is widely used, e.g., to check conformance of protocol implementations, its applicability is limited by restrictions of the FSM framework: the fact that inputs and outputs alternate in an FSM, and outputs are fully determined by the previous input and state. Labeled Transition Systems with inputs and outputs (LTSs), as studied in ioco testing theory, provide a richer framework for testing component oriented systems, but lack the algorithms for test generation from FSM theory. In this article, we propose an algorithm for the fundamental problem of state identification during testing of LTSs. Our algorithm is a direct generalization of the well-known algorithm for computing adaptive distinguishing sequences for FSMs proposed by Lee & Yannakakis. Our algorithm has to deal with so-called compatible states, states that cannot be distinguished in case of an adversarial system-under-test. Analogous to the result of Lee & Yannakakis, we prove that if an (adaptive) test exists that distinguishes all pairs of incompatible states of an LTS, our algorithm will find one. In practice, such adaptive tests typically do not exist. However, in experiments with an implementation of our algorithm on an industrial benchmark, we find that tests produced by our algorithm still distinguish more than 99% of the incompatible state pairs

Robotic sensorimotor interaction strategies

Abstract. In this thesis we investigate the mathematical modeling of cognition using sensorimotor transition systems. The focus of the thesis is enactivism, where an agent learns to think through actions. As a theoretical basis for our implementation, we discuss a mathematical model of enactivist cognition, sensorimotor interaction and how they can be used as algorithmic aides for studying theoretical problems in robotic systems. In Chapter 3 of this thesis, we introduce a platform which was developed in the University of Oulu as a software project and explain how enactivism and sensorimotor interaction have been taken advantage of, in developing a 2D platform. This platform enables one to concretely implement and explore different interaction strategies that allow an agent to construct internal models of its surroundings. The agent in the platform is a multi-jointed robotic arm, which maneuvers through an obstacle-filled environment. The robotic arm tries to explore its environment with minimal sensory feedback, using algorithms created by the user of the platform. Our main goal on this thesis is to implement new features to this platform. We implement a memory functionality which allows the robotic arm to store all its performed actions. The memory helps the agent infer to a greater extent its surroundings from a limited sequence of action-observation pairs, and helps it in getting a better grasp of the environment. In addition, we implement other methods and functionalities, such as an obstacle sensor, a graph visualization of the internal models, etc. to enhance the perceptual ability of the robotic arm. In Section 5, we develop an algorithm for a simple 2D environment with no obstacles. Here the robotic arm makes a 360-degree move in four steps to perceive its surroundings and generates a state machine graph to visualize its internal model of the environment. The goal of the algorithm is to build an accurate representation of the environment with the help of memory. Through this algorithm we are able to evaluate the performance of the newly implemented features. We also test the platform through unit testing for finding and resolving bugs.Sensorimotoriset vuorovaikutusstrategiat robotiikassa. Tiivistelmä. Tässä tutkielmassa tutkimme kognition matemaattista mallintamista käyttämällä sensorimotorisia transitio-järjestelmiä. Tutkielman keskiössä on enaktivismi, jossa agentti oppii ajattelemaan toiminnan kautta. Teoreettisena perustana toteutuksellemme käsittelemme matemaattista mallia enaktivistisesta kognitiosta, sensorimotorista vuorovaikutusta ja kuinka niitä voidaan käyttää algoritmien apuvälineinä teoreettisten ongelmien tutkimisessa robottiikkajärjestelmissä. Tutkielman luvussa 3 esittelemme alustan, joka on kehitetty Oulun yliopistossa ohjelmistoprojektina, ja selittämme miten enaktivismia ja sensomotorista vuorovaikutusta on hyödynnetty 2D-alustan kehittämisessä. Alusta mahdollistaa erilaisten vuorovaikutusstrategioiden konkreettisen toteuttamisen ja tutkimisen. Näiden avulla agentti rakentaa sisäisiä malleja ympäristöstään. Alustassa mallinnettu agentti on moninivelinen robottikäsi, joka liikkuu esteitä sisältävässä ympäristössä. Robottikäsi pyrkii tutkimaan ympäristöään minimaalisen sensoritiedon avulla käyttämällä alustan käyttäjän luomia algoritmeja. Tutkielmamme päätavoite on kehittää uusia ominaisuuksia tälle alustalle. Toteutamme muistitoiminnallisuuden, jonka avulla robottikäsi tallentaa kaikki suoritetut toiminnot. Muisti auttaa agenttia päättelemään enemmän ympäristöstään rajoitettujen toiminta-havainto-parien avulla, ja auttaa sitä ympäristön hahmottamisessa. Lisäksi kehitämme muita menetelmiä ja toiminnallisuuksia kuten estesensorin ja sisäisten mallien graafisen visualisoinnin parantaaksemme robottikäden havainnointikykyä. Tutkielman myöhemmässä osassa kehitämme algoritmin yksinkertaiselle 2D-ympäristölle ilman esteitä. Siinä robottikäsi tekee 360 asteen liikkeen neljässä vaiheessa havainnoidakseen ympäristönsä, ja luo tilasiirtymäkaavion visualisoidakseen sisäisen mallinsa ympäristöstä. Algoritmin tavoitteena on rakentaa tarkka malli ympäristöstä muistin avulla. Tämän algoritmin avulla pystymme arvioimaan kehittämiemme uusien ominaisuuksien toimintaa. Testaamme alustaa myös yksikkötesteillä löytääksemme ja korjataksemme virheitä

Serious Gaming for Building a Basis of Certification via Trust and Trustworthiness of Autonomous Systems

Autonomous systems governed by a variety of adaptive and nondeterministic algorithms are being planned for inclusion into safety-critical environments, such as unmanned aircraft and space systems in both civilian and military applications. However, until autonomous systems are proven and perceived to be capable and resilient in the face of unanticipated conditions, humans will be reluctant or unable to delegate authority, remaining in control aided by machine-based information and decision support. Proving capability, or trustworthiness, is a necessary component of certification. Perceived capability is a component of trust. Trustworthiness is an attribute of a cyber-physical system that requires context-driven metrics to prove and certify. Trust is an attribute of the agents participating in the system and is gained over time and multiple interactions through trustworthy behavior and transparency. Historically, artificial intelligence and machine learning systems provide answers without explanation - without a rationale or insight into the machine thinking. In order to function as trusted teammates, machines must be able to explain their decisions and actions. This transparency is a product of both content and communication. NASAs Autonomy Teaming & TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project seeks to build a basis for certification of autonomous systems via establishing metrics for trustworthiness and trust in multi-agent team interactions, using AI (Artificial Intelligence) explainability and persistent modeling and simulation, in the context of mission planning and execution, with analyzable trajectories. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ATTRACTOR modeling and simulation environment is similar to online gaming environments in which player (aka agent) participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual agents active participation. This persistent simulation environment will accommodate individual agents, groups of self-organizing agents, and large-scale infrastructure behavior. The effects of the emerging adaptation and coevolution can be observed and measured to building a basis of measurable trustworthiness and trust, toward certification of safety-critical autonomous systems