Testing End-To-End Chains using Domain Specific Languages

Testing systems is a time consuming (and hence expensive) activity. Nevertheless, it is a very important and necessary step before using systems, especially safety critical systems. Therefore, many different test procedures are used: Unit Tests, Black-Box Tests, Software Integration Tests (SWI), Hardware-Software Integration Tests (HSI), Hardware-In-The-Loop Tests, just to name a few. Especially in the avionics domain, a variety of systems and applications communicate which each other. Furthermore, they depend and rely on the received information. However, some faults are only detected when all systems are connected and in operational mode. A new testing approach is to create model based End-To-End Chain scenarios with original and simulated equipment in any combination. The first aim is to automatically derive test data and test cases from the model, which is defined by a Domain Specific Language (DSL). Test data generators can be attached to quickly create a variety of stimuli for the systems under test. Furthermore, the system under test can be stimulated by either original equipment â which is connected to the test bench â or the test bench can simulate equipment and create inputs for the tested systems. Any mixture of simulated and original equipment is possible and can be changed on the fly. In the end, the results from the system under test are collected. These results can then be displayed back in the model. This method was used and improved in the project â E-Enabled Cabin (E-Cab)â in which the author was involved. Passengers traveling by plane are in the focus of this project. Complete services and service chains â ranging from booking at home with a computer, being en route using mobile devices to leaving the destination airport â are created and used by many systems communicating with each other. A special focus is set on a guidance scenario at an airport. The user of the system signs in to a guidance and notification system which will inform him via his own digital equipment (mobile device/smart phone). The system notifies the user about his in-time status. Either he is in-time for his flight, or he needs to hurry up and proceed to the next area or he will be too late and cannot catch his flight. The DSL itself is designed according to the comprehension of information processes. The ability of the human brain to process visual information in parallel â in contrast to sequential processing of textual information â is described and applied in the design of the DSL and the concepts of the project. The development of the DSL and the workflow is developed with the â real worldâ in mind. This means that the work fits in established workflows and enhances the current situation. As this project took place in the aircraft industry, the appropriate development standards, like DO178B and ABD100/200 (Airbus Directives ABD100/ABD200), build the foundation. The generation of clean code is established by applying generator guidelines (through coding standards) in order to create maintainable tests and test data

Using DSLs for software testing

In her widely cited paper about the future of software testing, Bertolino claims that domain specific languages (DSLs) have emerged as an efficient solution towards allowing experts within a domain to express specifications in that domain. She goes on the claim that success of domain-specific approaches should be built upon and extended to the testing stage of software engineering. An intuitive place to start would be to explore DSLs in the context of software testing such that languages constructed by domain experts can be leveraged to specify not only requirements but also test cases which validate those requirements. In this talk, we present and discuss the outcomes of three exploratory case studies which we carried out in order to investigate the utility of DSLs as applied to specifying tests in different domains, with each case study focusing on a particular aspect/characteristic of this application of DSLs.peer-reviewe

Unit testing for domain-specific languages, in

Abstract. Domain-specific languages (DSLs) offer several advantages by providing idioms that are similar to the abstractions found in a specific problem domain. However, a challenge is that tool support for DSLs is lacking when compared to the capabilities offered in general-purpose languages (GPLs), such as Java and C++. For example, support for unit testing a DSL program is absent and debuggers for DSLs are rare. This limits the ability of a developer to discover the existence of software errors and to locate them in a DSL program. Currently, software developers using a DSL are generally forced to test and debug their DSL programs using available GPL tools, rather than tools that are informed by the domain abstractions at the DSL level. This reduces the utility of DSL adoption and minimizes the benefits of working with higher abstractions, which can bring into question the suitability of using DSLs in the development process. This paper introduces our initial investigation into a unit testing framework that can be customized for specific DSLs through a reusable mapping of GPL testing tool functionality. We provide examples from two different DSL categories that serve as case studies demonstrating the possibilities of a unit testing engine for DSLs

Automated model-based testing based on an agnostic-platform modeling language

Currently multiple Domain Specific Languages (DSLs) are used for model-driven software development, in some specific domains. Software development methods, such as agile development, are test-centered, and their application in model-based frameworks requires model support for test development. We introduce a specific language to define generic test models, which can be automatically transformed into executable tests for particular testing platforms. The resulting test models represent the test plan for applications also built according to a model-based approach. The approach presented here includes some customisations for the application of the developed languages and transformation tools for some specific testing platforms. These languages and tools have been integrated with some specific DSL designed for software development

Lessons learnt from using DSLs for automated software testing

Domain Specific Languages (DSLs) provide a means of unambiguously expressing concepts in a particular domain. Although they may not refer to it as such, companies build and maintain DSLs for software testing on a day-to-day basis, especially when they define test suites using the Gherkin language. However, although the practice of specifying and automating test cases using the Gherkin language and related technologies such as Cucumber has become mainstream, the curation of such languages presents a number of challenges. In this paper we discuss lessons learnt from five case studies on industry systems, two involving the use of Gherkin-type syntax and another three case studies using more rigidly defined language grammars. Initial observations indicate that the likelihood of success of such efforts is increased if one manages to use an approach which separates the concerns of domain experts who curate the language, users who write scripts with the language, and engineers who wire the language into test automation technologies thus producing executable test code. We also provide some insights into desirable qualities of testing DSLs in different contexts.peer-reviewe

Developing Embedded Multithreaded Applications with CATAPULTS, a Domain-specific Language for Generating Thread Schedulers

This paper describes CATAPULTS, a domain-specific language for creating and testing application-specific user level thread schedulers. Using a domain-specific language to write thread schedulers provides three advantages. First, it modularizes the thread scheduler, making it easy to plug in and experiment with different schedulers. Second, using a domain-specific language for scheduling code helps prevent several of the common programming mistakes that are easy to make when programming in low-level C or assembly. Finally, the CATAPULTS translator has multiple backends that generate code for different languages and libraries. This makes it easy to prototype an embedded application on a regular PC, and then develop the final version on the embedded hardware; the CATAPULTS translator will take care of generating the appropriate code for both the PC prototype and the final embedded version of the program. Using our implementation of CATAPULTS for Z-World’s embedded Rabbit processors, we obtained a performance gain of about 12.6 % at the expense of about 12.7 % increase in code size for a fairly typical embedded application

Scenic: A Language for Scenario Specification and Scene Generation

We propose a new probabilistic programming language for the design and analysis of perception systems, especially those based on machine learning. Specifically, we consider the problems of training a perception system to handle rare events, testing its performance under different conditions, and debugging failures. We show how a probabilistic programming language can help address these problems by specifying distributions encoding interesting types of inputs and sampling these to generate specialized training and test sets. More generally, such languages can be used for cyber-physical systems and robotics to write environment models, an essential prerequisite to any formal analysis. In this paper, we focus on systems like autonomous cars and robots, whose environment is a "scene", a configuration of physical objects and agents. We design a domain-specific language, Scenic, for describing "scenarios" that are distributions over scenes. As a probabilistic programming language, Scenic allows assigning distributions to features of the scene, as well as declaratively imposing hard and soft constraints over the scene. We develop specialized techniques for sampling from the resulting distribution, taking advantage of the structure provided by Scenic's domain-specific syntax. Finally, we apply Scenic in a case study on a convolutional neural network designed to detect cars in road images, improving its performance beyond that achieved by state-of-the-art synthetic data generation methods.Comment: 41 pages, 36 figures. Full version of a PLDI 2019 paper (extending UC Berkeley EECS Department Tech Report No. UCB/EECS-2018-8