Waveform Approach for Assessing Conformity of CISPR 16-1-1 Measuring Receivers

An alternative approach for assessing the conformity of electromagnetic interference measuring receivers with respect to the baseline CISPR 16-1-1 requirements is proposed. The method’s core is based on the generation of digitally synthesized complex waveforms comprising multisine excitation signals and modulated pulses. The superposition of multiple narrowband reference signals populating the standard frequency bands allows for a single-stage evaluation of the receiver’s voltage accuracy and frequency selectivity. Moreover, characterizing the response of the weighting detectors using modulated pulses is more repeatable and less restrictive than the conventional approach. This methodology significantly reduces the amount of time required to complete the verification of the receiver’s baseline magnitudes, because time-domain measurements enable a broadband assessment while the typical calibration methodology follows the time-consuming narrow band frequency sweep scheme. Since the reference signals are generated using arbitrary waveform generators, they can be easily reproduced from a standard numerical vector. For different test receivers, the results of such assessment are presented in the 9 kHz–1 GHz frequency range. Finally, a discussion on the measurement uncertainty of this methodology for assessing measuring receivers is given.Postprint (author's final draft

What Am I Testing and Where? Comparing Testing Procedures based on Lightweight Requirements Annotations

[Context] The testing of software-intensive systems is performed in different test stages each having a large number of test cases. These test cases are commonly derived from requirements. Each test stages exhibits specific demands and constraints with respect to their degree of detail and what can be tested. Therefore, specific test suites are defined for each test stage. In this paper, the focus is on the domain of embedded systems, where, among others, typical test stages are Software- and Hardware-in-the-loop. [Objective] Monitoring and controlling which requirements are verified in which detail and in which test stage is a challenge for engineers. However, this information is necessary to assure a certain test coverage, to minimize redundant testing procedures, and to avoid inconsistencies between test stages. In addition, engineers are reluctant to state their requirements in terms of structured languages or models that would facilitate the relation of requirements to test executions. [Method] With our approach, we close the gap between requirements specifications and test executions. Previously, we have proposed a lightweight markup language for requirements which provides a set of annotations that can be applied to natural language requirements. The annotations are mapped to events and signals in test executions. As a result, meaningful insights from a set of test executions can be directly related to artifacts in the requirements specification. In this paper, we use the markup language to compare different test stages with one another. [Results] We annotate 443 natural language requirements of a driver assistance system with the means of our lightweight markup language. The annotations are then linked to 1300 test executions from a simulation environment and 53 test executions from test drives with human drivers. Based on the annotations, we are able to analyze how similar the test stages are and how well test stages and test cases are aligned with the requirements. Further, we highlight the general applicability of our approach through this extensive experimental evaluation. [Conclusion] With our approach, the results of several test levels are linked to the requirements and enable the evaluation of complex test executions. By this means, practitioners can easily evaluate how well a systems performs with regards to its specification and, additionally, can reason about the expressiveness of the applied test stage.TU Berlin, Open-Access-Mittel - 202

Using formal methods to support testing

Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing

Uncertainty-Driven Black-Box Test Data Generation

We can never be certain that a software system is correct simply by testing it, but with every additional successful test we become less uncertain about its correctness. In absence of source code or elaborate specifications and models, tests are usually generated or chosen randomly. However, rather than randomly choosing tests, it would be preferable to choose those tests that decrease our uncertainty about correctness the most. In order to guide test generation, we apply what is referred to in Machine Learning as "Query Strategy Framework": We infer a behavioural model of the system under test and select those tests which the inferred model is "least certain" about. Running these tests on the system under test thus directly targets those parts about which tests so far have failed to inform the model. We provide an implementation that uses a genetic programming engine for model inference in order to enable an uncertainty sampling technique known as "query by committee", and evaluate it on eight subject systems from the Apache Commons Math framework and JodaTime. The results indicate that test generation using uncertainty sampling outperforms conventional and Adaptive Random Testing

Requirements traceability in model-driven development: Applying model and transformation conformance

The variety of design artifacts (models) produced in a model-driven design process results in an intricate relationship between requirements and the various models. This paper proposes a methodological framework that simplifies management of this relationship, which helps in assessing the quality of models, realizations and transformation specifications. Our framework is a basis for understanding requirements traceability in model-driven development, as well as for the design of tools that support requirements traceability in model-driven development processes. We propose a notion of conformance between application models which reduces the effort needed for assessment activities. We discuss how this notion of conformance can be integrated with model transformations

Increasing System Test Coverage in Production Automation Systems

An approach is introduced, which supports a testing technician in the identification of possibly untested behavior of control software of fully integrated automated production systems (aPS). Based on an approach for guided semi-automatic system testing, execution traces are recorded during testing, allowing a subsequent coverage assessment. As the behavior of an aPS is highly dependent on the software, omitted system behavior can be identified and assessed for criticality. Through close cooperation with industry, this approach represents the first coverage assessment approach for system testing in production automation to be applied on real industrial objects and evaluated by industrial experts

A research review of quality assessment for software

Measures were recommended to assess the quality of software submitted to the AdaNet program. The quality factors that are important to software reuse are explored and methods of evaluating those factors are discussed. Quality factors important to software reuse are: correctness, reliability, verifiability, understandability, modifiability, and certifiability. Certifiability is included because the documentation of many factors about a software component such as its efficiency, portability, and development history, constitute a class for factors important to some users, not important at all to other, and impossible for AdaNet to distinguish between a priori. The quality factors may be assessed in different ways. There are a few quantitative measures which have been shown to indicate software quality. However, it is believed that there exists many factors that indicate quality and have not been empirically validated due to their subjective nature. These subjective factors are characterized by the way in which they support the software engineering principles of abstraction, information hiding, modularity, localization, confirmability, uniformity, and completeness