Testbench qualification of SystemC TLM protocols through Mutation Analysis

Transaction-level modeling (TLM) has become the de-facto reference modeling style for system-level design and verification of embedded systems. It allows designers to implement high-level communication protocols for simulations up to 1000x faster than at register-transfer level (RTL). To guarantee interoperability between TLM IP suppliers and users, designers implement the TLM communication protocols by relying on a reference standard, such as the standard OSCI for SystemC TLM. Functional correctness of such protocols as well as their compliance to the reference TLM standard are usually verified through user-defined testbenches, which high-quality and completeness play a key role for an efficient TLM design and verification flow. This article presents a methodology to apply mutation analysis, a technique applied in literature for SW testing, for measuring the testbench quality in verifying TLM protocols. In particular, the methodology aims at (i) qualifying the testbenches by considering both the TLM protocol correctness and their compliance to a defined standard (i.e., OSCI TLM), (ii) optimizing the simulation time during mutation analysis by avoiding mutation redundancies, and (iii) driving the designers in the testbench improvement. Experimental results on benchmarks of different complexity and architectural characteristics are reported to analyze the methodology applicability

On the Mutation Analysis of SystemC TLM-2.0 Standard

Transaction-level modeling (TLM) is the most promising technique to deal with the increasing complexity of modern embedded systems. TLM provides designers with high-level interfaces and communication protocols for abstract modeling and efficient simulation of system platforms involving both hardware and software components. The Open SystemC Initiative (OSCI) has recently released the TLM-2.0 standard, to standardize the interface between component models for bus-based systems. The standard TLM aims at facilitating the interchange of models between suppliers and users, and thus encouraging the use of virtual platforms for fast simulation prior to the availability of register-transfer level (RTL) code. This paper discusses on the mutation analysis concept applied to the TLM context and proposes a mutation model for perturbing TLM SystemC descriptions. In particular, the main constructs provided by the latest OSCI TLM-2.0 standard are analyzed, and a set of mutants is proposed to perturb the primitives related to the TLM communication interfaces