Model Checker Execution Reports

Software model checking constitutes an undecidable problem and, as such, even an ideal tool will in some cases fail to give a conclusive answer. In practice, software model checkers fail often and usually do not provide any information on what was effectively checked. The purpose of this work is to provide a conceptual framing to extend software model checkers in a way that allows users to access information about incomplete checks. We characterize the information that model checkers themselves can provide, in terms of analyzed traces, i.e. sequences of statements, and safe cones, and present the notion of execution reports, which we also formalize. We instantiate these concepts for a family of techniques based on Abstract Reachability Trees and implement the approach using the software model checker CPAchecker. We evaluate our approach empirically and provide examples to illustrate the execution reports produced and the information that can be extracted

High-Level verification methodology for UVMF-based C++ reference model testbench implementation

Abstract. This thesis was completed for Nokia and in cooperation with Siemens EDA. In this thesis a UVM Predictor component, which wraps a C++ reference model, was generated with UVM Framework (UVMF) and implemented. The Predictor was generated and implemented to Universal Verification Methodology (UVM) testbench that had HLS generated Design Under Test (DUT). First, the UVMF generated Predictor was implemented for the UVM testbench with a small HLS-generated design to learn the verification flow. After the first trial run, the UVMF-generated Predictor was implemented into an existing UVM testbench with a bigger subsystem as a DUT. The subsystem contained two manually written RTLs and one HLS-generated RTL. First, this thesis presents the UVM theory and the UVM technologies that are used in the thesis work. The third chapter introduces code coverage, different coverage metrics, and the coverage metrics used in this thesis. After theory, practical work is presented. Chapter four explains the devices under test, UVM components, testbench connections with a UVM Predictor, Predictor generation, functionality testing, and simulation. Measured coverage metrics, tools, and technologies are also presented. Finally, coverage results from thesis work with testing strategies are presented. The results of coverage closure are discussed in chapter 6, and the thesis is summarized in chapter 7. Applying a UVMF-generated Predictor to the UVM testbench for verification flow showed promising results for obtaining a faster verification process as well as produced the possibility of using various versatile verification techniques with the Predictor, such as stimulus generation with constrained random (CR).Korkeatason verifiointi metodologian testipenkki-implementaatio UVM Framework pohjautuvalla C++ referenssi mallilla. Tiivistelmä. Tämä diplomityö on tehty Nokialle yhteistyössä Siemens EDA:n kanssa. Tässä diplomityössä UVM Framework työkalulla generoitiin ja toteutettiin UVM-prediktori komponentti, joka sisältää C++ referenssimallin. Generoitu prediktori integroitiin universaalin varmennusmenetelmän testipenkkiin, joka sisälsi HLS:llä luodun testattavan suunnitelman. Ensiksi UVMF:llä generoitu prediktori implementoitiin UVM-testipenkkiin pienellä HLS generoidulla alilohkolla, jotta verifiointivuo saatiin opeteltua. Ensimmäisen testivedoksen jälkeen, UVMF generoitu prediktori implementoitiin olemassa olevaan UVM-testipenkkiin, jossa varmennettavan suunnitelmana oli suurempi osajärjestelmä. Osajärjestelmä sisälsi kolme alilohkoa, joista kaksi oli manuaalisesti kirjoitettua RTL:ää ja yksi HLS generoitu RTL. Ensiksi tässä työssä käydään läpi UVM:n teoriaa, sekä käytettävät UVM teknologiat, joita sovelletaan diplomityössä. Kolmas kappale esittelee koodin kattavuutta ja erilaisia kattavuus parametreja. Teoriaosuuden jälkeen esitellään käytännön työn asiat. Kappale 4 esittelee varmennettavat suunnitelmat, UVM komponentit, testipenkkikytkennät prediktorin kanssa, sekä prediktorin generoinnin, testauksen ja simuloinnin. Myös työssä mitattavat kattavuusparametrit, sekä käytettävät työkalut ja teknologiat esitellään. Lopuksi esitellään diplomityössä saavutetut kattavuustulokset, sekä suunnitelmien varmennusstrategiat. Diplomityössä saavutetut tulokset käydään läpi seuraavassa kappaleessa, minkä jälkeen kappaleessa 7 tiivistetään koko diplomityö. UVMF generoidun prediktorin ottaminen mukaan osaksi UVM testipenkin verifiointivuota antoi lupaavia tuloksia verifiointiprosessin nopeuttamiseksi, ja mahdollisuuden käyttää erilaisia monipuolisia verifiointitekniikoita kuten testiherätteiden luontia rajoitetun satunnaisuuden menetelmällä

A TLM-RTL Systemverilog-Based Verification Franework for OCP Design

Open Core Protocol (OCP) establishes itself as the only non-proprietary, openly licensed, core-centric protocol that is used to support “plug-and-play” SoC (System-On-Chip) design practices. Designer can reuse OCP-compliance IP cores based on system integration and verification approach in multiple designs without reworking, reducing the development time and cutting down overall design costs. In this thesis, we develop a reusable verification framework of OCP. Assertion-based verification was chosen in order to enforce the flow. An OCP SystemVerilog monitor which is developed in house is used to verify the OCP SystemC TL1 (Cycle-accurate Level) design. The monitor can also be reused for OCP designs described at different abstraction level and thus dramatically reduce the time needed for OCP functional verification. To increase the functional coverage of OCP models, Cell-based Genetic Algorithm (CGA) with random number generators based on different probability distribution functions is provided on OCP TL1 models for generating and evolving the OCP transactions. Furthermore, SystemC Verification Library (SCV) is employed as pure random number generator to compare with the proposed CGA. The experiments show that some probability distributions have more effect on the coverage than others. The best population of the CGA can be reused on OCP RTL models to reduce the verification time

Design and Verification Environment for High-Performance Video-Based Embedded Systems

In this dissertation, a method and a tool to enable design and verification of computation demanding embedded vision-based systems is presented. Starting with an executable specification in OpenCV, we provide subsequent refinements and verification down to a system-on-chip prototype into an FPGA-Based smart camera. At each level of abstraction, properties of image processing applications are used along with structure composition to provide a generic architecture that can be automatically verified and mapped to the lower abstraction level. The result is a framework that encapsulates the computer vision library OpenCV at the highest level, integrates Accelera\u27s System-C/TLM with UVM and QEMU-OS for virtual prototyping and verification and mapping to a lower level, the last of which is the FPGA. This will relieve hardware designers from time-consuming and error-prone manual implementations, thus allowing them to focus on other steps of the design process. We also propose a novel streaming interface, called Component Interconnect and Data Access (CIDA), for embedded video designs, along with a formal model and a component composition mechanism to cluster components in logical and operational groups that reduce resource usage and power consumption

Modélisation à haut niveau d'abstraction pour les systèmes embarqués

Modern embedded systems have reached a level of complexity such that it is no longer possible to wait for the first physical prototypes to validate choices on the integration of hardware and software components. It is necessary to use models, early in the design flow. The work presented in this document contribute to the state of the art in several domains. First, we present some verification techniques based on abstract interpretation and SMT-solving for programs written in general-purpose languages like C, C++ or Java. Then, we use verification tools on models written in SystemC at the transaction level (TLM). Several approaches are presented, most of them using compilation techniques specific to SystemC to turn the models into a format usable by existing tools. The second part of the document deal with non-functional properties of models: timing performances, power consumption and temperature. In the context of TLM, we show how functional models can be enriched with non-functional information. Finally, we present contributions to the modular performance analysis (MPA) with real-time calculus (RTC) framework. We describe several ways to connect RTC to more expressive formalisms like timed automata and the synchronous language Lustre. These connections raise the problem of causality, which is defined formally and solved with the new causality closure algorithm.Les systèmes embarqués modernes ont atteint un niveau de complexité qui fait qu'il n'est plus possible d'attendre les premiers prototypes physiques pour valider les décisions sur l'intégration des composants matériels et logiciels. Il est donc nécessaire d'utiliser des modèles, tôt dans le flot de conception. Les travaux présentés dans ce document contribuent à l'état de l'art dans plusieurs domaines. Nous présentons dans un premier temps de nouvelles techniques de vérification de programmes écrits dans des langages généralistes comme C, C++ ou Java. Dans un second temps, nous utilisons des outils de vérification formelle sur des modèles écrits en SystemC au niveau transaction (TLM). Plusieurs approches sont présentées, la plupart d'entre elles utilisent des techniques de compilations spécifiques à SystemC pour transformer le programme SystemC en un format utilisable par les outils. La seconde partie du document s'intéresse aux propriétés non-fonctionnelles des modèles~: performances temporelles, consommation électrique et température. Dans le contexte de la modélisation TLM, nous proposons plusieurs techniques pour enrichir des modèles fonctionnels avec des informations non-fonctionnelles. Enfin, nous présentons les contributions faites à l'analyse de performance modulaire (MPA) avec le calcul temps-réel (RTC). Nous proposons plusieurs connections entre ces modèles analytiques et des formalismes plus expressifs comme les automates temporisés et le langage de programmation Lustre. Ces connexion posent le problème théorique de la causalité, qui est formellement défini et résolu avec un algorithme nouveau dit de " fermeture causale "

SystemC Through the Looking Glass : Non-Intrusive Analysis of Electronic System Level Designs in SystemC

Due to the ever increasing complexity of hardware and hardware/software co-designs, developers strive for higher levels of abstractions in the early stages of the design flow. To address these demands, design at the Electronic System Level (ESL) has been introduced. SystemC currently is the de-facto standard for ESL design. The extraction of data from system designs written in SystemC is thereby crucial e.g. for the proper understanding of a given system. However, no satisfactory support of reflection/introspection of SystemC has been provided yet. Previously proposed methods for this purpose %introduced to achieve the goal nonetheless either focus on static aspects only, restrict the language means of SystemC, or rely on modifications of the compiler and/or parser. In this thesis, approaches that overcome these limitations are introduced, allowing the extraction of information from a given SystemC design without changing the SystemC library or the compiler. The proposed approaches retrieve both, static and dynamic (i.e. run-time) information

Virtual Cycle-accurate Hardware and Software Co-simulation Platform for Cellular IoT

Modern embedded development flows often depend on FPGA board usage for pre-ASIC system verification. The purpose of this project is to instead explore the usage of Electronic System Level (ESL) hardware-software co-simulation through the usage of ARM SoC Designer tool to create a virtual prototype of a cellular IoT modem and thereafter compare the benefits of including such a methodology into the early development cycle. The virtual system is completely developed and executed on a host computer, without the requirement of additional hardware. The virtual prototype hardware is based on C++ ARM verified cycle-accurate models generated from RTL hardware descriptions, High-level synthesis (HLS) pre-synthesis SystemC HW accelerator models and behavioural models which implement the ARM Cycle-accurate Simulation Interface (CASI). The micro-controller of the virtual system which is based on an ARM Cortex-M processor, is capable of executing instructions from a memory module. This report documents the virtual prototype implementation and compares both the software performance and cycle-accuracy of various virtual micro-controller configurations to a commercial reference development board. By altering factors such as memory latencies and bus interconnect subsystem arbitration in co-simulations, the software cycle-count performance of the development board was shown possible to reproduce within a 5% error margin, at the cost of approximately 266 times slower execution speed. Furthermore, the validity of two HLS pre-synthesis hardware models is investigated and proven to be functionally accurate within three clock cycles of individual block latency compared to post-synthesis FPGA synthesized implementations. The final virtual prototype system consisted of the micro-controller and two cellular IoT hardware accelerators. The system runs a FreeRTOS 9.0.0 port, executing a multi-threaded program at an average clock cycle simulation frequency of 10.6 kHz.-Designing and simulating embedded computer systems virtually. Cellular internet of things (IoT) is a new technology that will enable the interconnection of everything: from street lights and parking meters to your gas or water meter at home, wireless cellular networks will allow information to be shared between devices. However, in order for these systems to provide any useful data, they need to include a computer chip with a system to manage the communication itself, enabling the connection to a cellular network and the actual transmission and reception of data. Such a chip is called an embedded chip or system. Traditionally, the design and verification of digital embedded systems, that is to say a system which has both hardware and software components, had to be done in two steps. The first step consists of designing all the hardware, testing it, integrating it and producing it physically on silicon in order to verify the intended functionality of all the components. The second step thus consists of taking the hardware that has been developed and designing the software: a program which will have to execute in complete compliance to the hardware that has been previously developed. This poses two main issues: the software engineers cannot begin their work properly until the hardware is finished, which makes the process very long, and the fact that the hardware has been printed on silicon greatly restricts the possibility of doing changes to accommodate late system requirement alterations; which is quite likely for a tailor-made application specific system such as a cellular IoT chip. A currently widespread technology used to mitigate the previously mentioned negative aspects of embedded design, is the employment of field-programmable gate array (FPGA) development boards which often contain a micro-controller (with a processor and some memories), and a gate array connected to it. The FPGA part consists of a lattice of digital logic gates which can be programmed to interconnect and represent the functionality of the hardware being designed. The processor can thus execute software instructions placed on the memories and the hardware being developed can be programmed into the gate array in order to integrate and verify a full hardware and software system. Nevertheless, this boards are expensive and limit the design to the hardware components available commercially in the different off-the-shelf models, e.g. a specific processor which might not be the desired one. Now imagine there is a way to design hardware components such as processors in the traditional way, however once the hardware has been implemented it can be integrated together with software without the need of printing a physical silicon chip specifically for this purpose. That would be extremely convenient and would save lots of time, would it not? Fortunately, this is already possible due to Electronic System Level (ESL) design, which is compilation of techniques that allow to design, simulate and partially verify a digital chip, all within any normal laptop or desktop computer. Moreover, some ESL tools such as the one investigated in this project, allow you to even simulate a program code written specifically for this hardware; this is known as virtual hardware software co-simulation. The reliability of simulation must however be considered when compared to a traditional two-step methodology or FPGA board usage to verify a full system. This is because a virtual hardware simulation can have several degrees of accuracy, depending on the specificity of component models that make up the virtual prototype of the digital system. Therefore, in order to use co-simulation techniques with a high degree of confidence for verification, the highest accuracy degree should be employed if possible to guarantee that what is being simulated will match the reality of a silicon implementation. The clock cycle-accurate level is one of the highest accuracy system simulation methods available, and it consists of representing the digital states of all hardware components such as signals and registers, in a cycle-by-cycle manner. By using the ARM SoC Designer ESL tool, we have co-designed and co-simulated several microcontrollers on a detailed, cycle-accurate level and confirmed its behaviour by comparing it to a physical reference target development board. Finally, a more complex virtual prototype of a cellular IoT system was also simulated, including a micro-controller running a a real-time operating system (RTOS), hardware accelerators and serial data interfacing. Parts of this virtual prototype where compared to an FPGA board to evaluate the pros and cons of incorporating virtual system simulation into the development cycle and to what extent can ESL methods substitute traditional verification techniques. The ease of interchanging hardware, simplicity of development, simulation speed and the level of debug capabilities available when developing in a virtual environment are some of the aspects of ARM SoC Designer discussed in this thesis. A more in depth description of the methodology and results can be found in the report titled "Virtual Cycle-accurate Hardware and Software Co-simulation Platform for Cellular IoT"