UVM testbench in Python:feature and performance comparison with SystemVerilog implementation

Abstract. Python is emerging as a new language for functional verification of digital integrated circuits (ICs). With the Python verification framework cocotb enabling to write testbenches in Python, new libraries are being developed for various verification techniques and methodologies, such as functional coverage, constrained random verification and Universal Verification Methodology (UVM). Python testbenches have been used in some research and product development, but there is little information available on their performance, and no studies about applying UVM in Python have been published. In this thesis, a Python UVM testbench was developed using pyuvm and other Python verification libraries for an AHB-Lite slave IP, and a matching testbench in SystemVerilog was also built to examine the differences in their implementations. Testbench codebase sizes, simulation execution times, memory use and coverage accumulation were compared. The Python testbench had 30% less lines of code, suggesting that testbench development may be faster in Python than SystemVerilog. The execution times of the Python testbench on commercial simulators were 8 to 21 times longer than those of the SystemVerilog testbench in tests with AHB-Lite write operations and random stimulus. In conclusion, given the performance gap and the UVM Register Abstraction Layer (RAL) being at an early stage of development in pyuvm, the studied Python libraries are not competitive with SystemVerilog and its UVM implementation for verifying complex designs like systems-on-chip (SoCs) at this stage. Nevertheless, pyuvm enables Python programmers and users of open-source simulators without support for SystemVerilog UVM to start using the methodology. A Python UVM testbench based on pyuvm is currently viable for verifying simple designs, and it opens new avenues of research in digital IC verification.Tiivistelmä. Python on nousemassa uudeksi kieleksi digitaalisten integroitujen piirien varmennukseen. Cocotb-viitekehys mahdollistaa testipenkkien kirjoittamisen Pythonilla, ja uusia Python-kirjastoja kehitetään eri varmennusmenetelmille, kuten funktionaaliselle kattavuudelle, rajoitetulla satunnaisherätteellä verifioinnille ja universaalille varmennusmenetelmälle (engl. Universal Verification Methodology, UVM). Python-testipenkkejä on pienissä määrin käytetty tutkimuksissa ja tuotekehityksessä, mutta niiden suorituskyvystä on hyvin vähän tietoa, ja UVM:n käytöstä Pythonilla ei ole julkaistu tutkimuksia. Tässä työssä kehitettiin UVM-testipenkki Pythonilla AHB-Lite-orjana toimivalle IP-lohkolle käyttäen pyuvm:ää ja muita Python-verifiointikirjastoja, ja vastaava testipenkki luotiin myös SystemVerilogilla toteutusten vertailua varten. Testipenkeistä verrattiin koodikannan kokoa, suoritusaikaa, muistin käyttöä ja kattavuuden kertymistä. Python-testipenkissä oli 30 % vähemmän koodirivejä, mikä voi merkitä, että testipenkkien kehittäminen Pythonilla on nopeampaa kuin SystemVerilogilla. Suoritusajat kaupallisilla simulaattoreilla oli Python-testipenkillä 8–21 kertaa pidempiä kuin SystemVerilog-testipenkillä testeissä, joissa ajettiin AHB-Lite -kirjoitusoperaatioita ja satunnaisherätettä. Koska suorituskykyero oli näin merkittävä, ja koska UVM:n rekisteriabstraktiotaso (engl. Register Abstraction Layer, RAL) on vasta alkutekijöissään pyuvm:ssä, voidaan todeta, että tutkitut Python-kirjastot eivät ole vielä nykyisellä tasollaan kilpailukykyisiä SystemVerilogin ja sen UVM-implementaation kanssa monimutkaisten piirien kuten järjestelmäpiirien varmennukseen. Siitä huolimatta pyuvm mahdollistaa UVM:n käytön Python-ohjelmoijille ja avoimen lähdekoodin simulaattoreissa, joissa ei ole vielä SystemVerilog UVM:lle tukea. Pyuvm-pohjainen Python UVM-testipenkki soveltuu tällä hetkellä yksinkertaisten mallien varmennukseen ja avaa uusia tutkimussuuntia digitaalisten integroitujen piirien varmennukseen

Analyzing UVM reuse

Abstract. This thesis investigates Universal Verification Methodology’s (UVM) reuse possibilities. Initally, the object-oriented features of the UVM’s programming language SystemVerilog (SV), are introduced. Those features are one enabling factor in UVM reuse. The work also provides a brief overview to the development history of UVM and presents its properties. The structure of a conventional UVM testbench is also demonstrated. Finally, the features that make the UVM testbench more reusable are briefly introduced. In the practical part of the study, a UVM testbench is made for Nordic Semiconductor’s Introproject. The testbench was created with extensive comments so that beginners would get the most out of it. The methods that make the testbench reusable are also applied to the testbench. At the end of the practical part, the reuse possibilities of the testbench were tested by changing the Design Under Test (DUT). Modifications were made to the testbench in order to match the new features of the DUT.UVM uudelleenkäytön analysointi. Tiivistelmä. Tämä diplomityö tutkii Universaalin varmennusmenetelmän (UVM) uudelleenkäyttömahdollisuuksia. Aluksi UVM:n ohjelmointikielen, SystemVerilogin olio-ohjelmointipohjaisia ominaisuuksia käydään läpi. Nämä ominaisuudet ovat yksi mahdollistava tekijä UVM uudelleenkäytössä. Työssä tehdään lisäksi lyhyt katsaus UVM:n kehityshistoriaan ja esitellään myös sen ominaisuudet sekä tavanomaisen UVM-testipenkin rakenne. Lopuksi esitellään lyhyesti ominaisuuksia, jolla saa tehtyä UVM testipenkistä paremmin uudelleenkäytettävän. Työn käytännön osuudessa tehdään UVM-testipenkki Nordic Semiconductorin Introprojektiin. Testipenkki tehtiin laajasti kommentoimalla, jotta aloitteleva testipenkin tekijä saa siitä mahdollisimman paljon irti. Testipenkin tekemisessä käytettiin myös menetelmiä, joita esiteltiin aiemmassa teoriakappaleessa. Käytännön osuuden lopuksi testattiin testipenkin uudelleenkäyttöä muuttamalla testissä olevaa komponenttia. Testipenkkiin tehtiin muutokset, jolla se saatiin taas vastaamaan komponentin tarpeita

Development of open verification ip for I2C controller

Before any IC is fabricated it is desired to check whether the required functionalities are preserved or not. Otherwise this may lead to a huge loss to the company in case of any failure in during the design/coding stage. Verification engineers have to make sure that before fabrication all the properties of the IC can be successfully implicated. So functional verification provides a lot of benefits to the IC designers. Today, testing and verification are alternatively used for the same thing. Testing of a large design using FPGA consumes longer compilation time in case of debugging and committing small mistakes. Simulation based testing is faster and also provides capability to check all the signals buried under the design. But due to the increasing complexity in design and the concurrency behavior of the design it has become very difficult to verify the functionality using traditional testbenches. So new languages called Hardware Verification Languages (HVL) are introduced. System Verilog is an IEEE standard Verification language. The library and package oriented feature provide an efficient way of writing testbenches. The Open Verification Methodology (OVM) Class Library provides the building blocks needed to quickly develop reusable and well-constructed verification components and test environments using SystemVerilog. In this paper we have developed testing environment using system Verilog implementation of OVM for I2C controller core. Our work introduces an automated stimulus generating testing environment for the design and checks the functionality of the I2C bus controller

Graphical framework for automatic generation of custom UVM testbenches in SystemVerilog applied for the validation of a SerDes DUT

A novel graphical tool designed to assist Pre-Silicon validators in the creation of complete, functional, and compile-clean UVM testbenches is presented in this case study. A detailed description of the user-friendly interface is documented and demonstrated to auto-generate a validation environment template for the verification of an ALU and SerDes chip. The output obtained from the tool is later customized and optional sections are filled up to perform the full validation of the circuit. For the SerDes DUT, this case study takes over from the work of the latest 2017 ITESO SerDes circuit design. Both authors of this document worked on the 2016 iteration and are very familiar with the design, but this time instead of the actual design of the chip, the primary focus is how this new validation tool can be an essential asset to ensure the quality of the chip and to improve the efficiency of the verification process

A Unique Test Bench for Various System-on-a-Chip

This paper discusses a standard flow on how an automated test bench environment which is randomized with constraints can verify a SOC efficiently for its functionality and coverage. Today, in the time of multimillion gate ASICs, reusable intellectual property (IP), and system-on-a-chip (SoC) designs, verification consumes about 70 % of the design effort. Automation means a machine completes a task autonomously, quicker and with predictable results. Automation requires standard processes with well-defined inputs and outputs. By using this efficient methodology it is possible to provide a general purpose automation solution for verification, given today’s technology. Tools automating various portions of the verification process are being introduced. Here, we have Communication based SOC The content of the paper discusses about the methodology used to verify such a SOC-based environment. Cadence Efficient Verification Methodology libraries are explored for the solution of this problem. We can take this as a state of art approach in verifying SOC environments. The goal of this paper is to emphasize the unique testbench for different SOC using Efficient Verification Constructs implemented in system verilog for SOC verification

A Cross-level Verification Methodology for Digital IPs Augmented with Embedded Timing Monitors

Smart systems are characterized by the integration in a single device of multi-domain subsystems of different technological domains, namely, analog, digital, discrete and power devices, MEMS, and power sources. Such challenges, emerging from the heterogeneous nature of the whole system, combined with the traditional challenges of digital design, directly impact on performance and on propagation delay of digital components. This article proposes a design approach to enhance the RTL model of a given digital component for the integration in smart systems with the automatic insertion of delay sensors, which can detect and correct timing failures. The article then proposes a methodology to verify such added features at system level. The augmented model is abstracted to SystemC TLM, which is automatically injected with mutants (i.e., code mutations) to emulate delays and timing failures. The resulting TLM model is finally simulated to identify timing failures and to verify the correctness of the inserted delay monitors. Experimental results demonstrate the applicability of the proposed design and verification methodology, thanks to an efficient sensor-aware abstraction methodology, by applying the flow to three complex case studies

Использование языка ПРАЛУ для верификации цифровых устройств

The task of creating a testbench for functional verification is considered. This verification process establishes the reconvergence (equivalence) of the device specification and the register-transfer level (RTL) model - a logical network which was built in the synthesis process. In the UVM methodology, usually used in the modern design of digital devices for functional verification, a testing strategy, that determines the way in which a test case is constructed, is the random selection of space-driven constrained-random transaction-level self-checking testbenches. The rules and recommendations of UVM contain a standardized structure of the test bench, which is oriented towards the development of transformational devices. For the case where the model of the design is a behavior algorithm, it is proposed to build a testbench as a model of the environment of the design presented in the language of PRALU. The environment model of the developed device allows to avoid situations when the device under test is verified with sufficient coverage, but in an incomplete environment. The environment model on PRALU can be automatically converted into a transaction level model to develop a testbench in the simulator environment of the hardware description language.Рассматривается задача создания испытательного стенда для функциональной верификации. В процессе верификации устанавливается сводимость (эквивалентность) спецификации устройства и модели уровня регистровых передач (register-transfer level, RTL) - логической сети, построенной в процессе синтеза. В универсальной методологии верификации (universal verification methodology, UVM), наиболее часто используемой в современном проектировании цифровых устройств для функциональной верификации, стратегией тестирования, определяющей способ построения тестового примера, является случайный выбор в пространстве входных воздействий (coverage-driven constrained-random transaction-level self-checking testbenches). Правила и рекомендации UVM содержат стандартизованную структуру испытательного стенда, которая ориентирована на разработку трансформационных устройств. В случае если моделью разрабатываемого устройства является алгоритм поведения, предлагается строить испытательный стенд как модель окружающей среды проектируемого устройства, представленную на языке ПРАЛУ. Модель среды разрабатываемого устройства позволяет избегать ситуаций, когда испытуемое устройство верифицируется с достаточным покрытием схемы тестами, но в неполном окружении. Для разработки испытательного стенда в среде симулятора языка описания аппаратуры модель окружающей среды на ПРАЛУ может быть автоматически преобразована в модель уровня транзакций

Automated UVM Testbench Generation Using EMF

La verifica di dispositivi digitali complessi richiede lo sviluppo di testbench che diventano sempre più complessi con un aumento continuo dei tempi di realizzazione e di manutenzione. La metodologia UVM (Universal Verification Methodology) è stata introdotta dall'industria per permettere un'astrazione dell'ambiente di verifica ed allo stesso tempo aumentare la capacità di riutilizzo dei componenti. Rimane però complicata la creazione. Questo elaborata esplora una possibile strategia, basata su EMF (Eclipse Modeling Framework), Sirius ed Acceleo, per automatizzare la stesura dei testbench. Si comincia con una presentazione di alcuni strumenti utilizzati nella verifica, quali Verilog, SystemVerilog ed UVM, seguita da una presentazione dell'insieme di strumenti che si possono utilizzare per la generazione automatica di codice. In particolare, EMF (Eclipse Modeling Framework), Sirius ed Acceleo. L'elaborato si conclude con una discussione sull'utilizzo degli strumenti nel progetto sviluppato durante il tirocinio in azienda.Verifying complex digital devices requires developing testbenches of ever growing complexity, whose creation and maintenance times keep increasing. UVM (Universal Verification Methodology) was introduced by the industry to allow the abstraction of the verification environment and, at the same time, increase reusability. Testbench creation remains complex and time consuming. This dissertation explores a possible strategy, based on EMF (Eclipse Modeling Framework), Sirius and Acceleo, for automating testbench generation. The work begins with an introduction of some of the state-of-the-art tools used in verification, i.e. Verilog, SystemVerilog and UVM, followed by an introduction to a set of tools that can be used for automatic code generation. In particular, EMF (Eclipse Modeling Framework), Sirius ed Acceleo. The dissertation concludes with a discussion on the use of the tools for a project developed during the internship