Enhancement of fault injection techniques based on the modification of VHDL code

Deep submicrometer devices are expected to be increasingly sensitive to physical faults. For this reason, fault-tolerance mechanisms are more and more required in VLSI circuits. So, validating their dependability is a prior concern in the design process. Fault injection techniques based on the use of hardware description languages offer important advantages with regard to other techniques. First, as this type of techniques can be applied during the design phase of the system, they permit reducing the time-to-market. Second, they present high controllability and reachability. Among the different techniques, those based on the use of saboteurs and mutants are especially attractive due to their high fault modeling capability. However, implementing automatically these techniques in a fault injection tool is difficult. Especially complex are the insertion of saboteurs and the generation of mutants. In this paper, we present new proposals to implement saboteurs and mutants for models in VHDL which are easy-to-automate, and whose philosophy can be generalized to other hardware description languages.Baraza Calvo, JC.; Gracia-Morán, J.; Blanc Clavero, S.; Gil Tomás, DA.; Gil Vicente, PJ. (2008). Enhancement of fault injection techniques based on the modification of VHDL code. IEEE Transactions on Very Large Scale Integration (VLSI) Systems. 16(6):693-706. doi:10.1109/TVLSI.2008.2000254S69370616

Simulating the effects of logic faults in implementation-level VITAL-compliant models

[EN] Simulation-based fault injection is a well-known technique to assess the dependability of hardware designs specified using hardware description languages (HDL). Although logic faults are usually introduced in models defined at the register transfer level (RTL), most accurate results can be obtained by considering implementation-level ones, which reflect the actual structure and timing of the circuit. These models consist of a list of interconnected technology-specific components (macrocells), provided by vendors and annotated with post-place-and-route delays. Macrocells described in the very high speed integrated circuit HDL (VHDL) should also comply with the VHDL initiative towards application specific integrated circuit libraries (VITAL) standard to be interoperable across standard simulators. However, the rigid architecture imposed by VITAL makes that fault injection procedures applied at RTL cannot be used straightforwardly. This work identifies a set of generic operations on VITAL-compliant macrocells that are later used to define how to accurately simulate the effects of common logic fault models. The generality of this proposal is supported by the definition of a platform-specific fault procedure based on these operations. Three embedded processors, implemented using the Xilinx¿s toolchain and SIMPRIM library of macrocells, are considered as a case study, which exposes the gap existing between the robustness assessment at both RTL and implementation-level.This work has been partially funded by the Ministerio de Economia, Industria y Competitividad of Spain under grant agreement no TIN2016-81075-R, and the "Programa de Ayudas de Investigacion y Desarrollo" (PAID) of Universitat Politecnica de Valencia.Tuzov, I.; De-Andrés-Martínez, D.; Ruiz, JC. (2019). Simulating the effects of logic faults in implementation-level VITAL-compliant models. Computing. 101(2):77-96. https://doi.org/10.1007/s00607-018-0651-4S77961012Baraza JC, Gracia J, Blanc S, Gil D, Gil P (2008) Enhancement of fault injection techniques based on the modification of vhdl code. IEEE Tran Very Large Scale Integr Syst 16:693–706Baraza JC, Gracia J, Gil D, Gil P (2002) A prototype of a vhdl-based fault injection tool: description and application. Journal of Systems Architecture 47(10):847–867Benites LAC, Kastensmidt FL (2017) Fault injection methodology for single event effects on clock-gated asics. In: IEEE Latin American test symposium. IEEE, pp 1–4Benso A, Prinetto P (2003) Fault injection techniques and tools for VLSI reliability evaluation. Frontiers in electronic testing. Kluwer Academic Publishers, BerlinCobham Gaisler AB: LEON3 processor product sheet (2016). https://www.gaisler.com/doc/leon3_product_sheet.pdfCohen B (2012) VHDL coding styles and methodologies. Springer, New YorkDas SR, Mukherjee S, Petriu EM, Assaf MH, Sahinoglu M, Jone WB (2006) An improved fault simulation approach based on verilog with application to ISCAS benchmark circuits. In: IEEE instrumentation and measurement technology conference, pp 1902–1907Fernandez V, Sanchez P, Garcia M, Villar E (1994) Fault modeling and injection in VITAL descriptions. In: Third annual Atlantic test workshop, pp o1–o4Gil D, Gracia J, Baraza JC, Gil P (2003) Study, comparison and application of different vhdl-based fault injection techniques for the experimental validation of a fault-tolerant system. J Syst Archit 34(1):41–51Gil P, Arlat J, Madeira H, Crouzet Y, Jarboui T, Kanoun K, Marteau T, Duraes J, Vieira M, Gil D, Baraza JC, Gracia J (2002) Fault representativeness. Technical report, dependability benchmarking projectGuthaus MR, Ringenberg JS, Ernst D, Austin TM, Mudge T, Brown RB (2001) MiBench: a free, commercially representative embedded benchmark suite. In: IEEE 4th annual workshop on workload characterization, pp 3–14IEEE Standard for VITAL ASIC (Application Specific Integrated Circuit) (2000) Modeling specification. Institute of Electrical and Electronic Engineers, StandardIEEE Standard VHDL Language Reference Manual (2008) Institute of Electrical and Electronic Engineers, StandardIEEE Standard for Standard Delay Format (SDF) for the Electronic Design Process. Institute of Electrical and Electronic Engineers, Standard (2001)Jenn E, Arlat J, Rimen M, Ohlsson J, Karlsson J (1994) Fault injection into VHDL models: the MEFISTO tool. In: International symposium on fault-tolerant computing, pp 66–75Kochte MA, Schaal M, Wunderlich HJ, Zoellin CG (2010) Efficient fault simulation on many-core processors. In: Design automation conference, pp 380–385Mansour W, Velazco R (2013) An automated seu fault-injection method and tool for HDL-based designs. IEEE Trans Nucl Sci 60(4):2728–2733Mentor Graphics (2016) Questa SIM command reference manual 10.7b, Document Revision 3.5. https://www.mentor.com/products/fv/modelsim/Munden R (2000) Inverter, STDN library. Free model foundry VHDL model list. https://freemodelfoundry.com/fmf_models/stnd/std04.vhdMunden R (2004) ASIC and FPGA verification: a guide to component modeling. Systems on silicon. Elsevier, AmsterdamNa J, Lee D (2011) Simulated fault injection using simulator modification technique. ETRI J 33(1):50–59Nimara S, Amaricai A, Popa M (2015) Sub-threshold cmos circuits reliability assessment using simulated fault injection based on simulator commands. In: IEEE International Symposium on Applied Computational Intelligence and Informatics, pp 101–104Oregano Systems GmbH (2013) MC8051 IP Core, user guide (V 1.2) 2013. http://www.oreganosystems.at/download/mc8051_ug.pdfRomani E (1998) Structural PIC165X microcontroller. Hamburg VHDL archive. https://tams-www.informatik.uni-hamburg.de/vhdlShaw D, Al-Khalili D, Rozon C (2006) Automatic generation of defect injectable VHDL fault models for ASIC standard cell libraries. Integr VLSI J 39(4):382–406Shaw DB, Al-Khalili D (2003) IC bridge fault modeling for IP blocks using neural network-based VHDL saboteurs. IEEE Trans Comput 10:1285–1297Short KL (2008) VHDL for engineers, 1st edn. Pearson, LondonSieh V, Tschache O, Balbach F (1997) Verify: evaluation of reliability using VHDL-models with embedded fault descriptions. In: International symposium on fault-tolerant computing, pp 32–36Singh L, Drucker L (2004) Advanced verification techniques. Frontiers in electronic testing. Springer, New YorkTuzov I, de Andrés D, Ruiz JC (2017) Dependability-aware design space exploration for optimal synthesis parameters tuning. In: IEEE/IFIP international conference on dependable systems and networks, pp 1–12Tuzov I, de Andrés D, Ruiz JC (2017) Robustness assessment via simulation-based fault injection of the implementation level models of the LEON3, MC8051, and PIC microcontrollers in presence of stuck-at, bit-flip, pulse, and delay fault models [Data set], Zenodo. https://doi.org/10.5281/zenodo.891316Tuzov I, de Andrés D, Ruiz JC (2018) DAVOS: EDA toolkit for dependability assessment, verification, optimization and selection of hardware models. In: IEEE/IFIP international conference on dependable systems and networks, pp 322–329Tuzov I, Ruiz JC, de Andrés D (2017) Accurately simulating the effects of faults in VHDL models described at the implementation-level. In: European dependable computing conference, pp 10–17Wang LT, Chang YW, Cheng KT (2009) Electronic design automation: synthesis, verification, and test. Morgan Kaufmann, BurlingtonXilinx: Synthesis and simulation design guide, UG626 (v14.4) (2012). https://www.xilinx.com/support/documentation/sw_manuals/xilinx14_7/sim.pd

Studying the effects of intermittent faults on a microcontroller

As CMOS technology scales to the nanometer range, designers have to deal with a growing number and variety of fault types. Particularly, intermittent faults are expected to be an important issue in modern VLSI circuits. The complexity of manufacturing processes, producing residues and parameter variations, together with special aging mechanisms, may increase the presence of such faults. This work presents a case study of the impact of intermittent faults on the behavior of a commercial microcontroller. In order to carry out an exhaustive reliability assessment, the methodology used lies in VHDL-based fault injection technique. In this way, a set of intermittent fault models at logic and register transfer abstraction levels have been generated and injected in the VHDL model of the system. From the simulation traces, the occurrences of failures and latent errors have been logged. The impact of intermittent faults has been also compared to that got when injecting transient and permanent faults. Finally, some injection experiments have been reproduced in a RISC microprocessor and compared with those of the microcontroller. © 2012 Elsevier Ltd. All rights reserved.This work has been funded by the Spanish Government under the Research Project TIN2009-13825.Gil Tomás, DA.; Gracia-Morán, J.; Baraza Calvo, JC.; Saiz-Adalid, L.; Gil Vicente, PJ. (2012). Studying the effects of intermittent faults on a microcontroller. Microelectronics Reliability. 52(11):2837-2846. https://doi.org/10.1016/j.microrel.2012.06.004S28372846521

Large-Scale Application of Fault Injection into PyTorch Models -- an Extension to PyTorchFI for Validation Efficiency

Transient or permanent faults in hardware can render the output of Neural Networks (NN) incorrect without user-specific traces of the error, i.e. silent data errors (SDE). On the other hand, modern NNs also possess an inherent redundancy that can tolerate specific faults. To establish a safety case, it is necessary to distinguish and quantify both types of corruptions. To study the effects of hardware (HW) faults on software (SW) in general and NN models in particular, several fault injection (FI) methods have been established in recent years. Current FI methods focus on the methodology of injecting faults but often fall short of accounting for large-scale FI tests, where many fault locations based on a particular fault model need to be analyzed in a short time. Results need to be concise, repeatable, and comparable. To address these requirements and enable fault injection as the default component in a machine learning development cycle, we introduce a novel fault injection framework called PyTorchALFI (Application Level Fault Injection for PyTorch) based on PyTorchFI. PyTorchALFI provides an efficient way to define randomly generated and reusable sets of faults to inject into PyTorch models, defines complex test scenarios, enhances data sets, and generates test KPIs while tightly coupling fault-free, faulty, and modified NN. In this paper, we provide details about the definition of test scenarios, software architecture, and several examples of how to use the new framework to apply iterative changes in fault location and number, compare different model modifications, and analyze test results.Comment: accepted in DSN202

A Cross-level Verification Methodology for Digital IPs Augmented with Embedded Timing Monitors

Smart systems implement the leading technology advances in the context of embedded devices. Current design methodologies are not suitable to deal with tightly interacting subsystems of different technological domains, namely analog, digital, discrete and power devices, MEMS and power sources. The interaction effects between the components and between the environment and the system must be modeled and simulated at system level to achieve high performance. Focusing on digital subsystem, additional design constraints have to be considered as a result of the integration of multi-domain subsystems in a single device. The main digital design challenges combined with those emerging from the heterogeneous nature of the whole system directly impact on performance, hence propagation delay, of the digital component. In this paper we propose a design approach to enhance the RTL model of a given digital component for the integration in smart systems, and a methodology to verify the added features at system-level. The design approach consists of ``augmenting'' the RTL model through the automatic insertion of delay sensors, which are capable of detecting and correcting timing failures. The verification methodology consists of an automatic flow of two steps. Firstly the augmented model is abstracted to system-level (i.e., SystemC TLM); secondly mutants, which are code mutations to emulate timing failures, are automatically injected into the abstracted model. Experimental results demonstrate the applicability of the proposed design and verification methodology and the effectiveness of the simulation performance

Analysis and RTL Correlation of Instruction Set Simulators for Automotive Microcontroller Robustness Verification

© ACM 2015 This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM, In Proceedings of the 52nd Annual Design Automation Conference (p. 40). http://dx.doi.org/10.1145/2744769.2744798.Increasingly complex microcontroller designs for safety-relevant automotive systems require the adoption of new methods and tools to enable a cost-effective verification of their robustness. In particular, costs associated to the certification against the IS026262 safety standard must be kept low for economical reasons. In this context, simulation-based verification using instruction set simulators (ISS) arises as a promising approach to partially cope with the increasing cost of the verification process as it allows taking design decisions in early design stages when modifications can be performed quickly and with low cost. However, it remains to be proven that verification in those stages provides accurate enough information to be used in the context of automotive microcontrollers. In this paper we analyze the existing correlation between fault injection experiments in an RTL microcontroller description and the information available at the ISS to enable accurate ISS-based fault injection.The research leading to these results has received funding from the ARTEMIS Joint Undertaking VeTeSS project under grant agreement number 295311. This work has also been funded by the Ministry of Science and Technology of Spain under contract TIN2012-34557 and HiPEAC. Jaume Abella is partially supported by the Ministry of Economy and Competitiveness under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717.Espinosa García, J.; Hernández Luz, C.; Abella, J.; Andrés Martínez, DD.; Ruiz García, JC. (2015). Analysis and RTL Correlation of Instruction Set Simulators for Automotive Microcontroller Robustness Verification. ACM. https://doi.org/10.1145/2744769.2744798SARTEMIS Joint Undertaking.VeTeSS project:www.vetess.eu.J.-C. Baraza, et al. Enhancement of fault injection techniques based on the modification of vhdl code.IEEE Transactions on VLSI, 16(6):693--706, June 2008.Alfredo Benso et al.Fault Injection Techniques and Tools for Embedded Systems Reliability Evaluation.Kluwer Academic Publishers, 2003.D. Borodin et al. Protective redundancy overhead reduction using instruction vulnerability factor. InCF, 2010.R. N. Charette. This car runs on code. InIEEE Spectrum online, 2009.Pedro Gil, et al. Fault representativeness. Technical report, DBench project, IST 2000-25425 [Online]. Available: http://www.laas.fr/DBench, 2002.C. Hernandez et al. Live: Timely error detection in light-lockstep safety critical systems. InDAC, 2014.Infineon. AURIX - TriCore datasheet. highly integrated and performance optimized 32-bit microcontrollers for automotive and industrial applications, 2012. http://www.infineon.com/.International Organization for Standardization.ISO/DIS 26262. Road Vehicles--Functional Safety, 2009.E. Jenn, et al. Fault injection into VHDL models: the mefisto tool. InFTCS, 1994.G. Leen et al. Expanding automotive electronic systems.IEEE Computer, 35(1), 2002.Man-Lap Li, et al. Accurate microarchitecture-level fault modeling for studying hardware faults. InHPCA, 2009.Michail Maniatakos, et al. Instruction-level impact analysis of low-level faults in a modern microprocessor controller.IEEE Transactions on Computers, 60(9):1260--1273, 2011.S. S. Mukherjee, et al. A systematic methodology to compute the architectural vulnerability factors for a high-performance microprocessor. InMICRO, 2003.J.-H. Oetjens, et al. Safety evaluation of automotive electronics using virtual prototypes: State of the art and research challenges. InDAC, 2014.J. Poovey.Characterization of the EEMBC Benchmark Suite.North Carolina State University, 2007.M. Psarakis, et al. Microprocessor software-based self-testing.Design Test of Computers, IEEE, 27(3):4--19, May 2010.S. Rehman, et al. Reliable software for unreliable hardware: Embedded code generation aiming at reliability. InCODES+ISSS, 2011.S. Rohr, et al. An integrated approach to automotive safety systems.SAE Automotive Engineering International magazine, September 2000.B. Sangchoolie, et al. A study of the impact of bit-flip errors on programs compiled with different optimization levels. InEDCC, 2014.STMicroelectronics.32-bit Power Architecture microcontroller for automotive SIL3/ASILD chassis and safety applications, 2014.http://www.gaisler.com/cms/index.php?option=com_content&task=view&id=13&Itemid=53.Leon3 Processor.Areroflex Gaisler

Towards Certification-aware Fault Injection Methodologies Using Virtual Prototypes

© 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Safety-critical applications are required today to meet more and more stringent standards than ever. In the need of reducing the costs associated with the certification step, early robustness evaluation can provide valuable information, as long as it is fast and accurate enough. Microarchitectural simulators have been employed for testing reliability properties in several domains in the past, but their use in the process of robustness verification of safety critical systems has not been validated yet, as opposed to RTL or gate-level simulations. In the present work, we propose a methodology to improve the accuracy of faultinjection results when targeting robustness verification, by using microarchitectural simulators and virtual prototypes for an early estimation of deviations with respect to the certification standards.The research leading to these results has received funding from the Ministry of Science and Technology of Spain under contract TIN2012-34557 and HiPEAC. Likewise, Jaume Abella is partially supported by the Ministry of Economy and Competitiveness under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717.Espinosa García, J.; Andrés Martínez, DD.; Ruiz García, JC.; Hernández Luz, C.; Abella, J. (2015). Towards Certification-aware Fault Injection Methodologies Using Virtual Prototypes. IEEE Conference Publications. http://hdl.handle.net/10251/65831

Analysing system susceptibility to faults with simulation tools

In the paper we present original fault simulation tools developed in our Institute. These tools are targeted at system dependability evaluation. They provide mechanisms for detailed and aggregated fault effect analysis. Based on our experience with testing various software applications we outline the most important problems and discuss a sample of simulation results

A Cross-level Verification Methodology for Digital IPs Augmented with Embedded Timing Monitors

Smart systems are characterized by the integration in a single device of multi-domain subsystems of different technological domains, namely, analog, digital, discrete and power devices, MEMS, and power sources. Such challenges, emerging from the heterogeneous nature of the whole system, combined with the traditional challenges of digital design, directly impact on performance and on propagation delay of digital components. This article proposes a design approach to enhance the RTL model of a given digital component for the integration in smart systems with the automatic insertion of delay sensors, which can detect and correct timing failures. The article then proposes a methodology to verify such added features at system level. The augmented model is abstracted to SystemC TLM, which is automatically injected with mutants (i.e., code mutations) to emulate delays and timing failures. The resulting TLM model is finally simulated to identify timing failures and to verify the correctness of the inserted delay monitors. Experimental results demonstrate the applicability of the proposed design and verification methodology, thanks to an efficient sensor-aware abstraction methodology, by applying the flow to three complex case studies