54,550 research outputs found
Application of Bayesian Networks to Coverage Directed Test Generation for the Verification of Digital Hardware Designs
Functional verification is generally regarded as the most critical phase in the successful development of digital integrated circuits. The increasing complexity and size of chip designs make it more challenging to find bugs and meet test coverage goals in time for market demands. These challenges have led to more automated methods of simulation with constrained random test generation and coverage analysis. Recent goals in industry have focused on improving the process further by applying Coverage Directed Test Generation (CDG) to automate the feedback from coverage analysis to test input generation. Previous research has presented Bayesian networks as a way to achieve CDG. Bayesian networks provide a means of capturing behaviors of a design under verification and making predictions to help guide test input generation to reach coverage goals more quickly. Previous research has shown methods for defining a Bayesian network for a design domain and generating input parameters for dynamic simulation. This thesis demonstrates that existing commercial verification tools can be combined with a Bayesian inference engine as a feasible solution for creating a fully automated CDG environment. This solution is demonstrated using methods from previous research for applying Bayesian networks to verification. The CDG framework was implemented by combining the Questa verification platform with the Bayesian inference engine SMILE (Structural Modeling, Inference, and Learning Engine) in a single simulation environment. SystemVerilog testbenches and custom software were created to automatically find coverage holes, predict test input parameters, and dynamically change these parameters to complete verification with a fewer number of test cases. The CDG framework was demonstrated by performing verification on both a combinational logic design and a sequential logic design. The results show that Bayesian networks can be successfully used to improve the efficiency and quality of the verification process
Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing
Context: Verification and validation (V&V) activities make up 20 to 50
percent of the total development costs of a software system in practice. Test
automation is proposed to lower these V&V costs but available research only
provides limited empirical data from industrial practice about the maintenance
costs of automated tests and what factors affect these costs. In particular,
these costs and factors are unknown for automated GUI-based testing.
Objective: This paper addresses this lack of knowledge through analysis of
the costs and factors associated with the maintenance of automated GUI-based
tests in industrial practice.
Method: An empirical study at two companies, Siemens and Saab, is reported
where interviews about, and empirical work with, Visual GUI Testing is
performed to acquire data about the technique's maintenance costs and
feasibility.
Results: 13 factors are observed that affect maintenance, e.g. tester
knowledge/experience and test case complexity. Further, statistical analysis
shows that developing new test scripts is costlier than maintenance but also
that frequent maintenance is less costly than infrequent, big bang maintenance.
In addition a cost model, based on previous work, is presented that estimates
the time to positive return on investment (ROI) of test automation compared to
manual testing.
Conclusions: It is concluded that test automation can lower overall software
development costs of a project whilst also having positive effects on software
quality. However, maintenance costs can still be considerable and the less time
a company currently spends on manual testing, the more time is required before
positive, economic, ROI is reached after automation
Recommended from our members
Using formal methods to support testing
Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent
years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing
Automatic instantiation of abstract tests on specific configurations for large critical control systems
Computer-based control systems have grown in size, complexity, distribution
and criticality. In this paper a methodology is presented to perform an
abstract testing of such large control systems in an efficient way: an abstract
test is specified directly from system functional requirements and has to be
instantiated in more test runs to cover a specific configuration, comprising
any number of control entities (sensors, actuators and logic processes). Such a
process is usually performed by hand for each installation of the control
system, requiring a considerable time effort and being an error prone
verification activity. To automate a safe passage from abstract tests, related
to the so called generic software application, to any specific installation, an
algorithm is provided, starting from a reference architecture and a state-based
behavioural model of the control software. The presented approach has been
applied to a railway interlocking system, demonstrating its feasibility and
effectiveness in several years of testing experience
Discovery and Selection of Certified Web Services Through Registry-Based Testing and Verification
Reliability and trust are fundamental prerequisites for the establishment of functional relationships among peers in a Collaborative Networked Organisation (CNO), especially in the context of Virtual Enterprises where economic benefits can be directly at stake. This paper presents a novel approach towards effective service discovery and selection that is no longer based on informal, ambiguous and potentially unreliable service descriptions, but on formal specifications that can be used to verify and certify the actual Web service implementations. We propose the use of Stream X-machines (SXMs) as a powerful modelling formalism for constructing the behavioural specification of a Web service, for performing verification through the generation of exhaustive test cases, and for performing validation through animation or model checking during service selection
Metamodel Instance Generation: A systematic literature review
Modelling and thus metamodelling have become increasingly important in
Software Engineering through the use of Model Driven Engineering. In this paper
we present a systematic literature review of instance generation techniques for
metamodels, i.e. the process of automatically generating models from a given
metamodel. We start by presenting a set of research questions that our review
is intended to answer. We then identify the main topics that are related to
metamodel instance generation techniques, and use these to initiate our
literature search. This search resulted in the identification of 34 key papers
in the area, and each of these is reviewed here and discussed in detail. The
outcome is that we are able to identify a knowledge gap in this field, and we
offer suggestions as to some potential directions for future research.Comment: 25 page
Verifying the Safety of a Flight-Critical System
This paper describes our work on demonstrating verification technologies on a
flight-critical system of realistic functionality, size, and complexity. Our
work targeted a commercial aircraft control system named Transport Class Model
(TCM), and involved several stages: formalizing and disambiguating requirements
in collaboration with do- main experts; processing models for their use by
formal verification tools; applying compositional techniques at the
architectural and component level to scale verification. Performed in the
context of a major NASA milestone, this study of formal verification in
practice is one of the most challenging that our group has performed, and it
took several person months to complete it. This paper describes the methodology
that we followed and the lessons that we learned.Comment: 17 pages, 5 figure
- …