21,652 research outputs found
Recommended from our members
Comparing test sets and criteria in the presence of test hypotheses and fault domains
A number of authors have considered the problem of comparing test sets and criteria. Ideally
test sets are compared using a preorder with the property that test set T1 is at least as strong
as T2 if whenever T2 determines that an implementation p is faulty, T1 will also determine that
p is faulty. This notion can be extended to test criteria. However, it has been noted that very
few test sets and criteria are comparable under such an ordering; instead orderings are based
on weaker properties such as subsumes. This paper explores an alternative approach, in which
comparisons are made in the presence of a test hypothesis or fault domain. This approach allows
strong statements about fault detecting ability to be made and yet for a number of test sets and
criteria to be comparable. It may also drive incremental test generation
Recommended from our members
Using formal methods to support testing
Formal methods and testing are two important approaches that assist in the development of high quality software. While traditionally these approaches have been seen as rivals, in recent
years a new consensus has developed in which they are seen as complementary. This article reviews the state of the art regarding ways in which the presence of a formal specification can be used to assist testing
FORTEST: Formal methods and testing
Formal methods have traditionally been used for specification and development of software. However there are potential benefits for the testing stage as well. The panel session associated with this paper explores the usefulness
or otherwise of formal methods in various contexts for improving software testing. A number of different possibilities for the use of formal methods are explored and questions raised. The contributors are all members of the UK FORTEST Network on formal methods and testing. Although
the authors generally believe that formal methods
are useful in aiding the testing process, this paper is intended to provoke discussion. Dissenters are encouraged to put their views to the panel or individually to the authors
Making intelligent systems team players: Case studies and design issues. Volume 1: Human-computer interaction design
Initial results are reported from a multi-year, interdisciplinary effort to provide guidance and assistance for designers of intelligent systems and their user interfaces. The objective is to achieve more effective human-computer interaction (HCI) for systems with real time fault management capabilities. Intelligent fault management systems within the NASA were evaluated for insight into the design of systems with complex HCI. Preliminary results include: (1) a description of real time fault management in aerospace domains; (2) recommendations and examples for improving intelligent systems design and user interface design; (3) identification of issues requiring further research; and (4) recommendations for a development methodology integrating HCI design into intelligent system design
An Architecture for Dynamic Meta-Level Process Control for Model-Based Troubleshooting
There are numerous methods used for troubleshooting devices. Each method has certain domains, knowledge requirements, and assumptions required for it to perform well. However, oftentimes no one method by itself is sufficient to completely solve a troubleshooting problem. Therefore, an architecture is required to control the combined use of many problem solving methods. The combination of multiple problem solving methods makes the troubleshooting process more robust in terms of device domains that can be dealt with and quality of diagnoses produced. Troubleshooting has two tasks: diagnosis and problem resolution. This research provides an architecture that allows dynamic method selection during diagnosis. Dynamic method selection factors the current state of the diagnosis process along with other method parameters to determine which method to use to advance the diagnosis process. The architecture was developed by combining themes from diagnosis research that focused on dynamic multimethod diagnosis and its control. This work has produced several results. It provides an architecture to organize the methods and a basis for making control decisions concerning method use during diagnosis. It identifies a generous number of methods useful to perform diagnosis. It identifies the knowledge these methods require
Testing data types implementations from algebraic specifications
Algebraic specifications of data types provide a natural basis for testing
data types implementations. In this framework, the conformance relation is
based on the satisfaction of axioms. This makes it possible to formally state
the fundamental concepts of testing: exhaustive test set, testability
hypotheses, oracle. Various criteria for selecting finite test sets have been
proposed. They depend on the form of the axioms, and on the possibilities of
observation of the implementation under test. This last point is related to the
well-known oracle problem. As the main interest of algebraic specifications is
data type abstraction, testing a concrete implementation raises the issue of
the gap between the abstract description and the concrete representation. The
observational semantics of algebraic specifications bring solutions on the
basis of the so-called observable contexts. After a description of testing
methods based on algebraic specifications, the chapter gives a brief
presentation of some tools and case studies, and presents some applications to
other formal methods involving datatypes
A Fault-Based Model of Fault Localization Techniques
Every day, ordinary people depend on software working properly. We take it for granted; from banking software, to railroad switching software, to flight control software, to software that controls medical devices such as pacemakers or even gas pumps, our lives are touched by software that we expect to work. It is well known that the main technique/activity used to ensure the quality of software is testing. Often it is the only quality assurance activity undertaken, making it that much more important.
In a typical experiment studying these techniques, a researcher will intentionally seed a fault (intentionally breaking the functionality of some source code) with the hopes that the automated techniques under study will be able to identify the fault\u27s location in the source code. These faults are picked arbitrarily; there is potential for bias in the selection of the faults. Previous researchers have established an ontology for understanding or expressing this bias called fault size. This research captures the fault size ontology in the form of a probabilistic model. The results of applying this model to measure fault size suggest that many faults generated through program mutation (the systematic replacement of source code operators to create faults) are very large and easily found. Secondary measures generated in the assessment of the model suggest a new static analysis method, called testability, for predicting the likelihood that code will contain a fault in the future.
While software testing researchers are not statisticians, they nonetheless make extensive use of statistics in their experiments to assess fault localization techniques. Researchers often select their statistical techniques without justification. This is a very worrisome situation because it can lead to incorrect conclusions about the significance of research. This research introduces an algorithm, MeansTest, which helps automate some aspects of the selection of appropriate statistical techniques. The results of an evaluation of MeansTest suggest that MeansTest performs well relative to its peers. This research then surveys recent work in software testing using MeansTest to evaluate the significance of researchers\u27 work. The results of the survey indicate that software testing researchers are underreporting the significance of their work
CBR and MBR techniques: review for an application in the emergencies domain
The purpose of this document is to provide an in-depth analysis of current reasoning engine practice and the integration strategies of Case Based Reasoning and Model Based Reasoning that will be used in the design and development of the RIMSAT system.
RIMSAT (Remote Intelligent Management Support and Training) is a European Commission funded project designed to:
a.. Provide an innovative, 'intelligent', knowledge based solution aimed at improving the quality of critical decisions
b.. Enhance the competencies and responsiveness of individuals and organisations involved in highly complex, safety critical incidents - irrespective of their location.
In other words, RIMSAT aims to design and implement a decision support system that using Case Base Reasoning as well as Model Base Reasoning technology is applied in the management of emergency situations.
This document is part of a deliverable for RIMSAT project, and although it has been done in close contact with the requirements of the project, it provides an overview wide enough for providing a state of the art in integration strategies between CBR and MBR technologies.Postprint (published version
Testing conformance of a deterministic implementation against a non-deterministic stream X-machine
Stream X-machines are a formalisation of extended finite state machines that have been used to specify systems. One of the great benefits of using stream X-machines, for the purpose of specification, is the associated test generation technique which produces a test that is guaranteed to determine correctness under certain design for test conditions. This test generation algorithm has recently been extended to the case where the specification is non-deterministic. However, the algorithms for testing from a non-deterministic stream X-machine currently have limitations: either they test for equivalence, rather than conformance or they restrict the source of non-determinism allowed in the specification. This paper introduces a new test generation algorithm that overcomes both of these limitations, for situations where the implementation is known to be deterministic
- ā¦