On the use of testability measures for dependability assessment

Program “testability” is informally, the probability that a program will fail under test if it contains at least one fault. When a dependability assessment has to be derived from the observation of a series of failure free test executions (a common need for software subject to “ultra high reliability” requirements), measures of testability can-in theory-be used to draw inferences on program correctness. We rigorously investigate the concept of testability and its use in dependability assessment, criticizing, and improving on, previously published results. We give a general descriptive model of program execution and testing, on which the different measures of interest can be defined. We propose a more precise definition of program testability than that given by other authors, and discuss how to increase testing effectiveness without impairing program reliability in operation. We then study the mathematics of using testability to estimate, from test results: the probability of program correctness and the probability of failures. To derive the probability of program correctness, we use a Bayesian inference procedure and argue that this is more useful than deriving a classical “confidence level”. We also show that a high testability is not an unconditionally desirable property for a program. In particular, for programs complex enough that they are unlikely to be completely fault free, increasing testability may produce a program which will be less trustworthy, even after successful testin

Rigorously assessing software reliability and safety

This paper summarises the state of the art in the assessment of software reliability and safety ("dependability"), and describes some promising developments. A sound demonstration of very high dependability is still impossible before operation of the software; but research is finding ways to make rigorous assessment increasingly feasible. While refined mathematical techniques cannot take the place of factual knowledge, they can allow the decision-maker to draw more accurate conclusions from the knowledge that is available

Reliability of Mobile Agents for Reliable Service Discovery Protocol in MANET

Recently mobile agents are used to discover services in mobile ad-hoc network (MANET) where agents travel through the network, collecting and sometimes spreading the dynamically changing service information. But it is important to investigate how reliable the agents are for this application as the dependability issues(reliability and availability) of MANET are highly affected by its dynamic nature.The complexity of underlying MANET makes it hard to obtain the route reliability of the mobile agent systems (MAS); instead we estimate it using Monte Carlo simulation. Thus an algorithm for estimating the task route reliability of MAS (deployed for discovering services) is proposed, that takes into account the effect of node mobility in MANET. That mobility pattern of the nodes affects the MAS performance is also shown by considering different mobility models. Multipath propagation effect of radio signal is considered to decide link existence. Transient link errors are also considered. Finally we propose a metric to calculate the reliability of service discovery protocol and see how MAS performance affects the protocol reliability. The experimental results show the robustness of the proposed algorithm. Here the optimum value of network bandwidth (needed to support the agents) is calculated for our application. However the reliability of MAS is highly dependent on link failure probability

Software cost estimation

The paper gives an overview of the state of the art of software cost estimation (SCE). The main questions to be answered in the paper are: (1) What are the reasons for overruns of budgets and planned durations? (2) What are the prerequisites for estimating? (3) How can software development effort be estimated? (4) What can software project management expect from SCE models, how accurate are estimations which are made using these kind of models, and what are the pros and cons of cost estimation models

Uncertainty and sensitivity analysis in quantitative pest risk assessments : practical rules for risk assessors

Quantitative models have several advantages compared to qualitative methods for pest risk assessments (PRA). Quantitative models do not require the definition of categorical ratings and can be used to compute numerical probabilities of entry and establishment, and to quantify spread and impact. These models are powerful tools, but they include several sources of uncertainty that need to be taken into account by risk assessors and communicated to decision makers. Uncertainty analysis (UA) and sensitivity analysis (SA) are useful for analyzing uncertainty in models used in PRA, and are becoming more popular. However, these techniques should be applied with caution because several factors may influence their results. In this paper, a brief overview of methods of UA and SA are given. As well, a series of practical rules are defined that can be followed by risk assessors to improve the reliability of UA and SA results. These rules are illustrated in a case study based on the infection model of Magarey et al. (2005) where the results of UA and SA are shown to be highly dependent on the assumptions made on the probability distribution of the model inputs

Marker effects and examination reliability: a comparative exploration from the perspectives of generalizability theory, Rasch modelling and multilevel modelling

This study looked at how three different analysis methods could help us to understand rater effects on exam reliability. The techniques we looked at were: generalizability theory (G-theory) item response theory (IRT): in particular the Many-Facets Partial Credit Rasch Model (MFRM) multilevel modelling (MLM) We used data from AS component papers in geography and psychology for 2009, 2010 and 2011 from Edexcel.</p

Modeling software design diversity

Design diversity has been used for many years now as a means of achieving a degree of fault tolerance in software-based systems. Whilst there is clear evidence that the approach can be expected to deliver some increase in reliability compared with a single version, there is not agreement about the extent of this. More importantly, it remains difficult to evaluate exactly how reliable a particular diverse fault-tolerant system is. This difficulty arises because assumptions of independence of failures between different versions have been shown not to be tenable: assessment of the actual level of dependence present is therefore needed, and this is hard. In this tutorial we survey the modelling issues here, with an emphasis upon the impact these have upon the problem of assessing the reliability of fault tolerant systems. The intended audience is one of designers, assessors and project managers with only a basic knowledge of probabilities, as well as reliability experts without detailed knowledge of software, who seek an introduction to the probabilistic issues in decisions about design diversity

Acceptance Criteria for Critical Software Based on Testability Estimates and Test Results

Testability is defined as the probability that a program will fail a test, conditional on the program containing some fault. In this paper, we show that statements about the testability of a program can be more simply described in terms of assumptions on the probability distribution of the failure intensity of the program. We can thus state general acceptance conditions in clear mathematical terms using Bayesian inference. We develop two scenarios, one for software for which the reliability requirements are that the software must be completely fault-free, and another for requirements stated as an upper bound on the acceptable failure probability