Representation of Confidence in Assurance Cases Using the Beta Distribution

Assurance cases are used to document an argument that a system—such as a critical software system—satisfies some desirable property (e.g., safety, security, or reliability). Demonstrating high confidence that the claims made based on an assurance case can be trusted is crucial to the success of the case. Researchers have proposed quantification of confidence as a Baconian probability ratio of eliminated concerns about the assurance case to the total number of identified concerns. In this paper, we extend their work by mapping this discrete ratio to a continuous probability distribution—a beta distribution— enabling different visualizations of the confidence in a claim. Further, the beta distribution allows us to quantify and visualize the uncertainty associated with the expressed confidence. Additionally, by transforming the assurance case into a reasoning structure, we show how confidence calculations can be performed using beta distributions

The use of multilegged arguments to increase confidence in safety claims for software-based systems: A study based on a BBN analysis of an idealized example

The work described here concerns the use of so-called multi-legged arguments to support dependability claims about software-based systems. The informal justification for the use of multi-legged arguments is similar to that used to support the use of multi-version software in pursuit of high reliability or safety. Just as a diverse, 1-out-of-2 system might be expected to be more reliable than each of its two component versions, so a two-legged argument might be expected to give greater confidence in the correctness of a dependability claim (e.g. a safety claim) than would either of the argument legs alone. Our intention here is to treat these argument structures formally, in particular by presenting a formal probabilistic treatment of ‘confidence’, which will be used as a measure of efficacy. This will enable claims for the efficacy of the multi-legged approach to be made quantitatively, answering questions such as ‘How much extra confidence about a system’s safety will I have if I add a verification argument leg to an argument leg based upon statistical testing?’ For this initial study, we concentrate on a simplified and idealized example of a safety system in which interest centres upon a claim about the probability of failure on demand. Our approach is to build a BBN (“Bayesian Belief Network”) model of a two-legged argument, and manipulate this analytically via parameters that define its node probability tables. The aim here is to obtain greater insight than is afforded by the more usual BBN treatment, which involves merely numerical manipulation. We show that the addition of a diverse second argument leg can, indeed, increase confidence in a dependability claim: in a reasonably plausible example the doubt in the claim is reduced to one third of the doubt present in the original single leg. However, we also show that there can be some unexpected and counter-intuitive subtleties here; for example an entirely supportive second leg can sometimes undermine an original argument, resulting overall in less confidence than came from this original argument. Our results are neutral on the issue of whether such difficulties will arise in real life - i.e. when real experts judge real systems

Likelihood, Replicability, and Robbins' Confidence Sequences

The widely claimed replicability crisis in science may lead to revised standards of significance. The customary frequentist confidence intervals, calibrated through hypothetical repetitions of the experiment that is supposed to have produced the data at hand, rely on a feeble concept of replica- bility. In particular, contradictory conclusions may be reached when a substantial enlargement of the study is undertaken. To redefine statistical confidence in such a way that inferential conclusions are non-contradictory, with large enough probability, under enlargements of the sample, we give a new reading of a proposal dating back to the 60s, namely, Robbins\u2019 confidence sequences. Directly bounding the probability of reaching, in the future, conclusions that contradict the current ones, Robbins\u2019 confidence sequences ensure a clear-cut form of replicability when inference is performed on accumulating data. Their main frequentist property is easy to understand and to prove. We show that Robbins\u2019 confidence sequences may be justified under various views of inference: they are likelihood-based, can incorporate prior information and obey the strong likelihood principle. They are easy to compute, even when inference is on a parameter of interest, especially using a closed form approximation from normal asymptotic theory

Measuring Confidence of Assurance Cases in Safety-Critical Domains

Evaluation of assurance cases typically requires certifiers’ domain knowledge and experience, and, as such, most software certification has been conducted manually. Given the advancement in uncertainty theories and software traceability, we envision that these technologies can synergistically be combined and leveraged to offer some degree of automation to improve the certifiers’ capability to perform software certification. To this end, we present DS4AC, a novel confidence calculation framework that 1) applies the Dempster-Shafer theory to calculate the confidence between a parent claim and its children claims; and 2) uses the vector space model to evaluate the confidence for the evidence items using traceability information. We illustrate our approach on two different applications, where safety is the key property of interest for both systems. In both cases, we use the Goal Structuring Notation to represent the respective assurance cases and provide proof of concept results that demonstrate the DS4AC framework can automate portions of the evaluation of assurance cases, thereby reducing the burden of manual certification process

Risk modeling concepts relating to the design and rating of agricultural insurance contracts

The authors identify the key issues and concerns that arise in the design and rating of crop yield insurance plans, with a particular emphasis on production risk modeling. The authors show how the availability of data shapes the insurance scheme and the ratemaking procedures. Relying on the U.S. experience and recent developments in statistics and econometrics, they review risk modeling concepts and provide technical guidelines in the development of crop insurance plans. Finally, they show how these risk modeling techniques can be extended to price risk in order to develop crop revenue insurance schemes.Health Economics&Finance,Insurance Law,Environmental Economics&Policies,Insurance&Risk Mitigation,Labor Policies,Insurance&Risk Mitigation,Crops&Crop Management Systems,Health Economics&Finance,Insurance Law,Environmental Economics&Policies

An Investigation of Proposed Techniques for Quantifying Confidence in Assurance Arguments

The use of safety cases in certification raises the question of assurance argument sufficiency and the issue of confidence (or uncertainty) in the argument's claims. Some researchers propose to model confidence quantitatively and to calculate confidence in argument conclusions. We know of little evidence to suggest that any proposed technique would deliver trustworthy results when implemented by system safety practitioners. Proponents do not usually assess the efficacy of their techniques through controlled experiment or historical study. Instead, they present an illustrative example where the calculation delivers a plausible result. In this paper, we review current proposals, claims made about them, and evidence advanced in favor of them. We then show that proposed techniques can deliver implausible results in some cases. We conclude that quantitative confidence techniques require further validation before they should be recommended as part of the basis for deciding whether an assurance argument justifies fielding a critical system

Can we verify and intrinsically validate risk assessment results? What progress is being made to increase QRA trustworthiness?

PresentationThe purpose of a risk assessment is to make a decision whether the risk of a given situation is acceptable, and, if not, how we can reduce it to a tolerable level. For many cases, this can be done in a semi-quantitative fashion. For more complex or problematic cases a quantitative approach is required. Anybody who has been involved in such a study is aware of the difficulties and pitfalls. Despite proven software many choices of parameters must be made and many uncertainties remain. The thoroughness of the study can make quite a difference in the result. Independently, analysts can arrive at results that differ orders of magnitude, especially if uncertainties are not included. Because for important decisions on capital projects there are always proponents and opponents, there is often a tense situation in which conflict is looming. The paper will first briefly review a standard procedure introduced for safety cases on products that must provide more or less a guarantee that the risk of use is below a certain value. Next will be the various approaches how to deal with uncertainties in a quantitative risk assessment and the follow-on decision process. Over the last few years several new developments have been made to achieve, to a certain extent, a hold on so-called deep uncertainty. Expert elicitation and its limitations is another aspect. The paper will be concluded with some practical recommendations

Bacterial pathogens and resistance causing community acquired paediatric bloodstream infections in low- and middle-income countries: a systematic review and meta-analysis

Background Despite a high mortality rate in childhood, there is limited evidence on the causes and outcomes of paediatric bloodstream infections from low- and middle-income countries (LMICs). We conducted a systematic review and meta-analysis to characterize the bacterial causes of paediatric bloodstream infections in LMICs and their resistance profile. Methods We searched Pubmed and Embase databases between January 1st 1990 and October 30th 2019, combining MeSH and free-text terms for “sepsis” and “low-middle-income countries” in children. Two reviewers screened articles and performed data extraction to identify studies investigating children (1 month-18 years), with at least one blood culture. The main outcomes of interests were the rate of positive blood cultures, the distribution of bacterial pathogens, the resistance patterns and the case-fatality rate. The proportions obtained from each study were pooled using the Freeman-Tukey double arcsine transformation, and a random-effect meta-analysis model was used. Results We identified 2403 eligible studies, 17 were included in the final review including 52,915 children (11 in Africa and 6 in Asia). The overall percentage of positive blood culture was 19.1% [95% CI: 12.0–27.5%]; 15.5% [8.4–24.4%] in Africa and 28.0% [13.2–45.8%] in Asia. A total of 4836 bacterial isolates were included in the studies; 2974 were Gram-negative (63.9% [52.2–74.9]) and 1858 were Gram-positive (35.8% [24.9–47.5]). In Asia, Salmonella typhi (26.2%) was the most commonly isolated pathogen, followed by Staphylococcus aureus (7.7%) whereas in Africa, S. aureus (17.8%) and Streptococcus pneumoniae (16.8%) were predominant followed by Escherichia coli (10.7%). S. aureus was more likely resistant to methicillin in Africa (29.5% vs. 7.9%), whereas E. coli was more frequently resistant to third-generation cephalosporins (31.2% vs. 21.2%), amikacin (29.6% vs. 0%) and ciprofloxacin (36.7% vs. 0%) in Asia. The overall estimate for case-fatality rate among 8 studies was 12.7% [6.6–20.2%]. Underlying conditions, such as malnutrition or HIV infection were assessed as a factor associated with bacteraemia in 4 studies each. Conclusions We observed a marked variation in pathogen distribution and their resistance profiles between Asia and Africa. Very limited data is available on underlying risk factors for bacteraemia, patterns of treatment of multidrug-resistant infections and predictors of adverse outcomes