Reasoning about the Reliability of Diverse Two-Channel Systems in which One Channel is "Possibly Perfect"

This paper considers the problem of reasoning about the reliability of fault-tolerant systems with two "channels" (i.e., components) of which one, A, supports only a claim of reliability, while the other, B, by virtue of extreme simplicity and extensive analysis, supports a plausible claim of "perfection." We begin with the case where either channel can bring the system to a safe state. We show that, conditional upon knowing pA (the probability that A fails on a randomly selected demand) and pB (the probability that channel B is imperfect), a conservative bound on the probability that the system fails on a randomly selected demand is simply pA.pB. That is, there is conditional independence between the events "A fails" and "B is imperfect." The second step of the reasoning involves epistemic uncertainty about (pA, pB) and we show that under quite plausible assumptions, a conservative bound on system pfd can be constructed from point estimates for just three parameters. We discuss the feasibility of establishing credible estimates for these parameters. We extend our analysis from faults of omission to those of commission, and then combine these to yield an analysis for monitored architectures of a kind proposed for aircraft

The credibility of health economic models for health policy decision-making: the case of population screening for abdominal aortic aneurysm

<i>Objectives</i>: To review health economic models of population screening for abdominal aortic aneurysm (AAA) among elderly males and assess their credibility for informing decision-making. <i>Methods</i>: A literature review identified health economic models of ultrasound screening for AAA. For each model focussing on population screening in elderly males, model structure and input parameter values were critically appraised using published good practice guidelines for decision analytic models. <i>Results</i>: Twelve models published between 1989 and 2003 were identified. Converting costs to a common currency and base year, substantial variability in cost-effectiveness results were revealed. Appraisals carried out for the nine models focusing on population screening showed differences in their complexity, with the simpler models generating results most favourable to screening. Eight of the nine models incorporated two or more simplifying structural assumptions favouring screening; uncertainty surrounding these assumptions was not investigated by any model. Quality assessments on a small number of parameters revealed input values varied between models, methods used to identify and incorporate input data were often not described, and few sensitivity analyses were reported. <i>Conclusions</i>: Large variation exists in the cost-effectiveness results generated by AAA screening models. The substantial number of factors potentially contributing to such disparities means that reconciliation of model results is impossible. In addition, poor reporting of methods makes it difficult to identify the most plausible and thus most useful model of those developed

GRADE Evidence to Decision (EtD) frameworks : A systematic and transparent approach to making well-informed healthcare choices. 1. Introduction

Funding: Work on this article has been partially funded by the European Commission FP7 Program (grant agreement 258583) as part of the DECIDE project. Sole responsibility lies with the authors; the European Commission is not responsible for any use that may be made of the information contained therein.Peer reviewedPublisher PD

Modeling good research practices - overview: a report of the ISPOR-SMDM modeling good research practices task force - 1.

Models—mathematical frameworks that facilitate estimation of the consequences of health care decisions—have become essential tools for health technology assessment. Evolution of the methods since the first ISPOR modeling task force reported in 2003 has led to a new task force, jointly convened with the Society for Medical Decision Making, and this series of seven papers presents the updated recommendations for best practices in conceptualizing models; implementing state–transition approaches, discrete event simulations, or dynamic transmission models; dealing with uncertainty; and validating and reporting models transparently. This overview introduces the work of the task force, provides all the recommendations, and discusses some quandaries that require further elucidation. The audience for these papers includes those who build models, stakeholders who utilize their results, and, indeed, anyone concerned with the use of models to support decision making

Support and Assessment for Fall Emergency Referrals (SAFER 1) trial protocol. Computerised on-scene decision support for emergency ambulance staff to assess and plan care for older people who have fallen: evaluation of costs and benefits using a pragmatic cluster randomised trial

Background: Many emergency ambulance calls are for older people who have fallen. As half of them are left at home, a community-based response may often be more appropriate than hospital attendance. The SAFER 1 trial will assess the costs and benefits of a new healthcare technology - hand-held computers with computerised clinical decision support (CCDS) software - to help paramedics decide who needs hospital attendance, and who can be safely left at home with referral to community falls services. Methods/Design: Pragmatic cluster randomised trial with a qualitative component. We shall allocate 72 paramedics ('clusters') at random between receiving the intervention and a control group delivering care as usual, of whom we expect 60 to complete the trial. Patients are eligible if they are aged 65 or older, live in the study area but not in residential care, and are attended by a study paramedic following an emergency call for a fall. Seven to 10 days after the index fall we shall offer patients the opportunity to opt out of further follow up. Continuing participants will receive questionnaires after one and 6 months, and we shall monitor their routine clinical data for 6 months. We shall interview 20 of these patients in depth. We shall conduct focus groups or semi-structured interviews with paramedics and other stakeholders. The primary outcome is the interval to the first subsequent reported fall (or death). We shall analyse this and other measures of outcome, process and cost by 'intention to treat'. We shall analyse qualitative data thematically. Discussion: Since the SAFER 1 trial received funding in August 2006, implementation has come to terms with ambulance service reorganisation and a new national electronic patient record in England. In response to these hurdles the research team has adapted the research design, including aspects of the intervention, to meet the needs of the ambulance services. In conclusion this complex emergency care trial will provide rigorous evidence on the clinical and cost effectiveness of CCDS for paramedics in the care of older people who have fallen

Discussing uncertainty and risk in primary care: recommendations of a multi-disciplinary panel regarding communication around prostate cancer screening.

BackgroundShared decision making improves value-concordant decision-making around prostate cancer screening (PrCS). Yet, PrCS discussions remain complex, challenging and often emotional for physicians and average-risk men.ObjectiveIn July 2011, the Centers for Disease Control and Prevention convened a multidisciplinary expert panel to identify priorities for funding agencies and development groups to promote evidence-based, value-concordant decisions between men at average risk for prostate cancer and their physicians.DesignTwo-day multidisciplinary expert panel in Atlanta, Georgia, with structured discussions and formal consensus processes.ParticipantsSixteen panelists represented diverse specialties (primary care, medical oncology, urology), disciplines (sociology, communication, medical education, clinical epidemiology) and market sectors (patient advocacy groups, Federal funding agencies, guideline-development organizations).Main measuresPanelists used guiding interactional and evaluation models to identify and rate strategies that might improve PrCS discussions and decisions for physicians, patients and health systems/society. Efficacy was defined as the likelihood of each strategy to impact outcomes. Effort was defined as the relative amount of effort to develop, implement and sustain the strategy. Each strategy was rated (1-7 scale; 7 = maximum) using group process software (ThinkTank(TM)). For each group, intervention strategies were grouped as financial/regulatory, educational, communication or attitudinal levers. For each strategy, barriers were identified.Key resultsHighly ranked strategies to improve value-concordant shared decision-making (SDM) included: changing outpatient clinic visit reimbursement to reward SDM; development of evidence-based, technology-assisted, point-of-service tools for physicians and patients; reframing confusing prostate cancer screening messages; providing pre-visit decision support interventions; utilizing electronic health records to promote benchmarking/best practices; providing additional training for physicians around value-concordant decision-making; and using re-accreditation to promote training.ConclusionsConference outcomes present an expert consensus of strategies likely to improve value-concordant prostate cancer screening decisions. In addition, the methodology used to obtain agreement provides a model of successful collaboration around this and future controversial cancer screening issues, which may be of interest to funding agencies, educators and policy makers

Evaluation of software dependability

It has been said that the term software engineering is an aspiration not a description. We would like to be able to claim that we engineer software, in the same sense that we engineer an aero-engine, but most of us would agree that this is not currently an accurate description of our activities. My suspicion is that it never will be. From the point of view of this essay – i.e. dependability evaluation – a major difference between software and other engineering artefacts is that the former is pure design. Its unreliability is always the result of design faults, which in turn arise as a result of human intellectual failures. The unreliability of hardware systems, on the other hand, has tended until recently to be dominated by random physical failures of components – the consequences of the ‘perversity of nature’. Reliability theories have been developed over the years which have successfully allowed systems to be built to high reliability requirements, and the final system reliability to be evaluated accurately. Even for pure hardware systems, without software, however, the very success of these theories has more recently highlighted the importance of design faults in determining the overall reliability of the final product. The conventional hardware reliability theory does not address this problem at all. In the case of software, there is no physical source of failures, and so none of the reliability theory developed for hardware is relevant. We need new theories that will allow us to achieve required dependability levels, and to evaluate the actual dependability that has been achieved, when the sources of the faults that ultimately result in failure are human intellectual failures

What guidance are researchers given on how to present network meta-analyses to end-users such as policymakers and clinicians? A systematic review

© 2014 Sullivan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Introduction: Network meta-analyses (NMAs) are complex methodological approaches that may be challenging for non-technical end-users, such as policymakers and clinicians, to understand. Consideration should be given to identifying optimal approaches to presenting NMAs that help clarify analyses. It is unclear what guidance researchers currently have on how to present and tailor NMAs to different end-users. Methods: A systematic review of NMA guidelines was conducted to identify guidance on how to present NMAs. Electronic databases and supplementary sources were searched for NMA guidelines. Presentation format details related to sample formats, target audiences, data sources, analysis methods and results were extracted and frequencies tabulated. Guideline quality was assessed following criteria developed for clinical practice guidelines. Results: Seven guidelines were included. Current guidelines focus on how to conduct NMAs but provide limited guidance to researchers on how to best present analyses to different end-users. None of the guidelines provided reporting templates. Few guidelines provided advice on tailoring presentations to different end-users, such as policymakers. Available guidance on presentation formats focused on evidence networks, characteristics of individual trials, comparisons between direct and indirect estimates and assumptions of heterogeneity and/or inconsistency. Some guidelines also provided examples of figures and tables that could be used to present information. Conclusions: Limited guidance exists for researchers on how best to present NMAs in an accessible format, especially for non-technical end-users such as policymakers and clinicians. NMA guidelines may require further integration with end-users' needs, when NMAs are used to support healthcare policy and practice decisions. Developing presentation formats that enhance understanding and accessibility of NMAs could also enhance the transparency and legitimacy of decisions informed by NMAs.The Canadian Institute of Health Research (CIHR) Drug Safety and Effectiveness Network (Funding reference number – 116573)