Evaluating the cost-effectiveness of diagnostic tests in combination: is it important to allow for performance dependency?

OBJECTIVES: To investigate the importance of accounting for potential performance dependency when evaluating the cost-effectiveness of two diagnostic tests used in combination. METHODS: Two meta-analysis models were fitted to estimate the diagnostic accuracy of Wells score and Ddimer in combination. The first model assumes that the two tests perform independently of one another; thus, two separate meta-analyses were fitted to the Ddimer and Wells score data and then combined. The second model allows for any performance dependency of the two tests by incorporating published data on the accuracy of Ddimer stratified by Wells score, as well as studies of Ddimer alone and Wells score alone. The results from the two meta-analysis models were input into a decision model to assess the impact that assumptions regarding performance dependency have on the overall cost-effectiveness of the tests. RESULTS: The results highlight the importance of accounting for potential performance dependency when evaluating the cost-effectiveness of diagnostic tests used in combination. In our example, assuming the diagnostic performance of the two tests to be independent resulted in the strategy "Wells score moderate/high risk treated for DVT and Wells score low risk tested further with Ddimer" being identified as the most cost-effective at the £20,000 willingness-to-pay threshold (probability cost-effective 0.8). However, when performance dependency is modeled, the most cost-effective strategies were "Ddimer alone" and "Wells score low/moderate risk discharged and Wells score high risk further tested with Ddimer" (probability cost-effective 0.4). CONCLUSIONS: When evaluating the effectiveness and cost-effectiveness of diagnostic tests used in combination, failure to account for diagnostic performance dependency may lead to erroneous results and nonoptimal decision making

NETWORK META-ANALYSIS OF DIAGNOSTIC TEST ACCURACY STUDIES ALLOWING FOR MULTIPLE TESTS AT MULTIPLE THRESHOLDS FOR HEALTHCARE POLICY AND DECISION MAKING

Objectives Network meta-analyses have extensively been used to compare the effectiveness of multiple interventions for healthcare policy and decision-making. Methods for evaluating the performance of multiple diagnostic tests are less established. In a decision-making context, we are often interested in comparing and ranking the performance of multiple diagnostic tests, at varying levels of test thresholds. The aim of this research was to develop a network meta-analysis framework for evaluating multiple diagnostic tests, at varying test thresholds in one simultaneous analysis. Methods Motivated by an example of cognitive impairment diagnosis following stroke, we synthesized data from 13 studies assessing the efficiency of two diagnostic tests: Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA), at two test thresholds: MMSE <25/30 and <27/30, and MoCA <22/30 and <26/30. Using Markov Chain Monte Carlo (MCMC) methods, we fitted a bivariate network meta-analysis model, accounting for the correlations between multiple test accuracy measures from the same study, and incorporating constraints on increasing test thresholds assuming that higher test thresholds had an increased sensitivity but decreased specificity. Results We developed and successfully fitted a model comparing multiple tests/threshold combinations while imposing threshold constraints. Applying constraints on increasing test thresholds reduced the within-study variability and increased the precision in estimates of sensitivity and specificity. Using this model, we found that MoCA at threshold <26/30 appeared to have the best true positive rate (estimated sensitivity: 0.98; 95% credible interval (CrI): 0.94,0.99), whilst MMSE at threshold <25/30 appeared to have the best true negative rate (estimated specificity: 0.84, 95%CrI: 0.79,0.88). Conclusions In a health technology assessment setting, there is an increasing need to compare the efficiency of multiple diagnostics tests. The combined analysis of multiple tests at multiple thresholds allowed for more rigorous comparisons between competing diagnostics tests for decision-making

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

Background: The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate ‘lumping’ of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions. Methods: The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions. Results: The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective. Conclusions: Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making

The NightLife study — the clinical and cost-effectiveness of thrice-weekly, extended, in-centre nocturnal haemodialysis versus daytime haemodialysis using a mixed methods approach: study protocol for a randomised controlled trial

Background In-centre nocturnal haemodialysis (INHD) offers extended-hours haemodialysis, 6 to 8 h thrice-weekly overnight, with the support of dialysis specialist nurses. There is increasing observational data demonstrating potential benefits of INHD on health-related quality of life (HRQoL). There is a lack of randomised controlled trial (RCT) data to confirm these benefits and assess safety. Methods The NightLife study is a pragmatic, two-arm, multicentre RCT comparing the impact of 6 months INHD to conventional haemodialysis (thrice-weekly daytime in-centre haemodialysis, 3.5–5 h per session). The primary outcome is the total score from the Kidney Disease Quality of Life tool at 6 months. Secondary outcomes include sleep and cognitive function, measures of safety, adherence to dialysis and impact on clinical parameters. There is an embedded Process Evaluation to assess implementation, health economic modelling and a QuinteT Recruitment Intervention to understand factors that influence recruitment and retention. Adults (≥ 18 years old) who have been established on haemodialysis for > 3 months are eligible to participate. Discussion There are 68,000 adults in the UK that need kidney replacement therapy (KRT), with in-centre haemodialysis the treatment modality for over a third of cases. HRQoL is an independent predictor of hospitalisation and mortality in individuals on maintenance dialysis. Haemodialysis is associated with poor HRQoL in comparison to the general population. INHD has the potential to improve HRQoL. Vigorous RCT evidence of effectiveness is lacking. The NightLife study is an essential step in the understanding of dialysis therapies and will guide patient-centred decisions regarding KRT in the future.</p