Diagnostic accuracy studies: how to report and analyse inconclusive test results.

Failure to report inconclusive test results can lead to misleading conclusions regarding the accuracy and clinical usefulness of a diagnostic tool. We show that these results are often overlooked in research on test accuracy and provide guidance on suitable approaches to reporting and analysing these problematic results

Target Product Profiles for medical tests: a systematic review of current methods

Background: A Target Product Profile (TPP) outlines the necessary characteristics of an innovative product to address an unmet clinical need. TPPs could be used to better guide manufacturers in the development of ‘fit for purpose’ tests, thus increasing the likelihood that novel tests will progress from bench to bedside. However, there is currently no guidance on how to produce a TPP specifically for medical tests. Methods: A systematic review was conducted to summarise the methods currently used to develop TPPs for medical tests, the sources used to inform these recommendations and the test characteristics for which targets are made. Database and website searches were conducted in November 2018. TPPs written in English for any medical test were included. Based on an existing framework, test characteristics were clustered into commonly recognised themes. Results: Forty-four TPPs were identified, all of which focused on diagnostic tests for infectious diseases. Three core decision-making phases for developing TPPs were identified: scoping, drafting and consensus-building. Consultations with experts and the literature mostly informed the scoping and drafting of TPPs. All TPPs provided information on unmet clinical need and desirable analytical performance, and the majority specified clinical validity characteristics. Few TPPs described specifications for clinical utility, and none included cost-effectiveness. Conclusions: We have identified a commonly used framework that could be beneficial for anyone interested in drafting a TPP for a medical test. Currently, key outcomes such as utility and cost-effectiveness are largely overlooked within TPPs though and we foresee this as an area for further improvement

What factors affect patients’ access to healthcare? Protocol for an overview of systematic reviews

Background The importance of access to healthcare for all is internationally recognised as a global goal, high on the global agenda. Yet inequalities in health exist within and between countries which are exacerbated by inequalities in access to healthcare. In order to address these inequalities, we need to better understand what drives them. While there exists a wealth of research on access to healthcare in different countries and contexts, and for different patient groups, to date no attempt has been made to bring this evidence together through a global lens. This study aims to address that gap by bringing together evidence of what factors affect patients’ access to healthcare and exploring how those factors vary in different countries and contexts around the world. Methods An overview of reviews will be conducted using a comprehensive search strategy to search four databases: Medline, Embase, Global Health and Cochrane Systematic Reviews. Additional searches will be conducted on the Gates Foundation, the World Health Organisation (WHO) and World Bank websites. Titles and abstracts will be screened against the eligibility criteria and full-text articles will be obtained for all records that meet the inclusion criteria or where there is uncertainty around eligibility. A data extraction table will be developed during the review process and will be piloted and refined before full data extraction commences. Methodological quality/risk of bias will be assessed for each included study using the AMSTAR 2 tool. The quality assessment will be used to inform the narrative synthesis, but a low-quality score will not necessarily lead to study exclusion. Discussion Factors affecting patients’ ability to access healthcare will be identified and analysed according to different country and context characteristics to shed light on the importance of different factors in different settings. Results will be interpreted accounting for the usual challenges associated with conducting such reviews. The results may guide future research in this area and contribute to priority setting for development initiatives aimed at ensuring equitable access to healthcare for all

Point of care creatinine testing in diagnostic imaging: A feasibility study within the outpatient computed tomography setting

Introduction: Although the risks associated with iodinated contrast administration are acknowledged to be very low, screening of kidney function prior to administration is still standard practice in many hospitals. This study has evaluated the feasibility of implementing a screening form in conjunction with point of care (PoC) creatinine testing as a method to manage the risks of post contrast acute kidney injury (PC-AKI) within the CT imaging pathway. Method: Over an eight-week period 300 adult outpatients attending a UK CT department for contrast-enhanced scans were approached. Participants completed a screening questionnaire for co-morbidities linked to kidney dysfunction and consented to have a PoC and laboratory creatinine tests. Comparison was made against with previous baseline blood tests obtained within the preceding 3 months, as required by the study site. Participants were also invited to attend for follow up PoC and laboratory bloods tests at 48–72 h. Results: 14 patients (4.7%) had a scan-day eGFR below 45mL/min/1.73m2, all identified through screening. The majority of patients (n=281/300; 93.7%) fell in the same risk category based on previous and scan-day blood results. Six PoC test failures were recorded on the scan day. The constant error between the Abbott i-STAT PoC scan-day measurements and the laboratory scan-day measurements was -3.71 (95% CI: -6.41 to -0.50). Five patients had an elevated creatinine (≥25% from baseline) post contrast administration, but no instances of PC-AKI (≥50% from baseline) were identified. Conclusion: PoC creatinine testing is a practical method of ensuring renal function and is feasible in the radiology environment

Toward a Framework for Outcome-Based Analytical Performance Specifications: A Methodology Review of Indirect Methods for Evaluating the Impact of Measurement Uncertainty on Clinical Outcomes

Background: For medical tests that have a central role in clinical decision-making, current guidelines advocate outcome-based analytical performance specifications. Given that empirical (clinical-trial style) analyses are often impractical or unfeasible in this context, the ability to set such specifications is expected to rely on indirect studies to calculate the impact of test measurement uncertainty on downstream clinical, operational and economic outcomes. Currently however, a lack of awareness and guidance concerning available alternative indirect methods is limiting the production of outcome-based specifications. Our aim therefore was to review available indirect methods and present an analytical framework to inform future outcome-based performance goals. Content: A methodology review consisting of database searches and extensive citation tracking was conducted to identify studies using indirect methods to incorporate or evaluate the impact of test measurement uncertainty on downstream outcomes (including clinical accuracy, clinical utility and/or costs). Eighty-two studies were identified, most of which evaluated the impact of imprecision and/or bias on clinical accuracy. A common analytical framework underpinning the various methods was identified, consisting of three key steps: (1) calculation of “true” test values; (2) calculation of measured test values (incorporating uncertainty); and (3) calculation of the impact of discrepancies between (1) and (2) on specified outcomes. A summary of the methods adopted is provided, and key considerations discussed. Conclusions: Various approaches are available for conducting indirect assessments to inform outcome-based performance specifications. This study provides an overview of methods and key considerations to inform future studies and research in this area

Common evidence gaps in point-of-care diagnostic test evaluation: a review of horizon scan reports

Objective: Since 2008, the Oxford Diagnostic Horizon Scan Programme has been identifying and summarising evidence on new and emerging diagnostic technologies relevant to primary care. We used these reports to determine the sequence and timing of evidence for new point-of-care diagnostic tests and to identify common evidence gaps in this process. Design: Systematic overview of diagnostic horizon scan reports. Primary outcome measures: We obtained the primary studies referenced in each horizon scan report (n=40) and extracted details of the study size, clinical setting and design characteristics. In particular, we assessed whether each study evaluated test accuracy, test impact or cost-effectiveness. The evidence for each point-of-care test was mapped against the Horvath framework for diagnostic test evaluation. Results: We extracted data from 500 primary studies. Most diagnostic technologies underwent clinical performance (ie, ability to detect a clinical condition) assessment (71.2%), with very few progressing to comparative clinical effectiveness (10.0%) and a cost-effectiveness evaluation (8.6%), even in the more established and frequently reported clinical domains, such as cardiovascular disease. The median time to complete an evaluation cycle was 9 years (IQR 5.5–12.5 years). The sequence of evidence generation was typically haphazard and some diagnostic tests appear to be implemented in routine care without completing essential evaluation stages such as clinical effectiveness. Conclusions: Evidence generation for new point-of-care diagnostic tests is slow and tends to focus on accuracy, and overlooks other test attributes such as impact, implementation and cost-effectiveness. Evaluation of this dynamic cycle and feeding back data from clinical effectiveness to refine analytical and clinical performance are key to improve the efficiency of point-of-care diagnostic test development and impact on clinically relevant outcomes. While the ‘road map’ for the steps needed to generate evidence are reasonably well delineated, we provide evidence on the complexity, length and variability of the actual process that many diagnostic technologies undergo

The diagnostic accuracy of a single CEA blood test in detecting colorectal cancer recurrence: Results from the FACS trial

Objective: To evaluate the diagnostic accuracy of a single CEA (carcinoembryonic antigen) blood test in detecting colorectal cancer recurrence. Background: Patients who have undergone curative resection for primary colorectal cancer are typically followed up with scheduled CEA testing for 5 years. Decisions to investigate further (usually by CT imaging) are based on single test results, reflecting international guidelines. Methods: A secondary analysis was undertaken of data from the FACS trial (two arms included CEA testing). The composite reference standard applied included CT-CAP imaging, clinical assessment and colonoscopy. Accuracy in detecting recurrence was evaluated in terms of sensitivity, specificity, likelihood ratios, predictive values, time-dependent area under the ROC curves, and operational performance when used prospectively in clinical practice are reported. Results: Of 582 patients, 104 (17.9%) developed recurrence during the 5 year follow-up period. Applying the recommended threshold of 5µg/L achieves at best 50.0% sensitivity (95% CI: 40.1-59.9%); in prospective use in clinical practice it would lead to 56 missed recurrences (53.8%; 95% CI: 44.2-64.4%) and 89 false alarms (56.7% of 157 patients referred for investigation). Applying a lower threshold of 2.5µg/L would reduce the number of missed recurrences to 36.5% (95% CI: 26.5-46.5%) but would increase the false alarms to 84.2% (924/1097 referred). Some patients are more prone to false alarms than others – at the 5µg/L threshold, the 89 episodes of unnecessary investigation were clustered in 29 individuals. Conclusion: Our results demonstrated very low sensitivity for CEA, bringing to question whether it could ever be used as an independent triage test. It is not feasible to improve the diagnostic performance of a single test result by reducing the recommended action threshold because of the workload and false alarms generated. Current national and international guidelines merit re-evaluation and options to improve performance, such as making clinical decisions on the basis of CEA trend, should be further assessed