Diagnostic accuracy studies: how to report and analyse inconclusive test results.

Failure to report inconclusive test results can lead to misleading conclusions regarding the accuracy and clinical usefulness of a diagnostic tool. We show that these results are often overlooked in research on test accuracy and provide guidance on suitable approaches to reporting and analysing these problematic results

What factors affect patients’ access to healthcare? Protocol for an overview of systematic reviews

Background The importance of access to healthcare for all is internationally recognised as a global goal, high on the global agenda. Yet inequalities in health exist within and between countries which are exacerbated by inequalities in access to healthcare. In order to address these inequalities, we need to better understand what drives them. While there exists a wealth of research on access to healthcare in different countries and contexts, and for different patient groups, to date no attempt has been made to bring this evidence together through a global lens. This study aims to address that gap by bringing together evidence of what factors affect patients’ access to healthcare and exploring how those factors vary in different countries and contexts around the world. Methods An overview of reviews will be conducted using a comprehensive search strategy to search four databases: Medline, Embase, Global Health and Cochrane Systematic Reviews. Additional searches will be conducted on the Gates Foundation, the World Health Organisation (WHO) and World Bank websites. Titles and abstracts will be screened against the eligibility criteria and full-text articles will be obtained for all records that meet the inclusion criteria or where there is uncertainty around eligibility. A data extraction table will be developed during the review process and will be piloted and refined before full data extraction commences. Methodological quality/risk of bias will be assessed for each included study using the AMSTAR 2 tool. The quality assessment will be used to inform the narrative synthesis, but a low-quality score will not necessarily lead to study exclusion. Discussion Factors affecting patients’ ability to access healthcare will be identified and analysed according to different country and context characteristics to shed light on the importance of different factors in different settings. Results will be interpreted accounting for the usual challenges associated with conducting such reviews. The results may guide future research in this area and contribute to priority setting for development initiatives aimed at ensuring equitable access to healthcare for all

Target Product Profiles for medical tests: a systematic review of current methods

Background: A Target Product Profile (TPP) outlines the necessary characteristics of an innovative product to address an unmet clinical need. TPPs could be used to better guide manufacturers in the development of ‘fit for purpose’ tests, thus increasing the likelihood that novel tests will progress from bench to bedside. However, there is currently no guidance on how to produce a TPP specifically for medical tests. Methods: A systematic review was conducted to summarise the methods currently used to develop TPPs for medical tests, the sources used to inform these recommendations and the test characteristics for which targets are made. Database and website searches were conducted in November 2018. TPPs written in English for any medical test were included. Based on an existing framework, test characteristics were clustered into commonly recognised themes. Results: Forty-four TPPs were identified, all of which focused on diagnostic tests for infectious diseases. Three core decision-making phases for developing TPPs were identified: scoping, drafting and consensus-building. Consultations with experts and the literature mostly informed the scoping and drafting of TPPs. All TPPs provided information on unmet clinical need and desirable analytical performance, and the majority specified clinical validity characteristics. Few TPPs described specifications for clinical utility, and none included cost-effectiveness. Conclusions: We have identified a commonly used framework that could be beneficial for anyone interested in drafting a TPP for a medical test. Currently, key outcomes such as utility and cost-effectiveness are largely overlooked within TPPs though and we foresee this as an area for further improvement

Point of care creatinine testing in diagnostic imaging: A feasibility study within the outpatient computed tomography setting

Introduction: Although the risks associated with iodinated contrast administration are acknowledged to be very low, screening of kidney function prior to administration is still standard practice in many hospitals. This study has evaluated the feasibility of implementing a screening form in conjunction with point of care (PoC) creatinine testing as a method to manage the risks of post contrast acute kidney injury (PC-AKI) within the CT imaging pathway. Method: Over an eight-week period 300 adult outpatients attending a UK CT department for contrast-enhanced scans were approached. Participants completed a screening questionnaire for co-morbidities linked to kidney dysfunction and consented to have a PoC and laboratory creatinine tests. Comparison was made against with previous baseline blood tests obtained within the preceding 3 months, as required by the study site. Participants were also invited to attend for follow up PoC and laboratory bloods tests at 48–72 h. Results: 14 patients (4.7%) had a scan-day eGFR below 45mL/min/1.73m2, all identified through screening. The majority of patients (n=281/300; 93.7%) fell in the same risk category based on previous and scan-day blood results. Six PoC test failures were recorded on the scan day. The constant error between the Abbott i-STAT PoC scan-day measurements and the laboratory scan-day measurements was -3.71 (95% CI: -6.41 to -0.50). Five patients had an elevated creatinine (≥25% from baseline) post contrast administration, but no instances of PC-AKI (≥50% from baseline) were identified. Conclusion: PoC creatinine testing is a practical method of ensuring renal function and is feasible in the radiology environment

What is the balance of benefits and harms for lung cancer screening with low-dose computed tomography?

After the National Lung Screening Trial (NLST)1 demonstrated improved lung cancer mortality almost a decade ago, hopes have been raised that low-dose computed tomography could be used to detect lung cancer in asymptomatic populations and improve outcomes by reducing the number of people diagnosed with advanced disease. The publication of the long-awaited Nederlands-Leuvens Longkanker Screenings Onderzoek (NELSON) trial last year2 has provided further evidence of a lung cancer mortality benefit and has provoked further calls to implement national population-based screening programmes, but the case for lung cancer screening remains controversial

The diagnostic accuracy of a single CEA blood test in detecting colorectal cancer recurrence: Results from the FACS trial

Objective: To evaluate the diagnostic accuracy of a single CEA (carcinoembryonic antigen) blood test in detecting colorectal cancer recurrence. Background: Patients who have undergone curative resection for primary colorectal cancer are typically followed up with scheduled CEA testing for 5 years. Decisions to investigate further (usually by CT imaging) are based on single test results, reflecting international guidelines. Methods: A secondary analysis was undertaken of data from the FACS trial (two arms included CEA testing). The composite reference standard applied included CT-CAP imaging, clinical assessment and colonoscopy. Accuracy in detecting recurrence was evaluated in terms of sensitivity, specificity, likelihood ratios, predictive values, time-dependent area under the ROC curves, and operational performance when used prospectively in clinical practice are reported. Results: Of 582 patients, 104 (17.9%) developed recurrence during the 5 year follow-up period. Applying the recommended threshold of 5µg/L achieves at best 50.0% sensitivity (95% CI: 40.1-59.9%); in prospective use in clinical practice it would lead to 56 missed recurrences (53.8%; 95% CI: 44.2-64.4%) and 89 false alarms (56.7% of 157 patients referred for investigation). Applying a lower threshold of 2.5µg/L would reduce the number of missed recurrences to 36.5% (95% CI: 26.5-46.5%) but would increase the false alarms to 84.2% (924/1097 referred). Some patients are more prone to false alarms than others – at the 5µg/L threshold, the 89 episodes of unnecessary investigation were clustered in 29 individuals. Conclusion: Our results demonstrated very low sensitivity for CEA, bringing to question whether it could ever be used as an independent triage test. It is not feasible to improve the diagnostic performance of a single test result by reducing the recommended action threshold because of the workload and false alarms generated. Current national and international guidelines merit re-evaluation and options to improve performance, such as making clinical decisions on the basis of CEA trend, should be further assessed

Common evidence gaps in point-of-care diagnostic test evaluation: a review of horizon scan reports

Objective: Since 2008, the Oxford Diagnostic Horizon Scan Programme has been identifying and summarising evidence on new and emerging diagnostic technologies relevant to primary care. We used these reports to determine the sequence and timing of evidence for new point-of-care diagnostic tests and to identify common evidence gaps in this process. Design: Systematic overview of diagnostic horizon scan reports. Primary outcome measures: We obtained the primary studies referenced in each horizon scan report (n=40) and extracted details of the study size, clinical setting and design characteristics. In particular, we assessed whether each study evaluated test accuracy, test impact or cost-effectiveness. The evidence for each point-of-care test was mapped against the Horvath framework for diagnostic test evaluation. Results: We extracted data from 500 primary studies. Most diagnostic technologies underwent clinical performance (ie, ability to detect a clinical condition) assessment (71.2%), with very few progressing to comparative clinical effectiveness (10.0%) and a cost-effectiveness evaluation (8.6%), even in the more established and frequently reported clinical domains, such as cardiovascular disease. The median time to complete an evaluation cycle was 9 years (IQR 5.5–12.5 years). The sequence of evidence generation was typically haphazard and some diagnostic tests appear to be implemented in routine care without completing essential evaluation stages such as clinical effectiveness. Conclusions: Evidence generation for new point-of-care diagnostic tests is slow and tends to focus on accuracy, and overlooks other test attributes such as impact, implementation and cost-effectiveness. Evaluation of this dynamic cycle and feeding back data from clinical effectiveness to refine analytical and clinical performance are key to improve the efficiency of point-of-care diagnostic test development and impact on clinically relevant outcomes. While the ‘road map’ for the steps needed to generate evidence are reasonably well delineated, we provide evidence on the complexity, length and variability of the actual process that many diagnostic technologies undergo

Development and Internal Validation of a Risk Prediction Model to Identify Myeloma Based on Routine Blood Tests: A Case-Control Study

Myeloma is one of the hardest cancers to diagnose in primary care due to its rarity and non-specific symptoms. A rate-limiting step in diagnosing myeloma is the clinician considering myeloma and initiating appropriate investigations. We developed and internally validated a risk prediction model to identify those with a high risk of having undiagnosed myeloma based on results from routine blood tests taken for other reasons. A case-control study, based on 367 myeloma cases and 1488 age- and sex-matched controls, was used to develop a risk prediction model including results from 15 blood tests. The model had excellent discrimination (C-statistic 0.85 (95%CI 0.83, 0.89)) and good calibration (calibration slope 0.87 (95%CI 0.75, 0.90)). At a prevalence of 15 per 100,000 population and a probability threshold of 0.4, approximately 600 patients would need additional reflex testing to detect one case. We showed that it is possible to combine signals and abnormalities from several routine blood test parameters to identify individuals at high-risk of having undiagnosed myeloma who may benefit from additional reflex testing. Further work is needed to explore the full potential of such a strategy, including whether it is clinically useful and cost-effective and how to make it ethically acceptable