Determining a risk-proportionate approach to the validation of statistical programming for clinical trials

Background: The contribution of the statistician to the design and analysis of a clinical trial is acknowledged as essential. Ability to reconstruct the statistical contribution to a trial requires rigorous and transparent documentation as evidenced by the reproducibility of results. The process of validating statistical programmes is a key requirement. While guidance relating to software development and life cycle methodologies details steps for validation by information systems developers, there is no guidance applicable to programmes written by statisticians. We aimed to develop a risk-based approach to the validation of statistical programming that would support scientific integrity and efficient resource use within clinical trials units. // Methods: The project was embedded within the Information Systems Operational Group and the Statistics Operational Group of the UK Clinical Research Collaboration Registered Clinical Trials Unit network. Members were asked to share materials relevant to validation of statistical programming. A review of the published literature, regulatory guidance and knowledge of relevant working groups was undertaken. Surveys targeting the Information Systems Operational Group and Statistics Operational Group were developed to determine current practices across the Registered Clinical Trials Unit network. A risk-based approach was drafted and used as a basis for a workshop with representation from statisticians, information systems developers and quality assurance managers (n = 15). The approach was subsequently modified and presented at a second, larger scale workshop (n = 47) to gain a wider perspective, with discussion of content and implications for delivery. The approach was revised based on the discussions and suggestions made. The workshop was attended by a member of the Medicines for Healthcare products Regulatory Agency Inspectorate who also provided comments on the revised draft. // Results: Types of statistical programming were identified and categorised into six areas: generation of randomisation lists; programmes to explore/understand the data; data cleaning, including complex checks; derivations including data transformations; data monitoring; or interim and final analysis. The risk-based approach considers each category of statistical programme against the impact of an error and its likelihood, whether the programming can be fully prespecified, the need for repeated use and the need for reproducibility. Approaches to the validation of programming within each category are proposed. // Conclusion: We have developed a risk-based approach to the validation of statistical programming. It endeavours to facilitate the implementation of targeted quality assurance measures while making efficient use of limited resources

Prevalence and socio-demographic correlates of cooking skills in UK adults: cross-sectional analysis of data from the UK National Diet and Nutrition Survey

Background Poor cooking skills may be a barrier to healthy eating and a contributor to overweight and obesity. Little population-representative data on adult cooking skills has been published. We explored prevalence and socio-demographic correlates of cooking skills among adult respondents to wave 1 of the UK National Diet and Nutrition Survey (2008–9).  Methods Socio-demographic variables of interest were sex, age group, occupational socio-economic group and whether or not respondents had the main responsibility for food in their households. Cooking skills were assessed as self-reported confidence in using eight cooking techniques, confidence in cooking ten foods, and ability to prepare four types of dish (convenience foods, a complete meal from ready-made ingredients, a main meal from basic ingredients, and cake or biscuits from basic ingredients). Frequency of preparation of main meals was also reported.  Results Of 509 respondents, almost two-thirds reported cooking a main meal at least five times per week. Around 90% reported being able to cook convenience foods, a complete meal from ready-made ingredient, and a main dish from basic ingredients without help. Socio-demographic differences in all markers of cooking skills were scattered and inconsistent. Where these were found, women and main food providers were most likely to report confidence with foods, techniques or dishes, and respondents in the youngest age (19–34 years) and lowest socio-economic group least likely.  Conclusions This is the only exploration of the prevalence and socio-demographic correlates of adult cooking skills using recent and population-representative UK data and adds to the international literature on cooking skills in developed countries. Reported confidence with using most cooking techniques and preparing most foods was high. There were few socio-demographic differences in reported cooking skills. Adult cooking skills interventions are unlikely to have a large population impact, but may have important individual effects if clearly targeted at: men, younger adults, and those in the least affluent social groups

Good Statistical Practice-development of tailored Good Clinical Practice training for statisticians

BACKGROUND: Statisticians are fundamental in ensuring clinical research, including clinical trials, are conducted with quality, transparency, reproducibility and integrity. Good Clinical Practice (GCP) is an international quality standard for the conduct of clinical trials research. Statisticians are required to undertake training on GCP but existing training is generic and, crucially, does not cover statistical activities. This results in statisticians undertaking training mostly unrelated to their role and variation in awareness and implementation of relevant regulatory requirements with regards to statistical conduct. The need for role-relevant training is recognised by the UK NHS Health Research Authority and the Medicines and Healthcare products Regulatory Agency (MHRA).METHODS: The Good Statistical Practice (GCP for Statisticians) project was instigated by the UK Clinical Research Collaboration (UKCRC) Registered Clinical Trials Unit (CTU) Statisticians Operational Group and funded by the National Institute for Health and Care Research (NIHR), to develop materials to enable role-specific GCP training tailored to statisticians. Review of current GCP training was undertaken by survey. Development of training materials were based on MHRA GCP. Critical review and piloting was conducted with UKCRC CTU and NIHR researchers with comment from MHRA. Final review was conducted through the UKCRC CTU Statistics group.RESULTS: The survey confirmed the need and desire for the development of dedicated GCP training for statisticians. An accessible, comprehensive, piloted training package was developed tailored to statisticians working in clinical research, particularly the clinical trials arena. The training materials cover legislation and guidance for best practice across all clinical trial processes with statistical involvement, including exercises and real-life scenarios to bridge the gap between theory and practice. Comprehensive feedback was incorporated. The training materials are freely available for national and international adoption.CONCLUSION: All research staff should have training in GCP yet the training undertaken by most academic statisticians does not cover activities related to their role. The Good Statistical Practice (GCP for Statisticians) project has developed and extensively piloted new, role-specific, comprehensive, accessible GCP training tailored to statisticians working in clinical research, particularly the clinical trials arena. This role-specific training will encourage best practice, leading to transparent and reproducible statistical activity, as required by regulatory authorities and funders.</p

Artificial intelligence in digital pathology: a diagnostic test accuracy systematic review and meta-analysis

Ensuring diagnostic performance of AI models before clinical use is key to the safe and successful adoption of these technologies. Studies reporting AI applied to digital pathology images for diagnostic purposes have rapidly increased in number in recent years. The aim of this work is to provide an overview of the diagnostic accuracy of AI in digital pathology images from all areas of pathology. This systematic review and meta-analysis included diagnostic accuracy studies using any type of artificial intelligence applied to whole slide images (WSIs) in any disease type. The reference standard was diagnosis through histopathological assessment and / or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. We identified 2976 studies, of which 100 were included in the review and 48 in the full meta-analysis. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model. 100 studies were identified for inclusion, equating to over 152,000 whole slide images (WSIs) and representing many disease types. Of these, 48 studies were included in the meta-analysis. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4) for AI. There was substantial heterogeneity in study design and all 100 studies identified for inclusion had at least one area at high or unclear risk of bias. This review provides a broad overview of AI performance across applications in whole slide imaging. However, there is huge variability in study design and available performance data, with details around the conduct of the study and make up of the datasets frequently missing. Overall, AI offers good accuracy when applied to WSIs but requires more rigorous evaluation of its performance.Comment: 26 pages, 5 figures, 8 tables + Supplementary material

The NAtional randomised controlled Trial of Tonsillectomy IN Adults (NATTINA): a clinical and cost-effectiveness study: study protocol for a randomised control trial

Background The role of tonsillectomy in the management of adult tonsillitis remains uncertain and UK regional variation in tonsillectomy rates persists. Patients, doctors and health policy makers wish to know the costs and benefits of tonsillectomy against conservative management and whether therapy can be better targeted to maximise benefits and minimise risks of surgery, hence maximising cost-effective use of resources. NATTINA incorporates the first attempt to map current NHS referral criteria against other metrics of tonsil disease severity. Methods/design A UK multi-centre, randomised, controlled trial for adults with recurrent tonsillitis to compare the clinical and cost-effectiveness of tonsillectomy versus conservative management. An initial feasibility study comprises qualitative interviews to investigate the practicality of the protocol, including willingness to randomise and be randomised. Approximately 20 otolaryngology staff, 10 GPs and 15 ENT patients will be recruited over 5 months in all 9 proposed main trial participating sites. A 6-month internal pilot will then recruit 72 patients across 6 of the 9 sites. Participants will be adults with recurrent acute tonsillitis referred by a GP to secondary care. Randomisation between tonsillectomy and conservative management will be according to a blocked allocation method in a 1:1 ratio stratified by centre and baseline disease severity. If the pilot is successful, the main trial will recruit a further 528 patients over 18 months in all 9 participating sites. All participants will be followed up for a total of 24 months, throughout which both primary and secondary outcome data will be collected. The primary outcome is the number of sore throat days experienced over the 24-month follow-up. The pilot and main trials include an embedded qualitative process evaluation. Discussion NATTINA is designed to evaluate the relative effectiveness and efficiency of tonsillectomy versus conservative management in patients with recurrent sore throat who are eligible for surgery. Most adult tonsil disease and surgery has an impact on economically active age groups, with individual and societal costs through loss of earnings and productivity. Avoidance of unnecessary operations and prioritisation of those individuals likely to gain most from tonsillectomy would reduce costs to the NHS and society

The NAtional randomised controlled Trial of Tonsillectomy IN Adults (NATTINA) : a clinical and cost-effectiveness study: study protocol for a randomised control trial

This project is funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (project number 12/146/06).BACKGROUND: The role of tonsillectomy in the management of adult tonsillitis remains uncertain and UK regional variation in tonsillectomy rates persists. Patients, doctors and health policy makers wish to know the costs and benefits of tonsillectomy against conservative management and whether therapy can be better targeted to maximise benefits and minimise risks of surgery, hence maximising cost-effective use of resources. NATTINA incorporates the first attempt to map current NHS referral criteria against other metrics of tonsil disease severity. METHODS/DESIGN: A UK multi-centre, randomised, controlled trial for adults with recurrent tonsillitis to compare the clinical and cost-effectiveness of tonsillectomy versus conservative management. An initial feasibility study comprises qualitative interviews to investigate the practicality of the protocol, including willingness to randomise and be randomised. Approximately 20 otolaryngology staff, 10 GPs and 15 ENT patients will be recruited over 5 months in all 9 proposed main trial participating sites. A 6-month internal pilot will then recruit 72 patients across 6 of the 9 sites. Participants will be adults with recurrent acute tonsillitis referred by a GP to secondary care. Randomisation between tonsillectomy and conservative management will be according to a blocked allocation method in a 1:1 ratio stratified by centre and baseline disease severity. If the pilot is successful, the main trial will recruit a further 528 patients over 18 months in all 9 participating sites. All participants will be followed up for a total of 24 months, throughout which both primary and secondary outcome data will be collected. The primary outcome is the number of sore throat days experienced over the 24-month follow-up. The pilot and main trials include an embedded qualitative process evaluation. DISCUSSION: NATTINA is designed to evaluate the relative effectiveness and efficiency of tonsillectomy versus conservative management in patients with recurrent sore throat who are eligible for surgery. Most adult tonsil disease and surgery has an impact on economically active age groups, with individual and societal costs through loss of earnings and productivity. Avoidance of unnecessary operations and prioritisation of those individuals likely to gain most from tonsillectomy would reduce costs to the NHS and society. TRIAL REGISTRATION: ISRCTN55284102, Date of Registration: 4 August 2014.Publisher PDFPeer reviewe

Use of proton pump inhibitors to treat persistent throat symptoms: Multicentre, double blind, randomised, placebo controlled trial

Objective. To assess the use of proton pump inhibitors (PPIs) to treat persistent throat symptoms. Design. Pragmatic, double blind, placebo controlled, randomised trial. Setting. Eight ear, nose, and throat outpatient clinics, United Kingdom. Participants. 346 patients aged 18 years or older with persistent throat symptoms who were randomised according to recruiting centre and baseline severity of symptoms (mild or severe): 172 to lansoprazole and 174 to placebo. Intervention. Random blinded allocation (1:1) to either 30 mg lansoprazole twice daily or matched placebo twice daily for 16 weeks. Main outcome measures. Primary outcome was symptomatic response at 16 weeks measured using the total reflux symptom index (RSI) score. Secondary outcomes included symptom response at 12 months, quality of life, and throat appearances. Results. Of 1427 patients initially screened for eligibility, 346 were recruited. The mean age of the study sample was 52.2 (SD 13.7) years, 196 (57%) were women, and 162 (47%) had severe symptoms at presentation; these characteristics were balanced across treatment arms. The primary analysis was performed on 220 patients who completed the primary outcome measure within a window of 14-20 weeks. Mean RSI scores were similar between treatment arms at baseline: lansoprazole 22.0 (95% confidence interval 20.4 to 23.6) and placebo 21.7 (20.5 to 23.0). Improvements (reduction in RSI score) were observed in both groups—score at 16 weeks: lansoprazole 17.4 (15.5 to19.4) and placebo 15.6 (13.8 to 17.3). No statistically significant difference was found between the treatment arms: estimated difference 1.9 points (95% confidence interval −0.3 to 4.2 points; P=0.096) adjusted for site and baseline symptom severity. Lansoprazole showed no benefits over placebo for any secondary outcome measure, including RSI scores at 12 months: lansoprazole 16.0 (13.6 to 18.4) and placebo 13.6 (11.7 to 15.5): estimated difference 2.4 points (−0.6 to 5.4 points). Conclusions. No evidence was found of benefit from PPI treatment in patients with persistent throat symptoms. RSI scores were similar between the lansoprazole and placebo groups after 16 weeks of treatment and at the 12 month follow-up

The value of source data verification in a cancer clinical trial

Background Source data verification (SDV) is a resource intensive method of quality assurance frequently used in clinical trials. There is no empirical evidence to suggest that SDV would impact on comparative treatment effect results from a clinical trial. Methods Data discrepancies and comparative treatment effects obtained following 100% SDV were compared to those based on data without SDV. Overall survival (OS) and Progression-free survival (PFS) were compared using Kaplan-Meier curves, log-rank tests and Cox models. Tumour response classifications and comparative treatment Odds Ratios (ORs) for the outcome objective response rate, and number of Serious Adverse Events (SAEs) were compared. OS estimates based on SDV data were compared against estimates obtained from centrally monitored data. Findings Data discrepancies were identified between different monitoring procedures for the majority of variables examined, with some variation in discrepancy rates. There were no systematic patterns to discrepancies and their impact was negligible on OS, the primary outcome of the trial (HR (95% CI): 1.18(0.99 to 1.41), p = 0.064 with 100% SDV; 1.18(0.99 to 1.42), p = 0.068 without SDV; 1.18(0.99 to 1.40), p = 0.073 with central monitoring). Results were similar for PFS. More extreme discrepancies were found for the subjective outcome overall objective response (OR (95% CI): 1.67(1.04 to 2.68), p = 0.03 with 100% SDV; 2.45(1.49 to 4.04), p = 0.0003 without any SDV) which was mostly due to differing CT scans. Interpretation Quality assurance methods used in clinical trials should be informed by empirical evidence. In this empirical comparison, SDV was expensive and identified random errors that made little impact on results and clinical conclusions of the trial. Central monitoring using an external data source was a more efficient approach for the primary outcome of OS. For the subjective outcome objective response, an independent blinded review committee and tracking system to monitor missing scan data could be more efficient than SDV