Determining a risk-proportionate approach to the validation of statistical programming for clinical trials

Background: The contribution of the statistician to the design and analysis of a clinical trial is acknowledged as essential. Ability to reconstruct the statistical contribution to a trial requires rigorous and transparent documentation as evidenced by the reproducibility of results. The process of validating statistical programmes is a key requirement. While guidance relating to software development and life cycle methodologies details steps for validation by information systems developers, there is no guidance applicable to programmes written by statisticians. We aimed to develop a risk-based approach to the validation of statistical programming that would support scientific integrity and efficient resource use within clinical trials units. // Methods: The project was embedded within the Information Systems Operational Group and the Statistics Operational Group of the UK Clinical Research Collaboration Registered Clinical Trials Unit network. Members were asked to share materials relevant to validation of statistical programming. A review of the published literature, regulatory guidance and knowledge of relevant working groups was undertaken. Surveys targeting the Information Systems Operational Group and Statistics Operational Group were developed to determine current practices across the Registered Clinical Trials Unit network. A risk-based approach was drafted and used as a basis for a workshop with representation from statisticians, information systems developers and quality assurance managers (n = 15). The approach was subsequently modified and presented at a second, larger scale workshop (n = 47) to gain a wider perspective, with discussion of content and implications for delivery. The approach was revised based on the discussions and suggestions made. The workshop was attended by a member of the Medicines for Healthcare products Regulatory Agency Inspectorate who also provided comments on the revised draft. // Results: Types of statistical programming were identified and categorised into six areas: generation of randomisation lists; programmes to explore/understand the data; data cleaning, including complex checks; derivations including data transformations; data monitoring; or interim and final analysis. The risk-based approach considers each category of statistical programme against the impact of an error and its likelihood, whether the programming can be fully prespecified, the need for repeated use and the need for reproducibility. Approaches to the validation of programming within each category are proposed. // Conclusion: We have developed a risk-based approach to the validation of statistical programming. It endeavours to facilitate the implementation of targeted quality assurance measures while making efficient use of limited resources

Good Statistical Practice-development of tailored Good Clinical Practice training for statisticians

BACKGROUND: Statisticians are fundamental in ensuring clinical research, including clinical trials, are conducted with quality, transparency, reproducibility and integrity. Good Clinical Practice (GCP) is an international quality standard for the conduct of clinical trials research. Statisticians are required to undertake training on GCP but existing training is generic and, crucially, does not cover statistical activities. This results in statisticians undertaking training mostly unrelated to their role and variation in awareness and implementation of relevant regulatory requirements with regards to statistical conduct. The need for role-relevant training is recognised by the UK NHS Health Research Authority and the Medicines and Healthcare products Regulatory Agency (MHRA).METHODS: The Good Statistical Practice (GCP for Statisticians) project was instigated by the UK Clinical Research Collaboration (UKCRC) Registered Clinical Trials Unit (CTU) Statisticians Operational Group and funded by the National Institute for Health and Care Research (NIHR), to develop materials to enable role-specific GCP training tailored to statisticians. Review of current GCP training was undertaken by survey. Development of training materials were based on MHRA GCP. Critical review and piloting was conducted with UKCRC CTU and NIHR researchers with comment from MHRA. Final review was conducted through the UKCRC CTU Statistics group.RESULTS: The survey confirmed the need and desire for the development of dedicated GCP training for statisticians. An accessible, comprehensive, piloted training package was developed tailored to statisticians working in clinical research, particularly the clinical trials arena. The training materials cover legislation and guidance for best practice across all clinical trial processes with statistical involvement, including exercises and real-life scenarios to bridge the gap between theory and practice. Comprehensive feedback was incorporated. The training materials are freely available for national and international adoption.CONCLUSION: All research staff should have training in GCP yet the training undertaken by most academic statisticians does not cover activities related to their role. The Good Statistical Practice (GCP for Statisticians) project has developed and extensively piloted new, role-specific, comprehensive, accessible GCP training tailored to statisticians working in clinical research, particularly the clinical trials arena. This role-specific training will encourage best practice, leading to transparent and reproducible statistical activity, as required by regulatory authorities and funders.</p

Determining a risk-proportionate approach to the validation of statistical programming for clinical trials

BackgroundThe contribution of the statistician to the design and analysis of a clinical trial is acknowledged as essential. Ability to reconstruct the statistical contribution to a trial requires rigorous and transparent documentation as evidenced by the reproducibility of results. The process of validating statistical programmes is a key requirement. While guidance relating to software development and life cycle methodologies details steps for validation by information systems developers, there is no guidance applicable to programmes written by statisticians. We aimed to develop a risk-based approach to the validation of statistical programming that would support scientific integrity and efficient resource use within clinical trials units.MethodsThe project was embedded within the Information Systems Operational Group and the Statistics Operational Group of the UK Clinical Research Collaboration Registered Clinical Trials Unit network. Members were asked to share materials relevant to validation of statistical programming. A review of the published literature, regulatory guidance and knowledge of relevant working groups was undertaken. Surveys targeting the Information Systems Operational Group and Statistics Operational Group were developed to determine current practices across the Registered Clinical Trials Unit network. A risk-based approach was drafted and used as a basis for a workshop with representation from statisticians, information systems developers and quality assurance managers (n = 15). The approach was subsequently modified and presented at a second, larger scale workshop (n = 47) to gain a wider perspective, with discussion of content and implications for delivery. The approach was revised based on the discussions and suggestions made. The workshop was attended by a member of the Medicines for Healthcare products Regulatory Agency Inspectorate who also provided comments on the revised draft.ResultsTypes of statistical programming were identified and categorised into six areas: generation of randomisation lists; programmes to explore/understand the data; data cleaning, including complex checks; derivations including data transformations; data monitoring; or interim and final analysis. The risk-based approach considers each category of statistical programme against the impact of an error and its likelihood, whether the programming can be fully prespecified, the need for repeated use and the need for reproducibility. Approaches to the validation of programming within each category are proposed.ConclusionWe have developed a risk-based approach to the validation of statistical programming. It endeavours to facilitate the implementation of targeted quality assurance measures while making efficient use of limited resources

Trial Steering Committees in randomised controlled trials: A survey of registered clinical trials units to establish current practice and experiences

Background: The Medical Research Council Guidelines for Good Clinical Practice outlines a three-committee trial oversight structure – the day-to-day Trial Management Group, the Data Monitoring Committee and the Trial Steering Committee. In this model, the Trial Steering Committee is the executive committee that oversees the trial and considers the recommendations from the Data Monitoring Committee. There is yet to be in-depth consideration establishing the Trial Steering Committee’s role and functionality. Methods: A survey to establish Trial Steering Committee’s current practices, role and the use and opinion on the Medical Research Council guidelines was undertaken within UK Clinical Research Collaborative registered Clinical Trials Units. Results: Completed surveys were obtained from 38 of 47 fully and partially registered Units. Individual items in the survey were analysed and reported spanning current Trial Steering Committee practices including its role, requirement and experience required for membership; methods to identify members; and meeting frequency. Terms (a document describing the committee’s remit, objectives and functionality) were obtained and analysed from 21 of 33 Units with documents in place at their Unit. A total of 20 responders suggested aspects of the current Medical Research Council Guidelines that need improvement. Conclusion: We present the first survey reporting on practices within UK Clinical Research Collaborative registered Clinical Trials Units on the experience and remits of Trial Steering Committees. We have identified a widespread adoption of Medical Research Council Guidelines for Trial Steering Committees in the United Kingdom, but limitations in this existing provision have been identified that need to be addressed. </jats:sec

Dimensionless cosmology

Although it is well known that any consideration of the variations of fundamental constants should be restricted to their dimensionless combinations, the literature on variations of the gravitational constant $G$ is entirely dimensionful. To illustrate applications of this to cosmology, we explicitly give a dimensionless version of the parameters of the standard cosmological model, and describe the physics of Big Bang Neucleosynthesis and recombination in a dimensionless manner. The issue that appears to have been missed in many studies is that in cosmology the strength of gravity is bound up in the cosmological equations, and the epoch at which we live is a crucial part of the model. We argue that it is useful to consider the hypothetical situation of communicating with another civilization (with entirely different units), comparing only dimensionless constants, in order to decide if we live in a Universe governed by precisely the same physical laws. In this thought experiment, we would also have to compare epochs, which can be defined by giving the value of any {\it one} of the evolving cosmological parameters. By setting things up carefully in this way one can avoid inconsistent results when considering variable constants, caused by effectively fixing more than one parameter today. We show examples of this effect by considering microwave background anisotropies, being careful to maintain dimensionlessness throughout. We present Fisher matrix calculations to estimate how well the fine structure constants for electromagnetism and gravity can be determined with future microwave background experiments. We highlight how one can be misled by simply adding $G$ to the usual cosmological parameter set

A cohort examination to establish reporting of the remit and function of Trial Steering Committees in randomised controlled trials

BACKGROUND: The DAMOCLES project established a widely used Data Monitoring Committee (DMC) Charter for randomised controlled trials (RCTs). Typically, within the UK, the DMC is advisory and recommends to another executive body; the Trial Steering Committee (TSC). Despite the executive role of the TSC, the CONSORT Statement does not explicitly require reporting of TSC activity, although is included as an example of good reporting. A lack of guidance on TSC reporting can impact transparency of trial oversight, ultimately leading to a misunderstanding regarding role and, subsequently, further variation in practice. This review aimed to establish reporting practice of TSC involvement in RCTs, and thus make recommendations for reporting. METHODS: A cohort examination identifying reporting practice was undertaken. The cohort comprised RCTs published in three leading medical journals (the British Medical Journal, The Lancet and the New England Journal of Medicine) within 6 months in 2012 and the full NIHR HTA Monograph series. Details of TSC constitution and impact were extracted from main publications and published supplements. RESULTS: Of 415 publications, 264 were eligible. These were typical in terms of trial design. Variations in reporting between journals and monographs was notable. TSC presence was identified in approximately half of trials (n = 144), of which 109 worked alongside a DMC. No publications justified not convening a TSC. When reported, the role of the committee and examples of impact in design, conduct and analysis were summarised. CONCLUSIONS: We present the first review of reporting TSC activity in the published academic literature. An absence of reporting standards with regards to TSC constitution, activity and impact on trial conduct was identified which can influence transparency of reporting trial oversight. Consistent reporting is vital for the benefits and impact of the TSC role to be understood to support adoption of this oversight structure and reduce global variations in practice

Ten Years of Surveillance for Invasive Streptococcus pneumoniae during the Era of Antiretroviral Scale-Up and Cotrimoxazole Prophylaxis in Malawi

OBJECTIVE: To document trends in invasive pneumococcal disease (IPD) in a central hospital in Malawi during the period of national scale-up of antiretroviral therapy (ART) and cotrimoxazole prophylaxis. METHODS: Between 1 January 2000 and 31 December 2009 almost 100,000 blood cultures and 40,000 cerebrospinal fluid (CSF) cultures were obtained from adults and children admitted to the Queen Elizabeth Central Hospital, Blantyre, Malawi with suspected severe bacterial infection. RESULTS: 4,445 pneumococcal isolates were obtained over the 10 year period. 1,837 were from children: 885 (19.9%) from blood and 952 (21.4%) from CSF. 2,608 were from adults: 1,813 (40.8%) from blood and 795 (17.9%) from CSF. At the start of the surveillance period cotrimoxazole resistance was 73.8% and at the end was 92.6%. Multidrug resistance (MDR) was present in almost one third of isolates and was constant over time. Free ART was introduced in Malawi in 2004. From 2005 onwards there was a decline in invasive pneumococcal infections with a negative correlation between ART scale-up and the decline in IPD (Pearson's correlation r = -0.91; p<0.001). CONCLUSION: During 2004-2009, national ART scale-up in Malawi was associated with a downward trend in IPD at QECH. The introduction of cotrimoxazole prophylaxis in HIV-infected groups has not coincided with a further increase in pneumococcal cotrimoxazole or multidrug resistance. These data highlight the importance of surveillance for high disease burden infections such as IPD in the region, which will be vital for monitoring pneumococcal conjugate vaccine introduction into national immunisation programmes

Plasma lipid profiles discriminate bacterial from viral infection in febrile children.

Funder: NIHR Imperial BRCFever is the most common reason that children present to Emergency Departments. Clinical signs and symptoms suggestive of bacterial infection are often non-specific, and there is no definitive test for the accurate diagnosis of infection. The 'omics' approaches to identifying biomarkers from the host-response to bacterial infection are promising. In this study, lipidomic analysis was carried out with plasma samples obtained from febrile children with confirmed bacterial infection (n = 20) and confirmed viral infection (n = 20). We show for the first time that bacterial and viral infection produces distinct profile in the host lipidome. Some species of glycerophosphoinositol, sphingomyelin, lysophosphatidylcholine and cholesterol sulfate were higher in the confirmed virus infected group, while some species of fatty acids, glycerophosphocholine, glycerophosphoserine, lactosylceramide and bilirubin were lower in the confirmed virus infected group when compared with confirmed bacterial infected group. A combination of three lipids achieved an area under the receiver operating characteristic (ROC) curve of 0.911 (95% CI 0.81 to 0.98). This pilot study demonstrates the potential of metabolic biomarkers to assist clinicians in distinguishing bacterial from viral infection in febrile children, to facilitate effective clinical management and to the limit inappropriate use of antibiotics

Exploring the role and function of trial steering committees:results of an expert panel meeting

BACKGROUND: The independent oversight of clinical trials, which is recommended by the Medical Research Council (MRC) Guidelines for Good Clinical Practice, is typically provided by an independent advisory Data Monitoring Committee (DMC) and an independent executive committee, to whom the DMC makes recommendations. The detailed roles and function of this executive committee, known as the Trial Steering Committee (TSC), have not previously been studied or reviewed since those originally proposed by the MRC in 1998. METHODS: An expert panel (n = 7) was convened comprising statisticians, clinicians and trial methodologists with prior TSC experience. Twelve questions about the role and responsibilities of the TSC were discussed by the panel at two full-day meetings. Each meeting was transcribed in full and the discussions were summarised. RESULTS: The expert panel reached agreement on the role of the TSC, to which it was accountable, the membership, the definition of independence, and the experience and training needed. The management of ethical issues, difficult/complex situations and issues the TSC should not ask the DMC to make recommendations on were more difficult to discuss without specific examples, but support existed for further work to help share issues and to provide appropriate training for TSC members. Additional topics discussed, which had not been identified by previous work relating to the DMCs but were pertinent to the role of the TSC, included the following: review of data sharing requests, indemnity, lifespan of the TSC, general TSC administration, and the roles of both the Funder and the Sponsor. CONCLUSIONS: This paper presents recommendations that will contribute to the revision and update of the MRC TSC terms of reference. Uncertainty remains in some areas due to the absence of real-life examples; future guidance on these issues would benefit from a repository of case studies. Notably, the role of a patient and public involvement (PPI) contributor was not discussed, and further work is warranted to explore the role of a PPI contributor in independent trial oversight