Systematic review about data quality and protocol compliance in clinical trials

Bei Investigator Initiated Trials (IITs) werden alternative risikoadaptierte Monitoring-Strategien in Abhängigkeit vom individuellen Studiendesign und dem Risikoprofil diskutiert, um bei oft restriktiven Ressourcen eine den gesetzlichen Vorgaben genügende Qualität der Studiendurchführung und der Daten zu gewährleisten. Aufgrund einer Literaturanalyse sollten in der vorliegenden Arbeit Untersuchungen ausgewertet werden, in denen quantitative Aussagen zu Datenqualität und Prüfplan-Compliance in klinischen Prüfungen gemacht wurden. Bei der Interpretation der Ergebnisse sollten die implementierten Qualitätssicherungsmaßnahmen berücksichtigt werden. Aufgrund einer systematischen Recherche in MEDLINE konnten 21 Publikationen identifiziert werden, bei denen die Daten- und Prozessqualität in klinischen Prüfungen untersucht, die Qualität durch Überprüfungen mit Source Data Verification vor Ort oder Überprüfung übermittelter Quelldaten in der Studienzentrale ermittelt wurde und quantitative Informationen zu den Bereichen Datenqualität, Protokoll-Compliance oder Defizite bei Einwilligungserklärungen vorlagen. Die Mehrzahl der Untersuchungen ist drei Organisationen zuzuordnen: European Organization für Research and Treatment of Cancer (EORTC) (n=7), National Cancer Institute (NCI) (n=7) und Untersuchungen der Trans-Tasman Radiation Oncology Group (TROG) (n=4). Darüber hinaus wurden drei Untersuchungen weiterer Studiengruppen identifiziert. Die Untersuchungen wurden im Zeitraum von 1981 bis 2003 publiziert. Überwiegend wurden in der Literatur onkologische Studien betrachtet (n=19), wobei die Radiotherapie im Vordergrund stand (n=8). Für die EORTC-Studien wurde in der Regel eine gute Datenqualität berichtet (80-90% korrekte Daten). Punktuelle Probleme wurden im Hinblick auf die Protokoll-Compliance und das Berichten von Nebenwirkungen/schwerwiegenden unerwünschten Ereignissen festgestellt. Eine gute Qualität wurde ebenfalls bzgl. des korrekten Einschlusses von Patienten beobachtet. Durch das NCI wurde ein standardisiertes Audit-System eingeführt und innerhalb von kooperativen Studiengruppen implementiert. Im Rahmen dieser Audits wurden verschiedene Kriterien überprüft und eine überwiegend gute Datenqualität und Protokoll-Compliance festgestellt. Mängel wurden in ca. 5% der Fälle im Hinblick auf die Einwilligungserklärung, die korrekte Anwendung der Einschlusskriterien, Protokollverletzungen, bei der Ermittlung des Zielkriteriums, der Erfassung der Toxizität, der adäquaten Datenlieferung und bei der Datenverifikation beobachtet. In einzelnen Untersuchungen ergaben sich Probleme mit der Behandlungscompliance (10-20%), bei Protokollabweichungen im Hinblick auf die Arzneimitteldosis (10%) und bei der Drug Accountability (15%). Von der TROG wurde ein Qualitätssicherungsprozess implementiert, der auf zentralem Monitoring von kopierten Quelldaten basiert. Durch den Qualitätssicherungsansatz der TROG konnten schwerwiegende Probleme mit der Protokoll-Compliance unter 10% gesenkt werden, ebenso konnte eine gute Datenqualität mit einer Fehlerrate unter 5% erreicht werden. Die korrekte Handhabung von Ein- und Ausschlusskriterien stellte in Einzelfällen ein Problem dar. Zusammenfassend kann festgestellt werden, dass die in dem Review erfassten Studiengruppen von einer guten Datenqualität und einer guten bis moderaten Protokoll-Compliance berichten. Diese basiert nach Aussage der Autoren im wesentlichen auf etablierten Qualitätssicherungs-Prozeduren, wobei das durchgeführte Audit ebenfalls einen potentiellen Einflussfaktor darstellt. Geringe Probleme wurden in der Regel im Hinblick auf die Einwilligungserklärung, die korrekte Handhabung der Ein- und Ausschlusskriterien und die Datenqualität beobachtet. In einzelnen Studien gab es jedoch Probleme mit der Protokoll-Compliance. Insgesamt hängen Anzahl und Art der Mängel von dem Studientyp, dem Qualitätsmanagement und der Organisation der Studiengruppe ab. Wissenschaftsbetrug wurde nur in sehr wenigen Fällen durch die Audits festgestellt. Die vorgelegten Informationen beziehen sich nahezu ausschließlich auf etablierte Studiengruppen; bezüglich Datenqualität und Protokoll-Compliance außerhalb der Studiengruppen liegen kaum Informationen in der Literatur vor. Bei der Bewertung der Ergebnisse sollte berücksichtigt werden, dass es sich zum Teil um Eigenauswertungen der Studiengruppen und nicht um unabhängige externe Prüfungen (z.B. externe Audits) handelt. Inwieweit die Ergebnisse einer konsequenten Überprüfung nach derzeitigen Good Clinical Practice (GCP) – Regeln standhalten würden, kann aus der Analyse nicht beantwortet werden. Aus der vorliegenden Literaturanalyse ergeben sich Konsequenzen für die Planung einer prospektiven kontrollierten Studie zum Vergleich unterschiedlicher Monitoring-Strategien. Wesentlicher Einflussfaktor für die Datenqualität und Protokollcompliance in einer klinischen Studie ist das Qualitätsmanagement. Dieses Qualitätsmanagement umfasst neben Monitoring zahlreiche andere Maßnahmen. Um zu einer Bewertung von Monitoringstrategien kommen zu können, müssen daher alle Qualitätssicherungsmaßnahmen im Rahmen einer Studie berücksichtigt werden. Für den Vergleich unterschiedlicher Monitoringstrategien sind geeignete Zielparameter zu definieren (z.B. schwerwiegende Defizite bzgl. Ein- und Ausschlusskriterien, Sicherheit). Die vorliegende Analyse ergibt, dass bei gutem Qualitätsmanagement ohne umfassendes vor Ort Monitoring schwerwiegende Fehler nur mit relativ niedriger Häufigkeit festgestellt wurden. Unterschiede zwischen Monitoringstrategien könnten, gegeben ein funktionierendes Qualitätsmanagementssystem, sich als quantitativ gering erweisen. Testet man auf Äquivalenz von Monitoringstrategien, sind nur niedrige Differenzen zu akzeptieren, was wiederum eine Auswirkung auf die Fallzahlplanung hat. Weiterhin muss berücksichtigt werden, dass zur Feststellung der Auswirkung unterschiedlicher Monitoringstrategien auf die Sicherheit der Patienten und die Validität der Daten im Rahmen einer kontrollierten Untersuchung ein unabhängiges Audit notwendig ist. Dabei ist zu berücksichtigen, dass ein Audit bereits einen möglichen Einflussfaktor für die Datenqualität und Protokoll-Compliance darstellen kann, und damit eine Bewertung des Nutzens einer Monitoringstrategie erschwert werden könnte. Schlüsselwörter: systematisches Review, Datenqualität, Protokoll-Compliance, klinische StudieFor Investigator Initiated Trials (IITs) alternative risk-adapted monitoring strategies are discussed in order to fulfill rules and regulations, taking into consideration the restricted resources. In this systematic review investigations, presenting quantitative data about data quality and protocol compliance in clinical trials, are analyzed. The results are discussed taking into account the quality assurance procedures implemented. Based on a systematic MEDLINE retrieval, 21 studies could be identified in which data and process quality in clinical trials were investigated and assessed by site visits with source data verification or review of copied source data in the study center and quantitative information about data quality and protocol compliance was available. The majority of investigations were performed by three organizations: European Organization for Research and Treatment of Cancer (EORTC) (n=7), National Cancer Institute (NCI) (n=7) and investigations of the Trans-Tasman Radiation Oncology Group (TROG) (n=4). In addition three investigations of other study groups were identified. The investigations were published between 1981 and 2003. In the majority of cases oncological trials were investigated (n=19) with a focus on radiotherapy trials (n=8). In the EORTC-trials an overall good data quality was assessed (80–90% correct data). Singular problems were found with respect to protocol compliance and reporting of adverse reactions and serious unexpected events. Good quality was also observed with respect to the correct inclusion of patients into trials. By the NCI a standardized audit system was introduced and implemented within cooperative study groups. In the context of these audits different criteria were assessed and a good data quality and protocol compliance were measured. Deficits occurred in about 5% of the cases with respect to informed consent, correct application of inclusion criteria, protocol compliance, assessment of outcome criteria, assessment of toxicity, adequate data reporting and data verification. In some investigations problems with treatment compliance (10-20%), drug dose deviations (10%) and drug accountability (15%) were identified. By the TROG a quality assurance procedure was implemented, based on central monitoring of copied source data. By this approach major problems with protocol compliance could be reduced to less than 10% together with a good data quality with an error rate under 5%. The correct handling of in- and exclusion criteria was a problem in individual cases. In summary we found out that good data quality and good to moderate protocol compliance were reported by the study groups that are included in the systematic review. Due to the authors this is mainly due to an established quality assurance system, taking into consideration that audits itself may be an influential factor. Generally, minor problems were observed with respect to informed consent, correct handling of in- and exclusion criteria and data quality, however, in some studies there were problems with protocol compliance. Overall, number and type of deficits depend on study type, quality management and organization of the study group. Fraud was detected only in very few cases. The available evidence refers mainly to established study groups; for data quality and protocol compliance outside these groups only few information is available. However, it should be taken into consideration, that the analysis was performed, at least partly, by the study groups themselves and is not based on independent audits (e.g. external audits). The analysis cannot answer the question whether the results would have been replicable if a strict review according to criteria of Good Clinical Practice (GCP) would have been performed. From the systematic review consequences have to be taken for planning a prospective controlled trial comparing different monitoring strategies. The main influence factor for data quality and protocol compliance in a clinical trial is the quality management system. Quality management covers several other measures apart from monitoring. In order to assess monitoring strategies, all quality assurance procedures within a clinical trial have to be taken into consideration. For the comparison of different monitoring strategies adequate outcome parameter have to be defined (e.g. severe deficits with respect to inclusion and exclusion criteria, safety). The analysis indicates that with good quality management and no extensive on-site monitoring severe errors were detected only at relative low frequency. It could well be that with an efficient quality management system differences between monitoring strategies would be small. In order to demonstrate statistical equivalence of monitoring strategies, only small differences can be accepted which again leads to consequences for the sample-size calculation. In addition, it must be taken into consideration that within a controlled trial an independent audit is necessary to assess the effect of different monitoring strategies on the safety of patients and the quality of data. Audits however may be a possible influence factor for data quality and protocol compliance and may complicate the evaluation of the benefit of a monitoring strategy. Keywords: systematic review, data quality, protocol compliance, clinical tria

Practices, patients and (im)perfect data - feasibility of a randomised controlled clinical drug trial in German general practices

<p>Abstract</p> <p>Background</p> <p>Randomised controlled clinical (drug) trials supply high quality evidence for therapeutic strategies in primary care. Until now, experience with drug trials in German general practice has been sparse. In 2007/2008, the authors conducted an investigator-initiated, non-commercial, double-blind, randomised controlled pilot trial (HWI-01) to assess the clinical equivalence of ibuprofen and ciprofloxacin in the treatment of uncomplicated urinary tract infection (UTI). Here, we report the feasibility of this trial in German general practices and the implementation of Good Clinical Practice (GCP) standards as defined by the International Conference on Harmonisation (ICH) in mainly inexperienced general practices.</p> <p>Methods</p> <p>This report is based on the experience of the HWI-01 study conducted in 29 German general practices. Feasibility was defined by 1) successful practice recruitment, 2) sufficient patient recruitment, 3) complete and accurate data collection and 4) appropriate protection of patient safety.</p> <p>Results</p> <p>The final practice recruitment rate was 18%. In these practices, 79 of 195 screened UTI patients were enrolled. Recruitment differed strongly between practices (range 0-12, mean 2.8 patients per practice) and was below the recruitment goal of approximately 100 patients. As anticipated, practice nurses became the key figures in the screening und recruitment of patients. Clinical trial demands, in particular for completing symptom questionnaires, documentation of source data and reporting of adverse events, did not agree well with GPs' documentation habits and required support from study nurses. In many cases, GPs and practice staff seemed to be overwhelmed by the amount of information and regulations. No sudden unexpected serious adverse reactions (SUSARs) were observed during the trial.</p> <p>Conclusions</p> <p>To enable drug trials in general practice, it is necessary to adapt the setup of clinical research infrastructure to the needs of GPs and their practice staff. Risk adaption of clinical trial regulations is necessary to facilitate non-commercial comparative effectiveness trials in primary health care.</p> <p>Trial Registration</p> <p>Trial registration number: <a href="http://www.controlled-trials.com/ISRCTN00470468">ISRCTN00470468</a></p

Typical investigational medicinal products follow relatively uniform regulations in 10 European Clinical Research Infrastructures Network (ECRIN) countries

<p>Abstract</p> <p>Background</p> <p>In order to facilitate multinational clinical research, regulatory requirements need to become international and harmonised. The EU introduced the Directive 2001/20/EC in 2004, regulating investigational medicinal products in Europe.</p> <p>Methods</p> <p>We conducted a survey in order to identify the national regulatory requirements for major categories of clinical research in ten European Clinical Research Infrastructures Network (ECRIN) countries-Austria, Denmark, France, Germany, Hungary, Ireland, Italy, Spain, Sweden, and United Kingdom-covering approximately 70% of the EU population. Here we describe the results for regulatory requirements for typical investigational medicinal products, in the ten countries.</p> <p>Results</p> <p>Our results show that the ten countries have fairly harmonised definitions of typical investigational medicinal products. Clinical trials assessing typical investigational medicinal products require authorisation from a national competent authority in each of the countries surveyed. The opinion of the competent authorities is communicated to the trial sponsor within the same timelines, i.e., no more than 60 days, in all ten countries. The authority to which the application has to be sent to in the different countries is not fully harmonised.</p> <p>Conclusion</p> <p>The Directive 2001/20/EC defined the term 'investigational medicinal product' and all regulatory requirements described therein are applicable to investigational medicinal products. Our survey showed, however, that those requirements had been adopted in ten European countries, not for investigational medicinal products overall, but rather a narrower category which we term 'typical' investigational medicinal products. The result is partial EU harmonisation of requirements and a relatively navigable landscape for the sponsor regarding typical investigational medicinal products.</p

A cluster randomised trial, cost-effectiveness analysis and psychosocial evaluation of insulin pump therapy compared with multiple injections during flexible intensive insulin therapy for type 1 diabetes: the REPOSE Trial.

BACKGROUND: Insulin is generally administered to people with type 1 diabetes mellitus (T1DM) using multiple daily injections (MDIs), but can also be delivered using infusion pumps. In the UK, pumps are recommended for patients with the greatest need and adult use is less than in comparable countries. Previous trials have been small, of short duration and have failed to control for training in insulin adjustment. OBJECTIVE: To assess the clinical effectiveness and cost-effectiveness of pump therapy compared with MDI for adults with T1DM, with both groups receiving equivalent structured training in flexible insulin therapy. DESIGN: Pragmatic, multicentre, open-label, parallel-group cluster randomised controlled trial, including economic and psychosocial evaluations. After participants were assigned a group training course, courses were randomly allocated in pairs to either pump or MDI. SETTING: Eight secondary care diabetes centres in the UK. PARTICIPANTS: Adults with T1DM for > 12 months, willing to undertake intensive insulin therapy, with no preference for pump or MDI, or a clinical indication for pumps. INTERVENTIONS: Pump or MDI structured training in flexible insulin therapy, followed up for 2 years. MDI participants used insulin analogues. Pump participants used a Medtronic Paradigm(®) Veo(TM) (Medtronic, Watford, UK) with insulin aspart (NovoRapid, Novo Nordisk, Gatwick, UK). MAIN OUTCOME MEASURES: Primary outcome - change in glycated haemoglobin (HbA1c) at 2 years in participants whose baseline HbA1c was ≥ 7.5% (58 mmol/mol). Key secondary outcome - proportion of participants with HbA1c ≤ 7.5% at 2 years. Other outcomes at 6, 12 and 24 months - moderate and severe hypoglycaemia; insulin dose; body weight; proteinuria; diabetic ketoacidosis; quality of life (QoL); fear of hypoglycaemia; treatment satisfaction; emotional well-being; qualitative interviews with participants and staff (2 weeks), and participants (6 months); and ICERs in trial and modelled estimates of cost-effectiveness. RESULTS: We randomised 46 courses comprising 317 participants: 267 attended a Dose Adjustment For Normal Eating course (132 pump; 135 MDI); 260 were included in the intention-to-treat analysis, of which 235 (119 pump; 116 MDI) had baseline HbA1c of ≥ 7.5%. HbA1c and severe hypoglycaemia improved in both groups. The drop in HbA1c% at 2 years was 0.85 on pump and 0.42 on MDI. The mean difference (MD) in HbA1c change at 2 years, at which the baseline HbA1c was ≥ 7.5%, was -0.24% [95% confidence interval (CI) -0.53% to 0.05%] in favour of the pump (p = 0.098). The per-protocol analysis showed a MD in change of -0.36% (95% CI -0.64% to -0.07%) favouring pumps (p = 0.015). Pumps were not cost-effective in the base case and all of the sensitivity analyses. The pump group had greater improvement in diabetes-specific QoL diet restrictions, daily hassle plus treatment satisfaction, statistically significant at 12 and 24 months and supported by qualitative interviews. LIMITATION: Blinding of pump therapy was not possible, although an objective primary outcome was used. CONCLUSION: Adding pump therapy to structured training in flexible insulin therapy did not significantly enhance glycaemic control or psychosocial outcomes in adults with T1DM. RESEARCH PRIORITY: To understand why few patients achieve a HbA1c of < 7.5%, particularly as glycaemic control is worse in the UK than in other European countries. TRIAL REGISTRATION: Current Controlled Trials ISRCTN61215213. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 21, No. 20. See the NIHR Journals Library website for further project information