Research techniques made simple: workflow for searching databases to reduce evidence selection bias in systematic reviews

Clinical trials and basic science studies without statistically significant results are less likely to be published than studies with statistically significant results. Systematic reviews and meta-analyses that omit unpublished data are at high risk of distorted conclusions. Here, we describe methods to search beyond bibliographical databases to reduce evidence selection bias in systematic reviews. Unpublished studies may be identified by searching conference proceedings. Moreover, clinical trial registries—databases of planned and ongoing trials—and regulatory agency websites such as the European Medicine Agency (EMA) and the United States Food and Drug Administration (FDA) may provide summaries of efficacy and safety data. Primary and secondary outcomes are prespecified in trial registries, thus allowing the assessment of outcome reporting bias by comparison with the trial report. The sources of trial data and documents are still evolving, with ongoing initiatives promoting broader access to clinical study reports and individual patient data. There is currently no established methodology to ensure that the multiple sources of information are incorporated. Nonetheless, systematic reviews must adapt to these improvements and cover the new sources in their search strategies

Contribution of systematic reviews and network metanalaysis in Dermatology

La décision médicale nécessite d’avoir à disposition une synthèse rigoureuse et actualisée des données de la recherche. Les revues systématiques (RS) et méta-analyses (MA) sont la méthode la plus rigoureuse pour produire ces synthèses. Nous avons réalisé des RS et MA Cochrane sur trois maladies chroniques en dermatologie. Notre objectif était de produire une synthèse des données, de proposer des pistes d’amélioration pour la recherche à venir et d’ouvrir sur des projets de méta-recherche.La première revue avait pour objectif d’évaluer l’efficacité et la tolérance des traitements de la pustulosepalmoplantaire. Nous avons inclus 36 essais contrôlés randomisés (ECR) comprenant un total de 1504 patients. Le comparateur était le placebo ou aucun traitement dans 31 ECR. Au total, nous avons trouvé un faible degré de certitude que l’analogue topique de la vitamine D (maxacalcitrol) était plus efficace que le placebo à court terme ; que l'alitrétinoïne et trois traitements biologiques n'étaient pas plus efficaces que le placebo ; enfin un degré de certitude modéré que le secukinumab (anti IL17) était supérieur au placebo. Dans cette revue, aucune donnée probante n’a été retrouvée pour les principaux traitements utilisés en pratique courante (corticoïdes locaux, photothérapie, acitrétine, méthotrexate, ciclosporine).La méta-analyse en réseau est une méthode statistique permettant de déterminer l’efficacité relative de l’ensemble des traitements y compris pour les comparaisons qui n’ont pas été évaluées dans un ECR. Cette méthode a été appliquée dans les deux revues suivantes.La deuxième revue avait pour objectif d’évaluer l’efficacité et la tolérance relative d’un traitement suppressif par aciclovir, valaciclovir ou famciclovir dans l’herpès génital du sujet immunocompétent. Nous avons inclus 26 ECR portant sur un total de 6950 patients. Les résultats montraient que l'aciclovir, le famciclovir et le valaciclovir diminuaient le risque d’avoir au moins une récidive d'herpès génital sous traitement comparativement au placebo. La méta-analyse en réseau n'a révélé aucune différence statistiquement significative entre les traitements. Dans cette revue, le manque de comparaisons directes, l’absence des critères de jugement pertinents pour les patients, le risque de biais élevé et non clair pour une majorité d’essais et une hétérogénéité importante limitent la pertinence et le niveau de confiance dans les résultats.La troisième revue avait pour objectif d’évaluer l’efficacité et la tolérance relative des traitements systémiques du psoriasis modéré à sévère chez l’adulte. Nous avons identifié 109 essais pertinents pour un total de 39,882 patients. Soixante-quatorze essais ont été inclus dans la méta-analyse en réseau. Cette étude a montré que l’ixekizumab était le meilleur traitementversus placebo (RR 32.45, 95% IC 23.61 à 44.60; (haut degré de certitude)), suivi par le secukinumab (haut degré de certitude), puis le brodalumab (degré de certitude modéré), puis guselkumab (degré de certitude modéré), le certolizumab (degré de certitude modéré), et l’ustekinumab (haut degré de certitude). Dans cette revue, malgré un nombre conséquent d’essais, un certain nombre de points limites diminuaient le degré de confiance dans les résultats : peu d’essais face-face, peu d’essais évaluant et rapportant la qualité de vie, l’absence d’évaluation au long court et l’inclusion d’une population sélectionnée.Ces trois revues ont permis de mettre en évidence des biais récurrents pouvant s’appliquer au traitement de maladies chroniques en dermatologie notamment le manque d’essai face-face, l’évaluation le plus souvent à court terme et le peu d’essais rapportant un critère de qualité de vie. Ce travail permet également de guider les essais à venir dans ces trois domaines afin d’améliorer l’agenda de la recherche, la diffusion des résultats et la rédaction de recommandations.Medical decision-making requires a rigorous and up-to-date synthesis of research data. Systematic reviews (SR) and meta-analyses (MA) are the most rigorous method for producing these syntheses. Cochrane is an independent international organization whose mission is to produce and disseminate systematic reviews. We have performed RS and MA Cochrane on three chronic diseases in dermatology. Our objective was threefold: to produce a rigorous synthesis of the data, to propose ways of improving future research in these fields and finally to open up to meta-research projects. The first review aimed to evaluate the efficacy and tolerance of treatments for palmoplantarpustulosis, a chronic inflammatory disease characterized by the presence of pustules on palms and plants. We included 36 randomized controlled trials (RCTs) that encompassed a total of 1,504 patients. Overall, we found that the topical analogue of vitamin D (maxacalcitrol) was more effective than placebo in the short term; alitretinoin and three biological treatments (etanercept (anti TNF), ustekinumab (anti IL17-IL23) and guselkumab (anti IL23)) were no more effective than placebo (low degree of certainty); and secukinumab (anti IL17) was superior to placebo (moderate degree of certainty). In this first review, we observed a situation where no evidence was found for the main treatments (local corticosteroids, phototherapy, acitretin, methotrexate, and cyclosporine) used. Network meta-analysis is a statistical method for determining the relative effectiveness of all treatments, including comparisons that have not been evaluated in an RCT. This method was applied in the following two reviews.The objective of the second review was to evaluate the efficacy and relative safety of suppressive treatment with acyclovir, valaciclovir or famciclovir in genital herpes in the immunocompetent subject. We included 26 RCTs with a total of 6,950 patients. The results showed that acyclovir, famciclovir and valaciclovir reduced the risk of having at least one recurrence of genital herpes under treatment compared to placebo. The network meta-analysis revealed no statistically significant differences between treatments. No conclusions on adverse reactions could be drawn due to the poor quality of their description. The lack of direct comparison, the absence of the most relevant judgment criteria for patients, the risk of high and unclear bias in a majority of trials and a heterogeneity whose causes could not be explored alter the relevance and level of confidence in the results.The objective of the third review was to evaluate the efficacy and relative tolerance of systemic treatments for moderate to severe psoriasis in adults. We included 109 trials for a total of 39,882 patients. Seventy-four of these trials were included in the network meta-analysis. This study showed that ixekizumab was the best treatment. (versus placebo: RR 32.45, 95% CI 23.61 to 44.60; (high certainty), followed by secukinumab (high certainty), then brodalumab (moderate certainty), then guselkumab (moderate certainty), certolizumab (moderate certainty), and ustekinumab (high certainty). In this third review, the number of trials allowed us to conduct a network meta-analysis and classify the treatments. However, a number of points decreased the degree of confidence in the results: the few face-to-face trials, few trials evaluating and reporting on quality of life, the absence of long-term short evaluation and the inclusion of a selected population.These three reviews highlighted recurrent biases that could be applied to other chronic dermatological diseases, including the lack of face-to-face testing, primarily short term evaluations, and the lack of reports including a quality of life criterion. This work also provides guidance for future trials in these three areas in terms of comparisons to be made and bias to be avoided

Apport des revues systématiques et méta-analyses en réseau en Dermatologie

Medical decision-making requires a rigorous and up-to-date synthesis of research data. Systematic reviews (SR) and meta-analyses (MA) are the most rigorous method for producing these syntheses. Cochrane is an independent international organization whose mission is to produce and disseminate systematic reviews. We have performed RS and MA Cochrane on three chronic diseases in dermatology. Our objective was threefold: to produce a rigorous synthesis of the data, to propose ways of improving future research in these fields and finally to open up to meta-research projects. The first review aimed to evaluate the efficacy and tolerance of treatments for palmoplantarpustulosis, a chronic inflammatory disease characterized by the presence of pustules on palms and plants. We included 36 randomized controlled trials (RCTs) that encompassed a total of 1,504 patients. Overall, we found that the topical analogue of vitamin D (maxacalcitrol) was more effective than placebo in the short term; alitretinoin and three biological treatments (etanercept (anti TNF), ustekinumab (anti IL17-IL23) and guselkumab (anti IL23)) were no more effective than placebo (low degree of certainty); and secukinumab (anti IL17) was superior to placebo (moderate degree of certainty). In this first review, we observed a situation where no evidence was found for the main treatments (local corticosteroids, phototherapy, acitretin, methotrexate, and cyclosporine) used. Network meta-analysis is a statistical method for determining the relative effectiveness of all treatments, including comparisons that have not been evaluated in an RCT. This method was applied in the following two reviews.The objective of the second review was to evaluate the efficacy and relative safety of suppressive treatment with acyclovir, valaciclovir or famciclovir in genital herpes in the immunocompetent subject. We included 26 RCTs with a total of 6,950 patients. The results showed that acyclovir, famciclovir and valaciclovir reduced the risk of having at least one recurrence of genital herpes under treatment compared to placebo. The network meta-analysis revealed no statistically significant differences between treatments. No conclusions on adverse reactions could be drawn due to the poor quality of their description. The lack of direct comparison, the absence of the most relevant judgment criteria for patients, the risk of high and unclear bias in a majority of trials and a heterogeneity whose causes could not be explored alter the relevance and level of confidence in the results.The objective of the third review was to evaluate the efficacy and relative tolerance of systemic treatments for moderate to severe psoriasis in adults. We included 109 trials for a total of 39,882 patients. Seventy-four of these trials were included in the network meta-analysis. This study showed that ixekizumab was the best treatment. (versus placebo: RR 32.45, 95% CI 23.61 to 44.60; (high certainty), followed by secukinumab (high certainty), then brodalumab (moderate certainty), then guselkumab (moderate certainty), certolizumab (moderate certainty), and ustekinumab (high certainty). In this third review, the number of trials allowed us to conduct a network meta-analysis and classify the treatments. However, a number of points decreased the degree of confidence in the results: the few face-to-face trials, few trials evaluating and reporting on quality of life, the absence of long-term short evaluation and the inclusion of a selected population.These three reviews highlighted recurrent biases that could be applied to other chronic dermatological diseases, including the lack of face-to-face testing, primarily short term evaluations, and the lack of reports including a quality of life criterion. This work also provides guidance for future trials in these three areas in terms of comparisons to be made and bias to be avoided.La décision médicale nécessite d’avoir à disposition une synthèse rigoureuse et actualisée des données de la recherche. Les revues systématiques (RS) et méta-analyses (MA) sont la méthode la plus rigoureuse pour produire ces synthèses. Nous avons réalisé des RS et MA Cochrane sur trois maladies chroniques en dermatologie. Notre objectif était de produire une synthèse des données, de proposer des pistes d’amélioration pour la recherche à venir et d’ouvrir sur des projets de méta-recherche.La première revue avait pour objectif d’évaluer l’efficacité et la tolérance des traitements de la pustulosepalmoplantaire. Nous avons inclus 36 essais contrôlés randomisés (ECR) comprenant un total de 1504 patients. Le comparateur était le placebo ou aucun traitement dans 31 ECR. Au total, nous avons trouvé un faible degré de certitude que l’analogue topique de la vitamine D (maxacalcitrol) était plus efficace que le placebo à court terme ; que l'alitrétinoïne et trois traitements biologiques n'étaient pas plus efficaces que le placebo ; enfin un degré de certitude modéré que le secukinumab (anti IL17) était supérieur au placebo. Dans cette revue, aucune donnée probante n’a été retrouvée pour les principaux traitements utilisés en pratique courante (corticoïdes locaux, photothérapie, acitrétine, méthotrexate, ciclosporine).La méta-analyse en réseau est une méthode statistique permettant de déterminer l’efficacité relative de l’ensemble des traitements y compris pour les comparaisons qui n’ont pas été évaluées dans un ECR. Cette méthode a été appliquée dans les deux revues suivantes.La deuxième revue avait pour objectif d’évaluer l’efficacité et la tolérance relative d’un traitement suppressif par aciclovir, valaciclovir ou famciclovir dans l’herpès génital du sujet immunocompétent. Nous avons inclus 26 ECR portant sur un total de 6950 patients. Les résultats montraient que l'aciclovir, le famciclovir et le valaciclovir diminuaient le risque d’avoir au moins une récidive d'herpès génital sous traitement comparativement au placebo. La méta-analyse en réseau n'a révélé aucune différence statistiquement significative entre les traitements. Dans cette revue, le manque de comparaisons directes, l’absence des critères de jugement pertinents pour les patients, le risque de biais élevé et non clair pour une majorité d’essais et une hétérogénéité importante limitent la pertinence et le niveau de confiance dans les résultats.La troisième revue avait pour objectif d’évaluer l’efficacité et la tolérance relative des traitements systémiques du psoriasis modéré à sévère chez l’adulte. Nous avons identifié 109 essais pertinents pour un total de 39,882 patients. Soixante-quatorze essais ont été inclus dans la méta-analyse en réseau. Cette étude a montré que l’ixekizumab était le meilleur traitementversus placebo (RR 32.45, 95% IC 23.61 à 44.60; (haut degré de certitude)), suivi par le secukinumab (haut degré de certitude), puis le brodalumab (degré de certitude modéré), puis guselkumab (degré de certitude modéré), le certolizumab (degré de certitude modéré), et l’ustekinumab (haut degré de certitude). Dans cette revue, malgré un nombre conséquent d’essais, un certain nombre de points limites diminuaient le degré de confiance dans les résultats : peu d’essais face-face, peu d’essais évaluant et rapportant la qualité de vie, l’absence d’évaluation au long court et l’inclusion d’une population sélectionnée.Ces trois revues ont permis de mettre en évidence des biais récurrents pouvant s’appliquer au traitement de maladies chroniques en dermatologie notamment le manque d’essai face-face, l’évaluation le plus souvent à court terme et le peu d’essais rapportant un critère de qualité de vie. Ce travail permet également de guider les essais à venir dans ces trois domaines afin d’améliorer l’agenda de la recherche, la diffusion des résultats et la rédaction de recommandations

Twenty Percent of Secondary Publications of Randomized Controlled Trials of Drugs Did Not Provide New Results Relative to the Primary Publication

International audienceOBJECTIVE: The objective of the study was to estimate the proportion of secondary publications of randomized controlled trials (RCTs) that provide new results relative to the primary publication. STUDY DESIGN AND SETTING: We searched for RCTs published in 2014 in the five medical journals with the highest impact factors. Secondary publications for each primary publication were then identified by their registration number. The main outcome measure was the proportion of secondary publications providing results already reported in the primary publication and/or nonprespecified analyses and/or a meta-analysis pooling results of studies not identified by systematic review. RESULTS: A total of 144 primary publications were identified; 94 (65%) had at least one secondary publication within 30~months after a primary publication. Of the secondary publications, 20% reported only results present in the primary publication, and 35% reported results not prespecified or pooled analyses not based on a systematic review. Factors associated with having at least one secondary publication were a large number of randomized trial participants (odds ratio [95% confidence interval]: 3.2 [1.1-9.3] for trials with >1,000 vs.~≤q500 participants), investigating a biologic product (4.8 [1.4-16.3] vs. a nonbiologic product) and cardiologic field vs. other fields (7.6 [1.46-39.8]). CONCLUSION: Most drug RCTs with results published in high-impact-factor journals had secondary publications. More than half of these secondary publications provided results already reported in the primary publication or results of nonprespecified analyses

Comparative effectiveness of topical drugs in dermatologic priority diseases: geometry of randomized trial networks

Among the 100 initial priority topics for comparative effectiveness research, three concern topical drugs in the following dermatologic diseases: psoriasis, chronic lower-extremity wounds (CLEWs), and acne vulgaris (AV). Our objective was to explore the geometry of the corresponding networks of randomized controlled trials (RCTs). We performed a review of RCTs on topical drugs in psoriasis, CLEWs, and AV. We searched MEDLINE, Embase, and CENTRAL for published trials from 2007 to 2012 and ClinicalTrials.gov for unpublished trials registered since 2011. RCTs comparing at least one topical treatment with any active or inactive comparator, regardless of RCT design and outcomes, were eligible. We produced network graphs (each node representing a treatment and links between nodes representing trials) and tested for co-occurrence (preference or avoidance of specific comparisons). We included 60 RCTs on psoriasis (14,255 patients) and 19 registered RCTs, 50 of CLEWs (5,916 patients) and 7 registered RCTs, and 90 of AV (22,984 patients) and 21 registered RCTs. Head-to-head comparisons were made in 78%, 32%, and 57% of published RCTs of these conditions, respectively. The co-occurrence test suggested that no specific head-to-head comparison was significantly preferred or avoided (P-value=0.53, 0.20, and 0.57, respectively). This study has limitations, the main being that the search period was restricted to 5 years. In conclusion, more comparative effectiveness trials are needed for CLEWs, for which head-to-head comparisons are fewer than those for psoriasis and AV

Antistreptococcal interventions for guttate and chronic plaque psoriasis

International audienc

Most randomized controlled trials for psoriasis used placebo comparators despite the availability of effective treatments

International audienceBackground: The availability of effective treatments for psoriasis raises ethical questions about the use of a placebo group in therapeutic trials. We evaluated the use of the placebo over time in such trials.Methods: From trials in a living Cochrane review and network meta-analysis for psoriasis, we included trials comparing a biologic to a placebo or other systemic treatment. First, we tested the changes in placebo rate from 2001 to 2019 by linear regression, then constructed networks for 2004-2019 and evaluated the contribution of the placebo to the network meta-analysis estimates per trial and per comparison.Results: We included 81 trials (36,774 patients). The placebo rate did not decrease significantly over time. The proportion contribution of trials with a placebo decreased from 100% in 2004 to 86% in 2008 and 75% in 2019. However, the proportion contribution of trials without a placebo remained low (from 0% in 2004 to 25% in 2019).Conclusion: The design of future psoriasis trials should be reviewed to improve the number of patients to be included in a placebo group

Most randomized controlled trials for psoriasis used placebo comparators despite the availability of effective treatments

International audienceThe availability of effective treatments for psoriasis raises ethical questions about the use of a placebo group in therapeutic trials. We evaluated the use of the placebo over time in such trials