Hierarchical network meta-analysis models for synthesis of evidence from randomised and non-randomised studies

BACKGROUND: With the increased interest in the inclusion of non-randomised data in network meta-analyses (NMAs) of randomised controlled trials (RCTs), analysts need to consider the implications of the differences in study designs as such data can be prone to increased bias due to the lack of randomisation and unmeasured confounding. This study aims to explore and extend a number of NMA models that account for the differences in the study designs, assessing their impact on the effect estimates and uncertainty. METHODS: Bayesian random-effects meta-analytic models, including naïve pooling and hierarchical models differentiating between the study designs, were extended to allow for the treatment class effect and accounting for bias, with further extensions allowing for bias terms to vary depending on the treatment class. Models were applied to an illustrative example in type 2 diabetes; using data from a systematic review of RCTs and non-randomised studies of two classes of glucose-lowering medications: sodium-glucose co-transporter 2 inhibitors and glucagon-like peptide-1 receptor agonists. RESULTS: Across all methods, the estimated mean differences in glycated haemoglobin after 24 and 52 weeks remained similar with the inclusion of observational data. The uncertainty around these estimates reduced when conducting naïve pooling, compared to NMA of RCT data alone, and remained similar when applying hierarchical model allowing for class effect. However, the uncertainty around these effect estimates increased when fitting hierarchical models allowing for the differences in study design. The impact on uncertainty varied between treatments when applying the bias adjustment models. Hierarchical models and bias adjustment models all provided a better fit in comparison to the naïve-pooling method. CONCLUSIONS: Hierarchical and bias adjustment NMA models accounting for study design may be more appropriate when conducting a NMA of RCTs and observational studies. The degree of uncertainty around the effectiveness estimates varied depending on the method but use of hierarchical models accounting for the study design resulted in increased uncertainty. Inclusion of non-randomised data may, however, result in inferences that are more generalisable and the models accounting for the differences in the study design allow for more detailed and appropriate modelling of complex data, preventing overly optimistic conclusions

Methods for the inclusion of real-world evidence in network meta-analysis

Background Network Meta-Analysis (NMA) is a key component of submissions to reimbursement agencies world-wide, especially when there is limited direct head-to-head evidence for multiple technologies from randomised controlled trials (RCTs). Many NMAs include only data from RCTs. However, real-world evidence (RWE) is also becoming widely recognised as a valuable source of clinical data. This study aims to investigate methods for the inclusion of RWE in NMA and its impact on the level of uncertainty around the effectiveness estimates, with particular interest in effectiveness of fingolimod. Methods A range of methods for inclusion of RWE in evidence synthesis were investigated by applying them to an illustrative example in relapsing remitting multiple sclerosis (RRMS). A literature search to identify RCTs and RWE evaluating treatments in RRMS was conducted. To assess the impact of inclusion of RWE on the effectiveness estimates, Bayesian hierarchical and adapted power prior models were applied. The effect of the inclusion of RWE was investigated by varying the degree of down weighting of this part of evidence by the use of a power prior. Results Whilst the inclusion of the RWE led to an increase in the level of uncertainty surrounding effect estimates in this example, this depended on the method of inclusion adopted for the RWE. ‘Power prior’ NMA model resulted in stable effect estimates for fingolimod yet increasing the width of the credible intervals with increasing weight given to RWE data. The hierarchical NMA models were effective in allowing for heterogeneity between study designs, however, this also increased the level of uncertainty. Conclusion The ‘power prior’ method for the inclusion of RWE in NMAs indicates that the degree to which RWE is taken into account can have a significant impact on the overall level of uncertainty. The hierarchical modelling approach further allowed for accommodating differences between study types. Consequently, further work investigating both empirical evidence for biases associated with individual RWE studies and methods of elicitation from experts on the extent of such biases is warranted

Double-counting of populations in evidence synthesis in public health : a call for awareness and future methodological development

Background There is a growing interest in the inclusion of real-world and observational studies in evidence synthesis such as meta-analysis and network meta-analysis in public health. While this approach offers great epidemiological opportunities, use of such studies often introduce a significant issue of double-counting of participants and databases in a single analysis. Therefore, this study aims to introduce and illustrate the nuances of double-counting of individuals in evidence synthesis including real-world and observational data with a focus on public health. Methods The issues associated with double-counting of individuals in evidence synthesis are highlighted with a number of case studies. Further, double-counting of information in varying scenarios is discussed with potential solutions highlighted. Results Use of studies of real-world data and/or established cohort studies, for example studies evaluating the effectiveness of therapies using health record data, often introduce a significant issue of double-counting of individuals and databases. This refers to the inclusion of the same individuals multiple times in a single analysis. Double-counting can occur in a number of manners, such as, when multiple studies utilise the same database, when there is overlapping timeframes of analysis or common treatment arms across studies. Some common practices to address this include synthesis of data only from peer-reviewed studies, utilising the study that provides the greatest information (e.g. largest, newest, greater outcomes reported) or analysing outcomes at different time points. Conclusions While common practices currently used can mitigate some of the impact of double-counting of participants in evidence synthesis including real-world and observational studies, there is a clear need for methodological and guideline development to address this increasingly significant issue. Peer Review reports Introduction Both in the evaluation of health technologies and epidemiological research, systematic reviews and meta-analysis are regarded as providing high quality evidence [1, 2]. With the heightening interest in studies reporting the use of real-world data in health research literature, which include observational studies using registry or electronic health record data collected routinely in clinical practice, the incorporation of these studies in evidence synthesis is becoming increasingly common [3,4,5,6]. Utilising data from all available sources, including observational studies, can provide many benefits in epidemiology, such as increased power and more generalizable results. However, this can often introduce a number of analytical problems such as confounding, significant heterogeneity and misclassification bias within the non-randomised evidence. While methods such as meta-regression have been considered to address these issues [7, 8], a significant problem that has received little attention within public health research is the double-counting, also referred to as sample overlap, of individuals and databases when including such studies in evidence syntheses. With the increased use of cohort and real-world data in evidence synthesis, double-counting has the potential to become a significant issue. Some aspects of double-counting have been discussed by Senn (2009) and Lunny et al.,(2021), specifically in the context of whole studies or study arms which were being included multiple times in the meta-analysis [9, 10]. More attention has been given to this issue in the fields of social science, education, economics and finance, where analytical approaches to dealing with such issues have been suggested [11, 12]. However, currently there is no published guidance available on how to address this. Sample overlap between studies will lead to spuriously high precision in meta-analysis and is also potentially a source of bias. Due to this, many reviewers choose to exclude or adjust for studies where there is an overlap of participants. It may not be obvious if studies contain overlapping patients and so double-counting of individuals in a synthesis may exist without the reviewer’s knowledge. Whilst guidance documents for conducting systematic reviews and meta-analysis of intervention and prevalence/incidence studies exist, none of these consider the effect of the large magnitude of sample overlap expected in whole population studies on meta-analysis results [13, 14]. Therefore, this paper aims to highlight and illustrate some of the specific methodological and practical aspects of double-counting of individuals and datasets that should be considered in evidence syntheses that include real-world and observational data using a number of public health case studies

Global incidence, prevalence, years lived with disability (YLDs), disability-adjusted life-years (DALYs), and healthy life expectancy (HALE) for 371 diseases and injuries in 204 countries and territories and 811 subnational locations, 1990–2021: a systematic analysis for the Global Burden of Disease Study 2021

Background: Detailed, comprehensive, and timely reporting on population health by underlying causes of disability and premature death is crucial to understanding and responding to complex patterns of disease and injury burden over time and across age groups, sexes, and locations. The availability of disease burden estimates can promote evidence-based interventions that enable public health researchers, policy makers, and other professionals to implement strategies that can mitigate diseases. It can also facilitate more rigorous monitoring of progress towards national and international health targets, such as the Sustainable Development Goals. For three decades, the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) has filled that need. A global network of collaborators contributed to the production of GBD 2021 by providing, reviewing, and analysing all available data. GBD estimates are updated routinely with additional data and refined analytical methods. GBD 2021 presents, for the first time, estimates of health loss due to the COVID-19 pandemic. Methods: The GBD 2021 disease and injury burden analysis estimated years lived with disability (YLDs), years of life lost (YLLs), disability-adjusted life-years (DALYs), and healthy life expectancy (HALE) for 371 diseases and injuries using 100 983 data sources. Data were extracted from vital registration systems, verbal autopsies, censuses, household surveys, disease-specific registries, health service contact data, and other sources. YLDs were calculated by multiplying cause-age-sex-location-year-specific prevalence of sequelae by their respective disability weights, for each disease and injury. YLLs were calculated by multiplying cause-age-sex-location-year-specific deaths by the standard life expectancy at the age that death occurred. DALYs were calculated by summing YLDs and YLLs. HALE estimates were produced using YLDs per capita and age-specific mortality rates by location, age, sex, year, and cause. 95% uncertainty intervals (UIs) were generated for all final estimates as the 2·5th and 97·5th percentiles values of 500 draws. Uncertainty was propagated at each step of the estimation process. Counts and age-standardised rates were calculated globally, for seven super-regions, 21 regions, 204 countries and territories (including 21 countries with subnational locations), and 811 subnational locations, from 1990 to 2021. Here we report data for 2010 to 2021 to highlight trends in disease burden over the past decade and through the first 2 years of the COVID-19 pandemic. Findings: Global DALYs increased from 2·63 billion (95% UI 2·44–2·85) in 2010 to 2·88 billion (2·64–3·15) in 2021 for all causes combined. Much of this increase in the number of DALYs was due to population growth and ageing, as indicated by a decrease in global age-standardised all-cause DALY rates of 14·2% (95% UI 10·7–17·3) between 2010 and 2019. Notably, however, this decrease in rates reversed during the first 2 years of the COVID-19 pandemic, with increases in global age-standardised all-cause DALY rates since 2019 of 4·1% (1·8–6·3) in 2020 and 7·2% (4·7–10·0) in 2021. In 2021, COVID-19 was the leading cause of DALYs globally (212·0 million [198·0–234·5] DALYs), followed by ischaemic heart disease (188·3 million [176·7–198·3]), neonatal disorders (186·3 million [162·3–214·9]), and stroke (160·4 million [148·0–171·7]). However, notable health gains were seen among other leading communicable, maternal, neonatal, and nutritional (CMNN) diseases. Globally between 2010 and 2021, the age-standardised DALY rates for HIV/AIDS decreased by 47·8% (43·3–51·7) and for diarrhoeal diseases decreased by 47·0% (39·9–52·9). Non-communicable diseases contributed 1·73 billion (95% UI 1·54–1·94) DALYs in 2021, with a decrease in age-standardised DALY rates since 2010 of 6·4% (95% UI 3·5–9·5). Between 2010 and 2021, among the 25 leading Level 3 causes, age-standardised DALY rates increased most substantially for anxiety disorders (16·7% [14·0–19·8]), depressive disorders (16·4% [11·9–21·3]), and diabetes (14·0% [10·0–17·4]). Age-standardised DALY rates due to injuries decreased globally by 24·0% (20·7–27·2) between 2010 and 2021, although improvements were not uniform across locations, ages, and sexes. Globally, HALE at birth improved slightly, from 61·3 years (58·6–63·6) in 2010 to 62·2 years (59·4–64·7) in 2021. However, despite this overall increase, HALE decreased by 2·2% (1·6–2·9) between 2019 and 2021. Interpretation: Putting the COVID-19 pandemic in the context of a mutually exclusive and collectively exhaustive list of causes of health loss is crucial to understanding its impact and ensuring that health funding and policy address needs at both local and global levels through cost-effective and evidence-based interventions. A global epidemiological transition remains underway. Our findings suggest that prioritising non-communicable disease prevention and treatment policies, as well as strengthening health systems, continues to be crucially important. The progress on reducing the burden of CMNN diseases must not stall; although global trends are improving, the burden of CMNN diseases remains unacceptably high. Evidence-based interventions will help save the lives of young children and mothers and improve the overall health and economic conditions of societies across the world. Governments and multilateral organisations should prioritise pandemic preparedness planning alongside efforts to reduce the burden of diseases and injuries that will strain resources in the coming decades. Funding: Bill & Melinda Gates Foundation

Global burden and strength of evidence for 88 risk factors in 204 countries and 811 subnational locations, 1990–2021: a systematic analysis for the Global Burden of Disease Study 2021

Background: Understanding the health consequences associated with exposure to risk factors is necessary to inform public health policy and practice. To systematically quantify the contributions of risk factor exposures to specific health outcomes, the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2021 aims to provide comprehensive estimates of exposure levels, relative health risks, and attributable burden of disease for 88 risk factors in 204 countries and territories and 811 subnational locations, from 1990 to 2021. Methods: The GBD 2021 risk factor analysis used data from 54 561 total distinct sources to produce epidemiological estimates for 88 risk factors and their associated health outcomes for a total of 631 risk–outcome pairs. Pairs were included on the basis of data-driven determination of a risk–outcome association. Age-sex-location-year-specific estimates were generated at global, regional, and national levels. Our approach followed the comparative risk assessment framework predicated on a causal web of hierarchically organised, potentially combinative, modifiable risks. Relative risks (RRs) of a given outcome occurring as a function of risk factor exposure were estimated separately for each risk–outcome pair, and summary exposure values (SEVs), representing risk-weighted exposure prevalence, and theoretical minimum risk exposure levels (TMRELs) were estimated for each risk factor. These estimates were used to calculate the population attributable fraction (PAF; ie, the proportional change in health risk that would occur if exposure to a risk factor were reduced to the TMREL). The product of PAFs and disease burden associated with a given outcome, measured in disability-adjusted life-years (DALYs), yielded measures of attributable burden (ie, the proportion of total disease burden attributable to a particular risk factor or combination of risk factors). Adjustments for mediation were applied to account for relationships involving risk factors that act indirectly on outcomes via intermediate risks. Attributable burden estimates were stratified by Socio-demographic Index (SDI) quintile and presented as counts, age-standardised rates, and rankings. To complement estimates of RR and attributable burden, newly developed burden of proof risk function (BPRF) methods were applied to yield supplementary, conservative interpretations of risk–outcome associations based on the consistency of underlying evidence, accounting for unexplained heterogeneity between input data from different studies. Estimates reported represent the mean value across 500 draws from the estimate's distribution, with 95% uncertainty intervals (UIs) calculated as the 2·5th and 97·5th percentile values across the draws. Findings: Among the specific risk factors analysed for this study, particulate matter air pollution was the leading contributor to the global disease burden in 2021, contributing 8·0% (95% UI 6·7–9·4) of total DALYs, followed by high systolic blood pressure (SBP; 7·8% [6·4–9·2]), smoking (5·7% [4·7–6·8]), low birthweight and short gestation (5·6% [4·8–6·3]), and high fasting plasma glucose (FPG; 5·4% [4·8–6·0]). For younger demographics (ie, those aged 0–4 years and 5–14 years), risks such as low birthweight and short gestation and unsafe water, sanitation, and handwashing (WaSH) were among the leading risk factors, while for older age groups, metabolic risks such as high SBP, high body-mass index (BMI), high FPG, and high LDL cholesterol had a greater impact. From 2000 to 2021, there was an observable shift in global health challenges, marked by a decline in the number of all-age DALYs broadly attributable to behavioural risks (decrease of 20·7% [13·9–27·7]) and environmental and occupational risks (decrease of 22·0% [15·5–28·8]), coupled with a 49·4% (42·3–56·9) increase in DALYs attributable to metabolic risks, all reflecting ageing populations and changing lifestyles on a global scale. Age-standardised global DALY rates attributable to high BMI and high FPG rose considerably (15·7% [9·9–21·7] for high BMI and 7·9% [3·3–12·9] for high FPG) over this period, with exposure to these risks increasing annually at rates of 1·8% (1·6–1·9) for high BMI and 1·3% (1·1–1·5) for high FPG. By contrast, the global risk-attributable burden and exposure to many other risk factors declined, notably for risks such as child growth failure and unsafe water source, with age-standardised attributable DALYs decreasing by 71·5% (64·4–78·8) for child growth failure and 66·3% (60·2–72·0) for unsafe water source. We separated risk factors into three groups according to trajectory over time: those with a decreasing attributable burden, due largely to declining risk exposure (eg, diet high in trans-fat and household air pollution) but also to proportionally smaller child and youth populations (eg, child and maternal malnutrition); those for which the burden increased moderately in spite of declining risk exposure, due largely to population ageing (eg, smoking); and those for which the burden increased considerably due to both increasing risk exposure and population ageing (eg, ambient particulate matter air pollution, high BMI, high FPG, and high SBP). Interpretation: Substantial progress has been made in reducing the global disease burden attributable to a range of risk factors, particularly those related to maternal and child health, WaSH, and household air pollution. Maintaining efforts to minimise the impact of these risk factors, especially in low SDI locations, is necessary to sustain progress. Successes in moderating the smoking-related burden by reducing risk exposure highlight the need to advance policies that reduce exposure to other leading risk factors such as ambient particulate matter air pollution and high SBP. Troubling increases in high FPG, high BMI, and other risk factors related to obesity and metabolic syndrome indicate an urgent need to identify and implement interventions

Comparative Effectiveness of Sodium-Glucose Co-Transporter 2 Inhibitors and Glucagon-Like Peptide-1 Receptor Agonists in Type 2 Diabetes: Randomised and Real-World Evidence

Sodium-glucose co-transporter 2 inhibitors (SGLT-2i) and glucagon-like peptide-1 receptor agonists (GLP-1RA) are two of the most recently developed treatment classes in the management of blood-glucose levels in individuals with Type 2 diabetes. While individual treatments have been analysed in randomised controlled trials (RCTs) and observational studies, comparisons of treatments within and between treatment classes are not as well established. In this thesis, using a number of methodological techniques, SGLT-2is and GLP-1RAs were compared for their cardiovascular, cardiometabolic and safety risks in both randomised and observational settings. Throughout this thesis, network meta-analyses (NMA) techniques were used to assess the eficacy and effectiveness of SGLT-2is and GLP-1RAs in RCTs and observational data, separately. Using the real-world database, the Clinical Practice Research Datalink, the effectiveness of treatments within these classes were considered in a UK healthcare setting. In order to provide a comprehensive overview of treatments, RCT data and observational data were synthesised using an extension of NMA models with a power-prior transformation, which allowed for adjustments to be made for the various sources of information. Results from this thesis showed both classes of medications improved health outcomes in individuals with Type 2 diabetes in both RCTs and observational studies. Comparisons between treatment groups highlighted the bene?ts of individual treatments for cardiovascular risk, such as hospital admissions due to heart failure, as well as cardiometabolic risk, such as glycated haemoglobin (HbA1c), body weight and blood pressure levels. Assessment of adverse events showed increased risk of genito-mycotic infection with SGLT-2is and increased risk gastro-intestinal side e?ects in GLP-1RAs, in line with their pharmacological mechanisms. These analyses and results can contribute to the growing body of evidence in Type 2 diabetes management, as well as inform clinicians in the most appropriate treatment plan for individualised care in Type 2 diabetes.</p

Methods for the inclusion of real world evidence in network meta-analysis

Background: Network Meta-Analysis (NMA) is a key component of submissions to reimbursement agencies world-wide, especially when there is limited direct head-to-head evidence for multiple technologies from randomised controlled trials (RCTs). Many NMAs include only data from RCTs. However, real-world evidence (RWE) is also becoming widely recognised as a valuable source of clinical data. We investigate methods for the inclusion of RWE in NMA and its impact on the uncertainty around the effectiveness estimates. Methods: A range of methods for inclusion of RWE in evidence synthesis, including Bayesian hierarchical and power prior models, were investigated by applying them to an example in relapsing remitting multiple sclerosis. The effect of the inclusion of RWE was investigated by varying the degree of down weighting of this part of evidence by the use of a power prior. Results: Whilst the inclusion of the RWE led to an increase in the level of uncertainty surrounding effect estimates in this example, this depended on the method of inclusion adopted for the RWE. Power prior NMA model resulted in stable effect estimates for fingolimod yet increasing the width of the credible intervals with increasing weight given to RWE data. The hierarchical NMA models were effective in allowing for heterogeneity between study designs; however, this also increased the level of uncertainty. Conclusion: The power prior approach for the inclusion of RWE in NMAs indicates that the degree to which RWE is taken into account can have a significant impact on the overall level of uncertainty. The hierarchical modelling approach further allowed for accommodating differences between study types. Consequently, further work investigating both empirical evidence for biases associated with individual RWE studies and methods of elicitation from experts on the extent of such biases is warranted.Comment: 24 pages, 3 figure

Development, content validation and reliability of the Assessment of Real World Observational Studies (ArRoWS) critical appraisal tool

ObjectiveThe objective was to develop and test a pragmatic critical appraisal tool, the Assessment of Real World Observational Studies (ArRoWS), to quickly and easily assess the quality of real world evidence studies using electronic health records.MethodsThe initial ArRoWS tool was developed by identifying items frequently found in existing validated assessment instruments and adapting these items to specifically assess real world evidence studies. The tool was revised based on recommendations from an expert panel of 14 senior academic individuals specialising in epidemiology and content validity was measured. During March 2018-January 2019, 47 large, observational studies related to cardiometabolic medicine were identified through a search algorithm and assessed by three pairs of raters using the ArRoWS tool.ResultsThe final version of the ArRoWS had 16 items including nine core items and seven study design-specific items with item-specific content validity indexes ranging from 0.64-1.00. The scale-level content validity index of the ArRoWS appraisal tool was 0.91. When the ArRoWS tool was pilot tested, the observed agreement between assessor pairs on whether the study provided high quality real world evidence was 85.7%, 68.8%, and 58.8%. The prevalence adjusted bias adjusted kappa for the assessor pairs was 0.71, 0.38, and 0.18.ConclusionThe ArRoWS is a simple tool to standardise the assessment of real world evidence studies.</div

Synthesis, characterization and biological evaluation of some novel nitrogen and sulphur containing organometallic heterocycles

A series of some novel sulphur and nitrogen containing ferrocenyl linked heterocyclic compounds were synthesized by multistep reactions and evaluated for in vitro antimicrobial activity against 15 ATCC strains out of which 8 were bacterial (Pseudomonas aeruginosa, Streptococcus bovis, Enterococcus faecalis, Klebsiella pneumoniae, Escherichia coli, Enterobacter cloacae, Methicillin-resistant Staphylococcus aureus and Streptococcus mutans) and 7 were fungal (Candida albicans, Candida dubliniensis, Candida glabrata, Candida parapsilosis, Candida tropicalis, Candida kefyr and Candida krusei) strains. The results clearly depict that the compounds (1–12) gave an average antimicrobial activity against the tested strains with an exception of compound 12 which stood out in terms of its activity against the tested organisms. All these compounds gave a range of MIC value between 32–64 μg/ml against S. bovis, E. coli and C. tropicalis except compound 12 which gave a MIC of 16 μg/ml against each of them. The MIC values of all these compounds against biofilm forming P. aeruginosa and S. mutans were 64–256 μg/ml and 64–128 μg/ml respectively which is apparently high, concluding that these compounds hold immense potential to be employed as a two in one formulation of antibacterial as well as antifungal agents

Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development

Background There is a growing interest in the inclusion of real-world and observational studies in evidence synthesis such as meta-analysis and network meta-analysis in public health. While this approach offers great epidemiological opportunities, use of such studies often introduce a significant issue of double-counting of participants and databases in a single analysis. Therefore, this study aims to introduce and illustrate the nuances of double-counting of individuals in evidence synthesis including real-world and observational data with a focus on public health. Methods The issues associated with double-counting of individuals in evidence synthesis are highlighted with a number of case studies. Further, double-counting of information in varying scenarios is discussed with potential solutions highlighted. Results Use of studies of real-world data and/or established cohort studies, for example studies evaluating the effectiveness of therapies using health record data, often introduce a significant issue of double-counting of individuals and databases. This refers to the inclusion of the same individuals multiple times in a single analysis. Double-counting can occur in a number of manners, such as, when multiple studies utilise the same database, when there is overlapping timeframes of analysis or common treatment arms across studies. Some common practices to address this include synthesis of data only from peer-reviewed studies, utilising the study that provides the greatest information (e.g. largest, newest, greater outcomes reported) or analysing outcomes at different time points. Conclusions While common practices currently used can mitigate some of the impact of double-counting of participants in evidence synthesis including real-world and observational studies, there is a clear need for methodological and guideline development to address this increasingly significant issue.</p