Chapter 10: Deciding Whether to Complement a Systematic Review of Medical Tests with Decision Modeling

Limited by what is reported in the literature, most systematic reviews of medical tests focus on “test accuracy” (or better, test performance), rather than on the impact of testing on patient outcomes. The link between testing, test results and patient outcomes is typically complex: even when testing has high accuracy, there is no guarantee that physicians will act according to test results, that patients will follow their orders, or that the intervention will yield a beneficial endpoint. Therefore, test performance is typically not sufficient for assessing the usefulness of medical tests. Modeling (in the form of decision or economic analysis) is a natural framework for linking test performance data to clinical outcomes. We propose that (some) modeling should be considered to facilitate the interpretation of summary test performance measures by connecting testing and patient outcomes. We discuss a simple algorithm for helping systematic reviewers think through this possibility, and illustrate it by means of an example

Chapter 8: Meta-analysis of Test Performance When There is a “Gold Standard”

Synthesizing information on test performance metrics such as sensitivity, specificity, predictive values and likelihood ratios is often an important part of a systematic review of a medical test. Because many metrics of test performance are of interest, the meta-analysis of medical tests is more complex than the meta-analysis of interventions or associations. Sometimes, a helpful way to summarize medical test studies is to provide a “summary point”, a summary sensitivity and a summary specificity. Other times, when the sensitivity or specificity estimates vary widely or when the test threshold varies, it is more helpful to synthesize data using a “summary line” that describes how the average sensitivity changes with the average specificity. Choosing the most helpful summary is subjective, and in some cases both summaries provide meaningful and complementary information. Because sensitivity and specificity are not independent across studies, the meta-analysis of medical tests is fundamentaly a multivariate problem, and should be addressed with multivariate methods. More complex analyses are needed if studies report results at multiple thresholds for positive tests. At the same time, quantitative analyses are used to explore and explain any observed dissimilarity (heterogeneity) in the results of the examined studies. This can be performed in the context of proper (multivariate) meta-regressions

The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study

BACKGROUND: Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated. METHODS: We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error. RESULTS: The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation. CONCLUSIONS: Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation

Quantitative Analysis of Single Nucleotide Polymorphisms within Copy Number Variation

BACKGROUND: Single nucleotide polymorphisms (SNPs) have been used extensively in genetics and epidemiology studies. Traditionally, SNPs that did not pass the Hardy-Weinberg equilibrium (HWE) test were excluded from these analyses. Many investigators have addressed possible causes for departure from HWE, including genotyping errors, population admixture and segmental duplication. Recent large-scale surveys have revealed abundant structural variations in the human genome, including copy number variations (CNVs). This suggests that a significant number of SNPs must be within these regions, which may cause deviation from HWE. RESULTS: We performed a Bayesian analysis on the potential effect of copy number variation, segmental duplication and genotyping errors on the behavior of SNPs. Our results suggest that copy number variation is a major factor of HWE violation for SNPs with a small minor allele frequency, when the sample size is large and the genotyping error rate is 0~1%. CONCLUSIONS: Our study provides the posterior probability that a SNP falls in a CNV or a segmental duplication, given the observed allele frequency of the SNP, sample size and the significance level of HWE testing

Synopsis and meta-analysis of genetic association studies in osteoporosis for the focal adhesion family genes: the CUMAGAS-OSTEOporosis information system

<p>Abstract</p> <p>Background</p> <p>Focal adhesion (FA) family genes have been studied as candidate genes for osteoporosis, but the results of genetic association studies (GASs) are controversial. To clarify these data, a systematic assessment of GASs for FA genes in osteoporosis was conducted.</p> <p>Methods</p> <p>We developed Cumulative Meta-Analysis of GAS-OSTEOporosis (CUMAGAS-OSTEOporosis), a web-based information system that allows the retrieval, analysis and meta-analysis (for allele contrast, recessive, dominant, additive and codominant models) of data from GASs on osteoporosis with the capability of update. GASs were identified by searching the PubMed and HuGE PubLit databases.</p> <p>Results</p> <p>Data from 72 studies involving 13 variants of 6 genes were analyzed and catalogued in CUMAGAS-OSTEOporosis. Twenty-two studies produced significant associations with osteoporosis risk under any genetic model. All studies were underpowered (<50%). In four studies, the controls deviated from the Hardy-Weinberg equilibrium. Eight variants were chosen for meta-analysis, and significance was shown for the variants collagen, type I, α₁(<it>COL1A1</it>) G2046T (all genetic models), <it>COL1A1 </it>G-1997T (allele contrast and dominant model) and integrin β-chain β₃(<it>ITGB3</it>) T176C (recessive and additive models). In <it>COL1A1 </it>G2046T, subgroup analysis has shown significant associations for Caucasians, adults, females, males and postmenopausal women. A differential magnitude of effect in large versus small studies (that is, indication of publication bias) was detected for the variant <it>COL1A1 </it>G2046T.</p> <p>Conclusion</p> <p>There is evidence of an implication of FA family genes in osteoporosis. CUMAGAS-OSTEOporosis could be a useful tool for current genomic epidemiology research in the field of osteoporosis.</p

Online Survival Analysis Software to Assess the Prognostic Value of Biomarkers Using Transcriptomic Data in Non-Small-Cell Lung Cancer

In the last decade, optimized treatment for non-small cell lung cancer had lead to improved prognosis, but the overall survival is still very short. To further understand the molecular basis of the disease we have to identify biomarkers related to survival. Here we present the development of an online tool suitable for the real-time meta-analysis of published lung cancer microarray datasets to identify biomarkers related to survival. We searched the caBIG, GEO and TCGA repositories to identify samples with published gene expression data and survival information. Univariate and multivariate Cox regression analysis, Kaplan-Meier survival plot with hazard ratio and logrank P value are calculated and plotted in R. The complete analysis tool can be accessed online at: www.kmplot.com/lung. All together 1,715 samples of ten independent datasets were integrated into the system. As a demonstration, we used the tool to validate 21 previously published survival associated biomarkers. Of these, survival was best predicted by CDK1 (p<1E-16), CD24 (p<1E-16) and CADM1 (p = 7E-12) in adenocarcinomas and by CCNE1 (p = 2.3E-09) and VEGF (p = 3.3E-10) in all NSCLC patients. Additional genes significantly correlated to survival include RAD51, CDKN2A, OPN, EZH2, ANXA3, ADAM28 and ERCC1. In summary, we established an integrated database and an online tool capable of uni- and multivariate analysis for in silico validation of new biomarker candidates in non-small cell lung cancer

Discontinuation of Pneumocystis jirovecii Pneumonia Prophylaxis with CD4 Count <200 Cells/µL and Virologic Suppression: A Systematic Review

HIV viral load (VL) is currently not part of the criteria for Pneumocystis jirovecii pneumonia (PCP) prophylaxis discontinuation, but suppression of plasma viremia with antiretroviral therapy may allow for discontinuation of PCP prophylaxis even with CD4 count <200 cells/µL.A systematic review was performed to determine the incidence of PCP in HIV-infected individuals with CD4 count <200 cells/µL and fully suppressed VL on antiretroviral therapy but not receiving PCP prophylaxis.Four articles examined individuals who discontinued PCP prophylaxis with CD4 count <200 cells/µL in the context of fully suppressed VL on antiretroviral therapy. The overall incidence of PCP was 0.48 cases per 100 person-years (PY) (95% confidence interval (CI) (0.06-0.89). This was lower than the incidence of PCP in untreated HIV infection (5.30 cases/100 PY, 95% CI 4.1-6.8) and lower than the incidence in persons with CD4 count <200 cells/µL, before the availability of highly active antiretroviral therapy (HAART), who continued prophylaxis (4.85/100 PY, 95% CI 0.92-8.78). In one study in which individuals were stratified according to CD4 count <200 cells/µL, there was a greater risk of PCP with CD4 count ≤100 cells/µL compared to 101-200 cells/µL.Primary PCP prophylaxis may be safely discontinued in HIV-infected individuals with CD4 count between 101-200 cells/µL provided the VL is fully suppressed on antiretroviral therapy. However, there are inadequate data available to make this recommendation when the CD4 count is ≤100 cells/µL. A revision of guidelines on primary PCP prophylaxis to include consideration of the VL is merited

Genetic Mapping of Social Interaction Behavior in B6/MSM Consomic Mouse Strains

Genetic studies are indispensable for understanding the mechanisms by which individuals develop differences in social behavior. We report genetic mapping of social interaction behavior using inter-subspecific consomic strains established from MSM/Ms (MSM) and C57BL/6J (B6) mice. Two animals of the same strain and sex, aged 10 weeks, were introduced into a novel open-field for 10 min. Social contact was detected by an automated system when the distance between the centers of the two animals became less than ~12 cm. In addition, detailed behavioral observations were made of the males. The wild-derived mouse strain MSM showed significantly longer social contact as compared to B6. Analysis of the consomic panel identified two chromosomes (Chr 6 and Chr 17) with quantitative trait loci (QTL) responsible for lengthened social contact in MSM mice and two chromosomes (Chr 9 and Chr X) with QTL that inhibited social contact. Detailed behavioral analysis of males identified four additional chromosomes associated with social interaction behavior. B6 mice that contained Chr 13 from MSM showed more genital grooming and following than the parental B6 strain, whereas the presence of Chr 8 and Chr 12 from MSM resulted in a reduction of those behaviors. Longer social sniffing was observed in Chr 4 consomic strain than in B6 mice. Although the frequency was low, aggressive behavior was observed in a few pairs from consomic strains for Chrs 4, 13, 15 and 17, as well as from MSM. The social interaction test has been used as a model to measure anxiety, but genetic correlation analysis suggested that social interaction involves different aspects of anxiety than are measured by open-field test

Reporting of Human Genome Epidemiology (HuGE) association studies: An empirical assessment

<p>Abstract</p> <p>Background</p> <p>Several thousand human genome epidemiology association studies are published every year investigating the relationship between common genetic variants and diverse phenotypes. Transparent reporting of study methods and results allows readers to better assess the validity of study findings. Here, we document reporting practices of human genome epidemiology studies.</p> <p>Methods</p> <p>Articles were randomly selected from a continuously updated database of human genome epidemiology association studies to be representative of genetic epidemiology literature. The main analysis evaluated 315 articles published in 2001–2003. For a comparative update, we evaluated 28 more recent articles published in 2006, focusing on issues that were poorly reported in 2001–2003.</p> <p>Results</p> <p>During both time periods, most studies comprised relatively small study populations and examined one or more genetic variants within a single gene. Articles were inconsistent in reporting the data needed to assess selection bias and the methods used to minimize misclassification (of the genotype, outcome, and environmental exposure) or to identify population stratification. Statistical power, the use of unrelated study participants, and the use of replicate samples were reported more often in articles published during 2006 when compared with the earlier sample.</p> <p>Conclusion</p> <p>We conclude that many items needed to assess error and bias in human genome epidemiology association studies are not consistently reported. Although some improvements were seen over time, reporting guidelines and online supplemental material may help enhance the transparency of this literature.</p

Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal

Mounting evidence suggests that there is frequently considerable variation in the risk of the outcome of interest in clinical trial populations. These differences in risk will often cause clinically important heterogeneity in treatment effects (HTE) across the trial population, such that the balance between treatment risks and benefits may differ substantially between large identifiable patient subgroups; the "average" benefit observed in the summary result may even be non-representative of the treatment effect for a typical patient in the trial. Conventional subgroup analyses, which examine whether specific patient characteristics modify the effects of treatment, are usually unable to detect even large variations in treatment benefit (and harm) across risk groups because they do not account for the fact that patients have multiple characteristics simultaneously that affect the likelihood of treatment benefit. Based upon recent evidence on optimal statistical approaches to assessing HTE, we propose a framework that prioritizes the analysis and reporting of multivariate risk-based HTE and suggests that other subgroup analyses should be explicitly labeled either as primary subgroup analyses (well-motivated by prior evidence and intended to produce clinically actionable results) or secondary (exploratory) subgroup analyses (performed to inform future research). A standardized and transparent approach to HTE assessment and reporting could substantially improve clinical trial utility and interpretability