Chapter 9: Options for Summarizing Medical Test Performance in the Absence of a “Gold Standard”

The classical paradigm for evaluating test performance compares the results of an index test with a reference test. When the reference test does not mirror the “truth” adequately well (e.g. is an “imperfect” reference standard), the typical (“naïve”) estimates of sensitivity and specificity are biased. One has at least four options when performing a systematic review of test performance when the reference standard is “imperfect”: (a) to forgo the classical paradigm and assess the index test’s ability to predict patient relevant outcomes instead of test accuracy (i.e., treat the index test as a predictive instrument); (b) to assess whether the results of the two tests (index and reference) agree or disagree (i.e., treat them as two alternative measurement methods); (c) to calculate “naïve” estimates of the index test’s sensitivity and specificity from each study included in the review and discuss in which direction they are biased; (d) mathematically adjust the “naïve” estimates of sensitivity and specificity of the index test to account for the imperfect reference standard. We discuss these options and illustrate some of them through examples

Screening for hypoglycemia at the bedside in the neonatal intensive care unit (NICU) with the Abbott PCx glucose meter

BACKGROUND: Point of care (POC) glucose meters are routinely used as a screening tool for hypoglycemia in a neonatal setting. Glucose meters however, lack the same accuracy as laboratory instruments for glucose measurement. In this study we investigated potential reasons for this inaccuracy and established a cut off value for confirmatory testing. METHODS: In this prospective study, all patients in the neonatal intensive care unit who had a plasma glucose test ordered were eligible to participate. Demographic information, sample collection information (nine variables) and a recent hematocrit value were recorded for each sample. Glucose measurements were taken at the bedside on the glucose meter (RN PCx) as well as in the laboratory on both the glucose meter (LAB PCx) and the laboratory analyzer (PG). Data were analyzed by simple and mixed-effects regression analysis and by analysis of a receiver operator characteristics (ROC) curve. RESULTS: There were 475 samples analyzed from 132 patients. RN PCx values were higher than PG values (mean = 4.9%), while LAB PCx results were lower (mean = -5.2%) than PG values. Only 31% of the difference between RN PCx – PG and 46% of the difference for LAB PCx – PG could be accounted for by the variables tested. The largest proportion of variance between PCx and PG measurements was explained by hematocrit (about 30%) with a greater effect seen at glucose concentrations ≤4.0 mmol/L (≤72 mg/dL)(48% and 40% for RN PCx and LAB PCx, respectively). The ROC analysis showed that for detection of all cases of hypoglycemia (PG < 2.6 mmol/L)(PG < 47 mg/dL) the PCx screening cut off value would need to be set at 3.8 mmol/L (68 mg/dL) requiring 20% of all samples to have confirmatory analysis by the laboratory method. CONCLUSION: The large difference between glucose results obtained by PCx glucose meter compared to the laboratory analyzer can be explained in part by hematocrit and low glucose concentration. These results emphasize that the glucose meter is useful only as a screening device for neonatal hypoglycemia and that a screening cut off value must be established

Brain type carnosinase in dementia: a pilot study

Research articl

Chapter 8: Meta-analysis of Test Performance When There is a “Gold Standard”

Synthesizing information on test performance metrics such as sensitivity, specificity, predictive values and likelihood ratios is often an important part of a systematic review of a medical test. Because many metrics of test performance are of interest, the meta-analysis of medical tests is more complex than the meta-analysis of interventions or associations. Sometimes, a helpful way to summarize medical test studies is to provide a “summary point”, a summary sensitivity and a summary specificity. Other times, when the sensitivity or specificity estimates vary widely or when the test threshold varies, it is more helpful to synthesize data using a “summary line” that describes how the average sensitivity changes with the average specificity. Choosing the most helpful summary is subjective, and in some cases both summaries provide meaningful and complementary information. Because sensitivity and specificity are not independent across studies, the meta-analysis of medical tests is fundamentaly a multivariate problem, and should be addressed with multivariate methods. More complex analyses are needed if studies report results at multiple thresholds for positive tests. At the same time, quantitative analyses are used to explore and explain any observed dissimilarity (heterogeneity) in the results of the examined studies. This can be performed in the context of proper (multivariate) meta-regressions

The relation between DNA methylation patterns and serum cytokine levels in community-dwelling adults: a preliminary study

Background: The levels of circulating cytokines fluctuate with age, acute illness, and chronic disease, and are predictive of mortality; this is also true for patterns of DNA (CpG) methylation. Given that immune cells are particularly sensitive to changes in the concentration of cytokines in their microenvironment, we hypothesized that serum levels of TNF, IL-6, IL-8 and IL-10 would correlate with genome-wide alterations in the DNA methylation levels of blood leukocytes. To test this, we evaluated community-dwelling adults (n = 14; 48–78 years old) recruited to a pilot study for the Canadian Longitudinal Study on Aging (CLSA), examining DNA methylation patterns in peripheral blood mononuclear cells using the Illumina HumanMethylation 450 K BeadChip. Results: We show that, apart from age, serum IL-10 levels exhibited the most substantial association to DNA methylation patterns, followed by TNF, IL-6 and IL-8. Furthermore, while the levels of these cytokines were higher in elderly adults, no associations with epigenetic accelerated aging, derived using the epigenetic clock, were observed. Conclusions: As a preliminary study with a small sample size, the conclusions drawn from this work must be viewed with caution; however, our observations are encouraging and certainly warrant more suitably powered studies of this relationship.Medicine, Faculty ofOther UBCNon UBCMedical Genetics, Department ofReviewedFacult

Three on Three: Universal and High-Affinity Molecular Recognition of the Symmetric Homotrimeric Spike Protein of SARS-CoV‑2 with a Symmetric Homotrimeric Aptamer

Our previously discovered monomeric aptamer for SARS-CoV-2 (MSA52) possesses a universal affinity for COVID-19 spike protein variants but is ultimately limited by its ability to bind only one subunit of the spike protein. The symmetrical shape of the homotrimeric SARS-CoV-2 spike protein presents the opportunity to create a matching homotrimeric molecular recognition element that is perfectly complementary to its structural scaffold, causing enhanced binding affinity. Here, we describe a branched homotrimeric aptamer with three-fold rotational symmetry, named TMSA52, that not only possesses excellent binding affinity but is also capable of binding several SARS-CoV-2 spike protein variants with picomolar affinity, as well as pseudotyped lentiviruses expressing SARS-CoV-2 spike protein variants with femtomolar affinity. Using Pd–Ir nanocubes as nanozymes in an enzyme-linked aptamer binding assay (ELABA), TMSA52 was capable of sensitively detecting diverse pseudotyped lentiviruses in pooled human saliva with a limit of detection as low as 6.3 × 103 copies/mL. The ELABA was also used to test 50 SARS-CoV-2-positive and 60 SARS-CoV-2-negative patient saliva samples, providing sensitivity and specificity values of 84.0 and 98.3%, respectively, thus highlighting the potential of TMSA52 for the development of future rapid tests