Interobserver variability studies in diagnostic imaging: a methodological systematic review

OBJECTIVES: To review the methodology of interobserver variability studies; including current practice and quality of conducting and reporting studies. METHODS: Interobserver variability studies between January 2019 and January 2020 were included; extracted data comprised of study characteristics, populations, variability measures, key results, and conclusions. Risk of bias was assessed using the COSMIN tool for assessing reliability and measurement error. RESULTS: Seventy-nine full-text studies were included covering various imaging tests and clinical areas. The median number of patients was 47 (IQR:23-88), and observers were 4 (IQR:2-7), with sample size justified in 12 (15%) studies. Most studies used static images (n = 75, 95%), where all observers interpreted images for all patients (n = 67, 85%). Intraclass correlation coefficients (ICC) (n = 41, 52%), Kappa (κ) statistics (n = 31, 39%) and percentage agreement (n = 15, 19%) were most commonly used. Interpretation of variability estimates often did not correspond with study conclusions. The COSMIN risk of bias tool gave a very good/adequate rating for 52 studies (66%) including any studies that used variability measures listed in the tool. For studies using static images, some study design standards were not applicable and did not contribute to the overall rating. CONCLUSIONS: Interobserver variability studies have diverse study designs and methods, the impact of which requires further evaluation. Sample size for patients and observers was often small without justification. Most studies report ICC and κ values, which did not always coincide with the study conclusion. High ratings were assigned to many studies using the COSMIN risk of bias tool, with certain standards scored 'not applicable' when static images were used. ADVANCES IN KNOWLEDGE: The sample size for both patients and observers was often small without justification. For most studies, observers interpreted static images and did not evaluate the process of acquiring the imaging test, meaning it was not possible to assess many COSMIN risk of bias standards for studies with this design. Most studies reported intraclass correlation coefficient and κ statistics; study conclusions often did not correspond with results

Beliefs about Appropriate Antibacterial Therapy, California

To our knowledge, previous population-based surveys have not assessed misconceptions about antibacterial drug use over time. We documented a 26.3% decline in a key misconception in California women in 2003 compared to 2000; declines varied significantly by education level. Educational campaigns specifically designed to influence important subpopulations are needed

Eustachian tube dysfunction: A diagnostic accuracy study and proposed diagnostic pathway.

BACKGROUND AND AIMS: Eustachian tube dysfunction (ETD) is a commonly diagnosed disorder of Eustachian tube opening and closure, which may be associated with severe symptoms and middle ear disease. Currently the diagnosis of obstructive and patulous forms of ETD is primarily based on non-specific symptoms or examination findings, rather than measurement of the underlying function of the Eustachian tube. This has proved problematic when selecting patients for treatment, and when designing trial inclusion criteria and outcomes. This study aims to determine the correlation and diagnostic value of various tests of ET opening and patient reported outcome measures (PROMs), in order to generate a recommended diagnostic pathway for ETD. METHODS: Index tests included two PROMs and 14 tests of ET opening (nine for obstructive, five for patulous ETD). In the absence of an accepted reference standard two methods were adopted to establish index test accuracy: expert panel diagnosis and latent class analysis. Index test results were assessed with Pearson correlation and principle component analysis, and test accuracy was determined. Logistic regression models assessed the predictive value of grouped test results. RESULTS: The expert panel diagnosis and PROMs results correlated with each other, but not with ET function measured by tests of ET opening. All index tests were found to be feasible in clinic, and acceptable to patients. PROMs had very poor specificity, and no diagnostic value. Combining the results of tests of ET function appeared beneficial. The latent class model suggested tympanometry, sonotubometry and tubomanometry have the best diagnostic performance for obstructive ETD, and these are included in a proposed diagnostic pathway. CONCLUSIONS: ETD should be diagnosed on the basis of clinical assessment and tests of ET opening, as PROMs have no diagnostic value. Currently diagnostic uncertainty exists for some patients who appear to have intermittent ETD clinically, but have negative index test results.M.S. received funding from the Cambridge Hearing Trus

Interobserver variability studies in diagnostic imaging:a methodological systematic review

Objectives: To review the methodology of interobserver variability studies; including current practice and quality of conducting and reporting studies. Methods: Interobserver variability studies between January 2019 and January 2020 were included; extracted data comprised of study characteristics, populations, variability measures, key results, and conclusions. Risk of bias was assessed using the COSMIN tool for assessing reliability and measurement error. Results: Seventy-nine full-text studies were included covering various imaging tests and clinical areas. The median number of patients was 47 (IQR:23–88), and observers were 4 (IQR:2–7), with sample size justified in 12 (15%) studies. Most studies used static images (n = 75, 95%), where all observers interpreted images for all patients (n = 67, 85%). Intraclass correlation coefficients (ICC) (n = 41, 52%), Kappa (κ) statistics (n = 31, 39%) and percentage agreement (n = 15, 19%) were most commonly used. Interpretation of variability estimates often did not correspond with study conclusions. The COSMIN risk of bias tool gave a very good/adequate rating for 52 studies (66%) including any studies that used variability measures listed in the tool. For studies using static images, some study design standards were not applicable and did not contribute to the overall rating. Conclusions: Interobserver variability studies have diverse study designs and methods, the impact of which requires further evaluation. Sample size for patients and observers was often small without justification. Most studies report ICC and κ values, which did not always coincide with the study conclusion. High ratings were assigned to many studies using the COSMIN risk of bias tool, with certain standards scored ‘not applicable’ when static images were used. Advances in knowledge: The sample size for both patients and observers was often small without justification. For most studies, observers interpreted static images and did not evaluate the process of acquiring the imaging test, meaning it was not possible to assess many COSMIN risk of bias standards for studies with this design. Most studies reported intraclass correlation coefficient and κ statistics; study conclusions often did not correspond with results