Estimation and Inference of the Three-Level Intraclass Correlation Coefficient

Since the early 1900\u27s, the intraclass correlation coefficient (ICC) has been used to quantify the level of agreement among different assessments on the same object. By comparing the level of variability that exists within subjects to the overall error, a measure of the agreement among the different assessments can be calculated. Historically, this has been performed using subject as the only random effect. However, there are many cases where other nested effects, such as site, should be controlled for when calculating the ICC to determine the chance corrected agreement adjusted for other nested factors. We will present a unified framework to estimate both the two-level and three-level ICC for both binomial and multinomial outcomes. In addition, the corresponding standard errors and confidence intervals for both ICC measurements will be displayed. Finally, an example of the effect that controlling for site can have on ICC measures will be presented for subjects nested within genotyping plates comparing genetically determined race to patient reported race. In addition, when determining agreement on a multinomial response, the question of homogeneity of agreement of individual categories within the multinomial response is raised. One such scenario is the GO project at the University of Pennsylvania where subjects ages 8-21 were asked to rate a series of actors\u27 faces as happy, sad, angry, fearful or neutral. Methods exist to quantify overall agreement among the five responses, but only if the ICCs for each item-wise response are homogeneous. We will present a method to determine homogeneity of ICCs of the item-wise responses across a multinomial outcome and provide simulation results that demonstrate strong control of the type I error rate. This method will subsequently be extended to verify the assumptions of homogeneity of ICCs in the multinomial nested-level model to determine if the overall nested-level ICC is sufficient to describe the nested-level agreement

Confidence Intervals for Ratios of Proportions in Stratified Bilateral Correlated Data

Confidence interval (CI) methods for stratified bilateral studies use intraclass correlation to avoid misleading results. In this article, we propose four CI methods (sample-size weighted global MLE-based Wald-type CI, complete MLE-based Wald-type CI, profile likelihood CI, and complete MLE-based score CI) to investigate CIs of proportion ratios to clinical trial design with stratified bilateral data under Dallal's intraclass model. Monte Carlo simulations are performed, and the complete MLE-based score confidence interval (CS) method yields a robust outcome. Lastly, a real data example is conducted to illustrate the proposed four CIs.Comment: arXiv admin note: text overlap with arXiv:2303.1294

Does a Novice Technician Produce Results Similar to That of an Experienced DXA Technician When Assessing Body Composition and Bone Mineral Density?

Dual energy X-ray absorptiometry is a commonly used clinical assessment tool for body composition and bone mineral density, which is gaining popularity in athletic cohorts. Results from body composition scans are useful for athletic populations to track training and nutritional interventions, whilst bone mineral density scans are valuable for athletes at risk of developing stress fractures due to low bone mineral density. However, no research has ascertained if a novice technician (accredited but not experienced) could produce similar results to an experienced technician. Two groups of recreational athletes were scanned, one by an experienced technician, one by a novice technician. All participants were scanned twice with repositioning between scans. The experienced technician\u27s reliability (ICC 0.989 - 0.998, percentage change in mean -0.01 - 0.10), precision (typical error as CV% 0.01 to 0.47. standard error of measurement percentage 0.61% - 1.39%) and sensitivity to change (smallest real difference percentage 1.70% - 3.85%) were similar, however superior, to those of the novice technician. The novice technician results were: reliability (ICC 0.985 - 0.997, percentage change in mean -0.03 - 0.23), precision (typical error as CV% 0.03 - 0.75%, standard error of measurement percentage 1.06% - 2.12%) and sensitivity to change (smallest real difference percentage 2.73% - 5.86%). Extensive experience whilst valuable is not a necessary requirement to produce quality results when undertaking whole body dual energy X-ray absorptiometry scanning

Factor structure of The Opening Minds Stigma Scale for Health Care Providers and psychometric properties of its Hungarian version

BACKGROUND: The Opening Minds Stigma Scale for Health Care Providers (OMS-HC) is a widely used questionnaire to measure the stigmatising attitudes of healthcare providers towards patients with mental health problems. The psychometric properties of the scale; however, have never been investigated in Hungary. We aimed to thoroughly explore the factor structure of the OMS-HC and examine the key psychometric properties of the Hungarian version. METHODS: The OMS-HC is a self-report questionnaire that measures the overall stigmatising attitude by a total score, and three subscales can be calculated: Attitude, Disclosure and Help-seeking, and Social Distance. Our study population included specialists and trainees in adult and child psychiatry (n = 211). Exploratory and confirmatory factor analyses were performed, and higher-order factors were tested. We calculated the test-retest reliability on a subgroup of our sample (n = 31) with a follow-up period of 1 month. The concurrent validity of the scale was measured with the Mental Illness: Clinician\u27s Attitudes-4 scale (MICA-4). RESULTS: Three factors were extracted based on a parallel-analysis. A bifactor solution (a general factor and three specific factors) showed an excellent model-fit (root mean square error of approximation = 0.025, comparative fit index = 0.961, and Tucker-Lewis index = 0.944). The model-based reliability was low; however, the general factor showed acceptable reliability (coefficient omega hierarchical = 0.56). The scale demonstrated a good concurrent validity with the MICA-4 [intraclass correlation coefficient (ICC) = 0.77]. The test-retest reliability was excellent for the general factor (ICC = 0.95) and good for the specific factors (ICC = 0.90, 0.88, and 0.84, respectively). CONCLUSIONS: The three dimensions of the OMS-HC was confirmed, and the scale was found to be an adequate measure of the stigmatising attitude in Hungary. The bifactor model is more favourable as compared to the three correlated factor model; however, despite the excellent internal structure, its model-based reliability was low

Heterogeneity issues in the meta-analysis of cluster randomization trials.

An increasing number of systematic reviews summarize results from cluster randomization trials. Applying existing meta-analysis methods to such trials is problematic because responses of subjects within clusters are likely correlated. The aim of this thesis is to evaluate heterogeneity in the context of fixed effects models providing guidance for conducting a meta-analysis of such trials. The approaches include the adjusted Q statistic, adjusted heterogeneity variance estimators and their corresponding confidence intervals and adjusted measures of heterogeneity and their corresponding confidence intervals. Attention is limited to meta-analyses of completely randomized trials having a binary outcome. An analytic expression for power of Q test is derived, which may be useful in planning a meta-analysis. The Type I error and power for the Q statistic, bias and mean square errors for the estimators and the coverage, tail errors and interval width for the confidence interval methods are investigated using Monte Carlo simulation. Simulation results show that the adjusted Q statistic has a Type I error close to the nominal level of 0.05 as compared to the unadjusted Q statistic which has a highly inflated Type I error. Power estimated using the algebraic formula had similar results to empirical power. For the heterogeneity variance estimators, the iterative REML estimator consistently had little bias. However, the noniterative MVVC and DLVC estimators with relatively low bias may also be recommended for small and large heterogeneity, respectively. The Q profile confidence interval approach for heterogeneity variance had generally nominal coverage for large heterogeneity. The measures of heterogeneity had generally low bias for large number of trials. For confidence interval approaches, the MOVER consistently maintained nominal coverage for \u27low\u27 to \u27moderate\u27 heterogeneity. For the absence of heterogeneity, the approach based on the Q statistic is preferred. Data from four cluster randomization trials are used to illustrate methods of analysis

Variation of carapace morphology of bairdiacean and cytheracean Ostracoda from Bermuda

10 p., 2 fig.http://paleo.ku.edu/contributions.htm

Reliability and measurement of inter-limb asymmetries in four unilateral jump tests in elite youth female soccer players

Purpose: The purpose of this study was to determine the within and between-session reliability, and inter-limb asymmetries, in four unilateral jump tests in elite youth female soccer players. Given the low plyometric training age and paucity of data for this population, this research study was warranted. Methods: Nineteen elite youth female soccer players (age: 10 ± 1.1 years; height: 141 ± 7.9 cm; body mass: 35 ± 7.1 kg) were recruited from an elite Tier 1 Regional Talent Centre of a professional soccer club. Tests included the single leg countermovement jump (SLCMJ), single leg hop, triple hop, and crossover hops for distance with reliability quantified via the coefficient of variation (CV), intraclass correlation coefficient (ICC), and standard error of the measurement (SEM). Inter-limb asymmetries were also calculated. Results: Both test sessions resulted in excellent within-session reliability (ICC range = 0.81-0.99; SEM range = 0.11-0.49; and CV range = 2.6-6.0%). Between-session reliability was deemed good to excellent (ICC range = 0.72-0.99 and pooled CV = 2.7-5.7%). Asymmetries were deemed small across both test sessions with the highest value reported in the SLCMJ (6.12%). Conclusion: Results highlight that unilateral jump tests can be considered a reliable test protocol in elite youth female soccer players, which is important considering youth athletes likely do not have a vast plyometric training age. Furthermore, inter-limb differences appear small in the present sample which may also be explained by their limited training age, given that asymmetries have previously been highlighted to be a product of limb function over time

How accurate are national stereotypes? A test of different methodological approaches

We compared different methodological approaches in research on the accuracy of national stereotypes that use aggregated mean scores of real people's personality traits as criteria for stereotype accuracy. Our sample comprised 16,713 participants from the Central Europe and 1,090 participants from the Baltic Sea region. Participants rated national stereotypes of their own country using the National Character Survey (NCS) and their personality traits using either the Revised NEO Personality Inventory or the NCS. We examined the effects of different (i) methods for rating of real people (Revised NEO Personality Inventory vs. NCS) and national stereotypes (NCS); (ii) norms for converting raw scores into T‐scores (Russian vs. international norms); and (iii) correlation techniques (intraclass correlations vs. Pearson correlations vs. rank‐order correlations) on the resulting agreement between the ratings of national stereotypes and real people. We showed that the accuracy of national stereotypes depended on the employed methodology. The accuracy was the highest when ratings of real people and national stereotypes were made using the same method and when rank order correlations were used to estimate the agreement between national stereotypes and personality profiles of real people. We propose a new statistical procedure for determining national stereotype accuracy that overcomes limitations of past studies. We provide methodological recommendations applicable to a wider range of cross national stereotype accuracy studies