Spearman–Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications

Objectives There are similarities between the different forms of reliability, such as internal consistency (internal reliability) and interrater and intrarater reliability. Reliability coefficients that are based on classical test theory can be expressed as intraclass correlation coefficients (ICCs), such as Cronbach's alpha. The Spearman–Brown prophecy formula (SB formula) is used to calculate the reliability when the number of items in a questionnaire is changed. This paper aims to increase insight into reliability studies by pointing to the assumptions of reliability coefficients, similarities between various coefficients, and the subsequent new applications of reliability coefficients. Design, Settings and Results The origin and assumptions of Cronbach's alpha and the SB formula are discussed. Cronbach's alpha is written as an ICC formula, using the well-known property that taking the average value of a number of ratings increases the reliability of a measurement. We illustrate with an example that the ICC formulas for average measurements of multiple raters and the SB formula give similar results. This implies that the SB formula can be used to decide on the number of measurements to be averaged and thus on the number of raters required, for obtaining measurements with acceptable reliability, even if the variance components of the ICC formula are not known. Using the same example, we illustrate the principle of “Cronbach's alpha if item deleted” to decide on the poorest performing raters in a set of raters. Conclusion These applications have different assumptions: the principle of “Cronbach's alpha if item deleted” is based on the assumption of a fixed set of items/raters and the SB formula is based on the assumption of random raters. The example also emphasizes the need for more raters in the design of the reliability study to obtain a robust estimation of reliability

The use and reliability of SymNose for quantitative measurement of the nose and lip in unilateral cleft lip and palate patients

INTRODUCTION: It is essential to have a reliable assessment method in order to compare the results of cleft lip and palate surgery. In this study the computer-based program SymNose, a method for quantitative assessment of the nose and lip, will be assessed on usability and reliability. METHODS: The symmetry of the nose and lip was measured twice in 50 six-year-old complete and incomplete unilateral cleft lip and palate patients by four observers. For the frontal view the asymmetry level of the nose and upper lip were evaluated and for the basal view the asymmetry level of the nose and nostrils were evaluated. RESULTS: A mean inter-observer reliability when tracing each image once or twice was 0.70 and 0.75, respectively. Tracing the photographs with 2 observers and 4 observers gave a mean inter-observer score of 0.86 and 0.92, respectively. The mean intra-observer reliability varied between 0.80 and 0.84. CONCLUSIONS: SymNose is a practical and reliable tool for the retrospective assessment of large caseloads of 2D photographs of cleft patients for research purposes. Moderate to high single inter-observer reliability was found. For future research with SymNose reliable outcomes can be achieved by using the average outcomes of single tracings of two observers

Aesthetics Assessment and Patient Reported Outcome of Nasolabial Aesthetics in 18-Year-Old Patients With Unilateral Cleft Lip

Objective: To determine if there is a correlation between objective nasolabial aesthetics assessment using the Cleft Aesthetic Rating Scale (CARS) and patient satisfaction. Design: Retrospective analysis of a generic satisfaction questionnaire and independent assessment by three cleft surgeons of the nasolabial area of these patients on 2D frontal photographs, using the CARS. Setting: The Vrije Universiteit Medical Center and The Academic Center for Dentistry Amsterdam. Patients: Thirty-nine 18-year old patients with a repaired complete or incomplete unilateral cleft lip, with or without a cleft palate, and a completed satisfaction questionnaire. Exclusion criteria were an incomplete questionnaire; a history of facial trauma; and congenital syndromes affecting facial appearance. Main Outcome Measures: The correlation between surgeon evaluation (on a 5-point Likert scale) and patient satisfaction (not, moderately or very satisfied) on nasolabial appearance was assessed using Spearman rho (ρ). Results: There was a negligible correlation between surgeon evaluation and patient satisfaction on nose assessment (ρ = 0.20) and a moderate correlation on lip assessment (ρ = 0.32). Conclusions: Most literature supports this discrepancy between different objective aesthetics evaluation methods and subjective patient-reported outcome measures, suggesting there are factors playing a role in patient satisfaction that are impossible to objectify with assessment methods. Therefore, a strong emphasis should remain on clear communication between the physician and patient regarding their expectations, perception, and satisfaction of surgery results

The development of the cleft aesthetic rating scale: A new rating scale for the assessment of nasolabial appearance in complete unilateral cleft lip and palate patients

Objective: The development of the Cleft Aesthetic Rating Scale, a simple and reliable photographic reference scale for the assessment of nasolabial appearance in complete unilateral cleft lip and palate patients. Design: A blind retrospective analysis of photographs of cleft lip and palate patients was performed with this new rating scale. Setting: VU Medical Center Amsterdam and the Academic Center for Dentistry of Amsterdam. Patients: Complete unilateral cleft lip and palate patients at the age of 6 years. Main Outcome Measures: Photographs that showed the highest interobserver agreement in earlier assessments were selected for the photographic reference scale. Rules were attached to the rating scale to provide a guideline for the assessment and improve interobserver reliability. Cropped photographs revealing only the nasolabial area were assessed by six observers using this new Cleft Aesthetic Rating Scale in two different sessions. Results: Photographs of 62 children (6 years of age, 44 boys and 18 girls) were assessed. The interobserver reliability for the nose and lip together was 0.62, obtained with the intraclass correlation coefficient. To measure the internal consistency, a Cronbach alpha of .91 was calculated. The estimated reliability for three observers was .84, obtained with the Spearman Brown formula. Conclusion: A new, easy to use, and reliable scoring system with a photographic reference scale is presented in this study

Atypical Outcomes of Nasal and Lip Appearance After Unilateral Cleft Lip Repair: Judgment by Professionals, Patients, and Laypeople

Objective: To gain more insight into the assessment of “atypical” nasal and lip appearance outcomes compared to “typical” appearance outcomes after unilateral cleft lip and palate (UCLP) repair, when judged by professionals, patients with repaired UCLP, and laypeople. Design: An online survey containing 3 series of photographs with various degrees of “typical” and “atypical” nasal and lip appearance outcomes after UCLP repair was sent to 30 professionals, 30 patients with repaired UCLP, and 50 laypeople in 2 countries. Participants were instructed to rank the photographs from excellent to poor based on overall appearance. Mean rank positions of photographs were analyzed and differences in mean rank score between “typical” and “atypical” results were assessed using a T-test. Agreement of ranking between the 3 groups was assessed with an analysis of variance analysis. Setting: Amsterdam UMC, location VUmc, Netherlands and Boston Children’s Hospital, Boston, USA. Patients: Photographs of 6- to 18-year-old patients with repaired UCLP. Results: “Atypical” appearance outcomes were ranked significantly less favorably (small nostril: P = 0.00; low vermillion border: P = 0.02; whistling deformity: P = 0.00) compared to “typical” outcomes. Difference between professionals, patients and laypeople in rank positioning the photographs was not statistically significant (P = 0.89). Conclusions: Noses with a smaller nostril and lips containing a whistling deformity were perceived as poorer outcome compared to the “typical” results. Professionals, patients, and laypeople are in agreement when assessing these outcomes

Development of the submental nasal appearance scale for the assessment of repaired unilateral complete cleft lip: A pilot study

Objective: To develop the “Submental Nasal Appearance Scale” (SNAS), which is an easy-to-use objectified tool to represent a cleft surgeon’s standard for assessment of the nasal appearance from the submental perspective. Design: Eighty-five photographs of patients with unilateral complete cleft lip and palate were selected and cropped, displaying the submental view. Sixty-one photographs were used to develop 5 sets of reference photographs. Three cleft surgeons graded 24 photographs with these sets and subjectively graded the overall nasal appearance as well. Internal agreement for both methods was calculated, as well as correlation between them. The SNAS was created, by only using the combination of sets that showed the highest reliability and correlation. Setting: Boston Children’s Hospital, Boston, Massachusetts. Patients: Six-to 9-year-old patients with unilateral complete cleft lip and palate. Results: The intrarater and interrater reliability was 0.84 and 0.79, respectively, for the SNAS and 0.76 and 0.62, respectively, for the overall appearance assessment. The correlation was 0.74 between the methods. Conclusions: The SNAS is a reliable tool that reflects a cleft surgeon’s standard and could be used independently or in combination with existing rating scales using the frontal and/or lateral view, for assessment after cleft lip repair