Impact of interjudge agreement on perceptual judgment of hypernasality

Abstract

Objective: To investigate the effect of perceptual interjudge agreement of hypernasality on velopharyngeal (VP) closure prediction. Methods: Two logistic regression models were developed and compared in order to verify the effect of interjudge agreement of perceptual hypernasality evaluating on models results. The models were built aiming to verify the possibility of predicting the VP closure by using the following characteristics: rating of VP closure (adequate, borderline, inadequate), determined by the pressure-flow technique, degree of hypernasality (absent, mild, moderate, severe), and the presence/absence of nasal air emission and nasal rustle determined perceptually by three speech pathologists. In the first model 100 speech samples with a moderate agreement rate of hypernasality (kappa coefficient) were used. In the second model 43 speech samples with a perfect agreement among judges were included. The Qui-square test was used to compare the models (p≤5%). Results: In the first model, 65% (65/100) of the samples were rated in the correct VP closure category, with 93% (42/45) adequate and 35% (23/35) inadequate. However, the borderline VP closure was not predicted. The second model rated 72% (31/43) in the correct category, with 95.5% (21/22) adequate VP closure, 45.5% (5/11) in the borderline VP closure and 50% (5/10) inadequate. The data analysis showed that, although there was no significant difference (p=0.526) between the two models, the second one showed a higher proportion of 7% accuracy than the first model, and also it has predicted the borderline VP closure. Conclusion: These results showed the importance of high index of interjudge agreement when using subjective parameters of speech evaluation, especially when compared to an instrumental evaluation