What is the optimal assessment of speech?:A multicentre, international evaluation of speech assessment in 2500 patients with a cleft

OBJECTIVES: Speech problems in patients with a cleft palate are often complex and multifactorial. Finding the optimal way of monitoring these problems is challenging. The International Consortium of Health Outcomes Measurement (ICHOM) has developed a set of standardised outcome measures at specific ages for patients with a cleft lip and/or palate, including measures of speech assessment. This study evaluates the type and timing of speech outcome measures currently included in this ICHOM Standard Set. Additionally, speech assessments in other cleft protocols and initiatives are discussed. DESIGN, SETTING AND PARTICIPANTS: An international, multicentre study was set up including centres from the USA and the Netherlands. Outcomes of clinical measures and Patient Reported Outcome Measures (PROMs) were collected retrospectively according to the ICHOM set. PROM data from a field test of the CLEFT-Q, a questionnaire developed and validated for patients with a cleft, were collected, including participants from countries with all sorts of income statuses, to examine the value of additional moments of measurement that are used in other cleft initiatives.Data from 2500 patients were included. Measured outcomes contained univariate regression analyses, trend analyses, t-tests, correlations and floor and ceiling effects. RESULTS: PROMs correlated low to moderate with clinical outcome measures. Clinical outcome measures correlated low to moderate with each other too. In contrast, two CLEFT-Q Scales correlated strongly with each other. All PROMs and the Percent Consonants Correct (PCC) showed an effect of age. In patients with an isolated cleft palate, a ceiling effect was found in the Intelligibility in Context Scale. CONCLUSION: Recommendations for an optimal speech outcome assessment in cleft patients are made. Measurement moments of different cleft protocols and initiatives are considered in this proposition. Concerning the type of measures, adjustment of the current PCC score outcome seems appropriate. For centres with adequate resources and specific interest in research, translation and validation of an upcoming tool, the Cleft Audit Protocol for Speech Augmented, is recommended.</p

The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test:A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery

BackgroundRoutine use of patient-reported outcome measures (PROMs) and computerized adaptive tests (CATs) may improve care in a range of surgical conditions. However, most available CATs are neither condition-specific nor coproduced with patients and lack clinically relevant score interpretation. Recently, a PROM called the CLEFT-Q has been developed for use in the treatment of cleft lip or palate (CL/P), but the assessment burden may be limiting its uptake into clinical practice. ObjectiveWe aimed to develop a CAT for the CLEFT-Q, which could facilitate the uptake of the CLEFT-Q PROM internationally. We aimed to conduct this work with a novel patient-centered approach and make source code available as an open-source framework for CAT development in other surgical conditions. MethodsCATs were developed with the Rasch measurement theory, using full-length CLEFT-Q responses collected during the CLEFT-Q field test (this included 2434 patients across 12 countries). These algorithms were validated in Monte Carlo simulations involving full-length CLEFT-Q responses collected from 536 patients. In these simulations, the CAT algorithms approximated full-length CLEFT-Q scores iteratively, using progressively fewer items from the full-length PROM. Agreement between full-length CLEFT-Q score and CAT score at different assessment lengths was measured using the Pearson correlation coefficient, root-mean-square error (RMSE), and 95% limits of agreement. CAT settings, including the number of items to be included in the final assessments, were determined in a multistakeholder workshop that included patients and health care professionals. A user interface was developed for the platform, and it was prospectively piloted in the United Kingdom and the Netherlands. Interviews were conducted with 6 patients and 4 clinicians to explore end-user experience. ResultsThe length of all 8 CLEFT-Q scales in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set combined was reduced from 76 to 59 items, and at this length, CAT assessments reproduced full-length CLEFT-Q scores accurately (with correlations between full-length CLEFT-Q score and CAT score exceeding 0.97, and the RMSE ranging from 2 to 5 out of 100). Workshop stakeholders considered this the optimal balance between accuracy and assessment burden. The platform was perceived to improve clinical communication and facilitate shared decision-making. ConclusionsOur platform is likely to facilitate routine CLEFT-Q uptake, and this may have a positive impact on clinical care. Our free source code enables other researchers to rapidly and economically reproduce this work for other PROMs

Rasch Analysis of Patient- and Parent-Reported Outcome Measures in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set for Cleft Lip and Palate

Objectives: The aim of this study was to evaluate the psychometric performance of the patient- and parent-reported measures in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set for Cleft Care, and to identify ways of improving concept coverage. Methods: Data from 714 patients with cleft lip and/or palate, aged 8 to 9, 10 to 12.5, and 22 years were collected between November 2015 and April 2019 at Erasmus University Medical Center, Boston Children's Hospital, Duke Children's Hospital, and from participating sites in the CLEFT-Q Phase 3 study. The Standard Set includes 9 CLEFT-Q scales, the Nasal Obstruction Symptom Evaluation (NOSE) questionnaire, the Child Oral Health Impact Profile–Oral Symptoms Scale (COHIP-OSS), and the Intelligibility in Context Scale (ICS). Targeting, item-fit statistics, thresholds for item responses, and measurement precision (PSI) were analyzed using Rasch measurement theory. Results: The proportion of the sample to score within each instruments range of measurement varied from 69% (ICS) to 92% (CLEFT-Q teeth and COHIP-OSS). Specific problems with individual items within the NOSE and COHIP-OSS questionnaires were noted, such as poor item fit to the Rasch model and disordered thresholds (6 of 10). Reliability measured with PSI was above 0.82 for the ICS and all but one CLEFT-Q scale (speech distress). PSIs were lowest for the COHIP-OSS (0.43) and NOSE questionnaire (0.35). Conclusion: The patient- and parent-reported components within the facial appearance, psychosocial function, and speech domains are valid measures; however, the facial function and oral health domains are not sufficiently covered by the CLEFT-Q eating and drinking, NOSE, and COHIP-OSS, and these questionnaires may not be accurate enough to stratify cleft-related outcomes

Collection of Bilateral Cleft Lip and Palate Standard Set Variables: Establishing a Baseline

Background: The International Consortium for Healthcare Outcomes Measurement recently published a consensus Standard Set of clinical and patient-centered metrics to measure outcomes for patients with cleft lip and/or palate (CLP). This study aims to evaluate how the Standard Set compares to existing data collected to anticipate the impact that the Standard Set may have on quality and quantity of outcome data. Methods: Extraction of the Standard Set data points was attempted retrospectively for all nonsyndromic patients with bilateral cleft lip and/or palate who underwent primary lip and/or palate repair by a single surgeon (JGM) between June 2007 and June 2014. Results: Bilateral cleft lip repair was performed on 32 patients of which 29 also underwent palate repair. All but one of the baseline demographic and phenotypic variables were available. All perioperative variables were collected, but data quality was heterogeneous. There were no early complications. At 5 years, 29.6% of patients were lost to follow-up; however, a degree of data was available on 11 of the 12 clinical metrics for those remaining. Of patients with Veau IV cleft palate and follow-up at age 5, 1 patient (6.7%) had an oronasal fistula and 1 had velopharyngeal incompetence requiring Furlow palatoplasty (6.7%). No patient-reported data were collected for any time point. Conclusion: Prospective collection of the International Consortium for Healthcare Outcomes Measurement Standard Set will improve consistency of clinical data and add the patient perspective currently lacking in outcome measures collected for patients with bilateral cleft