An accurate and efficient identification of children with psychosocial problems by means of computerized adaptive testing

Background: Questionnaires used by health services to identify children with psychosocial problems are often rather short. The psychometric properties of such short questionnaires are mostly less than needed for an accurate distinction between children with and without problems. We aimed to assess whether a short Computerized Adaptive Test (CAT) can overcome the weaknesses of short written questionnaires when identifying children with psychosocial problems. Method: We used a Dutch national data set obtained from parents of children invited for a routine health examination by Preventive Child Healthcare with 205 items on behavioral and emotional problems (n = 2,041, response 84%). In a random subsample we determined which items met the requirements of an Item Response Theory (IRT) model to a sufficient degree. Using those items, item parameters necessary for a CAT were calculated and a cut-off point was defined. In the remaining subsample we determined the validity and efficiency of a Computerized Adaptive Test using simulation techniques, with current treatment status and a clinical score on the Total Problem Scale (TPS) of the Child Behavior Checklist as criteria. Results: Out of 205 items available 190 sufficiently met the criteria of the underlying IRT model. For 90% of the children a score above or below cut-off point could be determined with 95% accuracy. The mean number of items needed to achieve this was 12. Sensitivity and specificity with the TPS as a criterion were 0.89 and 0.91, respectively. Conclusion: An IRT-based CAT is a very promising option for the identification of psychosocial problems in children, as it can lead to an efficient, yet high-quality identification. The results of our simulation study need to be replicated in a real-life administration of this CAT

Comparing three short questionnaires to detect psychosocial dysfunction among primary school children: a randomized method

BACKGROUND: Good questionnaires are essential to support the early identification of children with psychosocial dysfunction in community based settings. Our aim was to assess which of three short questionnaires was most suitable for this identification among school-aged children METHODS: A community-based sample of 2,066 parents of children aged 7-12 years (85% of those eligible) filled out the Child Behavior Checklist (CBCL) and - randomly determined - one of three questionnaires to be compared: the Strengths and Difficulties Questionnaire with Impact Supplement (SDQ), the Pediatric Symptom Checklist (PSC) and the PSYBOBA, a Dutch-origin questionnaire. Preventive Child Healthcare professionals assessed children's psychosocial functioning during routine health examinations. We assessed the scale structure (by means of Structural Equation Modelling), validity (correlation coefficients, sensitivity and specificity) and usability (ratings by parents and professionals) of each questionnaire and the degree to which they could improve the identification based only on clinical assessment (logistic regression). RESULTS: For the three questionnaires, Cronbach's alphas varied between 0.80 and 0.89. Sensitivities for a clinical CBCL at a cut off point with specificity = 0.90 varied between 0.78 and 0.86 for the three questionnaires. Areas under the Receiver Operating Curve, using the CBCL as criterion, varied between 0.93 and 0.96. No differences were statistically significant. All three questionnaires added information to the clinical assessment. Odds ratios (95% confidence intervals) for added information were PSC: 29.3 (14.4-59.8), SDQ: 55.0 (23.1-131.2) and PSYBOBA: 68.5 (28.3-165.6). Parents preferred the SDQ and PSYBOBA. Preventive Child Health Care professionals preferred the SDQ. CONCLUSIONS: This randomized comparison of three questionnaires shows that each of the three questionnaires can improve the detection of psychosocial dysfunction among children substantially