Item Ordering and Computerized Classification Tests With Cluster-Based Scoring: An Investigation of the Countdown Method

The countdown method is a well-known approach to reducing the average length of screening instruments that are presented by computer. In the countdown method, testing is terminated once the result of the screener (“positive” or “negative”) has been unambiguously determined from prior answers. Previous research has examined whether presenting dichotomously scored items in order from “least to most frequently endorsed” or “most to least frequently endorsed” is more efficient when the countdown method is used. The current study describes the Mean Score procedure, an extension of the above item ordering procedures to polytomously scored items, and evaluates its efficiency relative to the distribution of other possible item orderings in 2 real-data simulations. Both simulations involve item responses to the Posttraumatic Stress Disorder (PTSD) Checklist for DSM–5 (PCL-5). In the first simulation, items were scored polytomously, and a single cutoff point was used to determine the screening result. In the second simulation, items were converted to dichotomous scores, as well as categorized into 4 clusters; a positive result for the entire assessment was obtained if and only if a positive result was obtained for each cluster. The latter simulation also investigated the effect of reordering the clusters themselves on the efficiency of the countdown method. Results indicated that the Mean Score procedure does not necessarily produce the optimal ordering, but tends to assemble an efficient item ordering relative to the distribution of possible orderings. In the second simulation, reordering the clusters themselves affected efficiency. Future research directions are suggested

Customized computer-based administration of the PCL-5 for the efficient assessment of PTSD:A proof-of-principle study

Objective: To investigate the potential of customized computer-based testing procedures to reduce the mean test length of the Posttraumatic Stress Checklist for DSM-5 (PCL-5). Method: A retrospective analysis was conducted using responses from 942 adults who had completed the full-length (20-item) PCL-5 in the aftermath of Hurricane Sandy. The abilities of 2 testing procedures, curtailment and stochastic curtailment, to lessen the instrument's mean test length while maintaining the same result as the full-length PCL-5 ("positive" or "negative") were evaluated in a post hoc simulation. Curtailment and stochastic curtailment track a respondent's answers as she takes the instrument and stop the test if future items are unable or unlikely to change the result. The performance of each procedure was recorded under 2 scoring methods: a total-score-based method and a cluster-based method. Each procedure's sensitivity, specificity, and overall agreement with the full-length PCL-5 were computed. Results: Curtailment reduced the mean test length by 40% under the total-score-based method, and by more than 70% under the cluster-based method, while exhibiting 100% sensitivity, specificity, and overall agreement with the full-length PCL-5. Stochastic curtailment reduced the mean test length by up to 88% under the total-score-based method, and up to 84% under the cluster-based method, while always exhibiting at least 92% sensitivity and 99.8% overall agreement, as well as 100% specificity, for the full-length PCL-5. Conclusions: Curtailment and stochastic curtailment have potential to enhance the efficiency of the PCL-5 when this assessment is administered by computer. The 2 procedures should be evaluated in future prospective studies. (PsycINFO Database Recor

Cross-validation of short forms of the Screener and Opioid Assessment for Patients with Pain-Revised (SOAPP-R)

Background The Screener and Opioid Assessment for Patients with Pain-Revised (SOAPP-R) is a 24-item assessment designed to assist in the prediction of aberrant drug-related behavior (ADB) among patients with chronic pain. Recent work has created shorter versions of the SOAPP-R, including a static 12-item short form and two computer-based methods (curtailment and stochastic curtailment) that monitor assessments in progress. The purpose of this study was to cross-validate these shorter versions in two new populations. Methods This retrospective study used data from patients recruited from a hospital-based pain center (n = 84) and pain patients followed and treated at primary care centers (n = 110). Subjects had been administered the SOAPP-R and assessed for ADB. In real-data simulation, the sensitivity, specificity, and area under the curve (AUC) of each form were calculated, as was the mean test length using curtailment and stochastic curtailment. Results Curtailment reduced the number of items administered by 30% to 34% while maintaining sensitivity and specificity identical to those of the full-length SOAPP-R. Stochastic curtailment reduced the number of items administered by 45% to 63% while maintaining sensitivity and specificity within 0.03 of those of the full-length SOAPP-R. The AUC of the 12-item form was equal to that of the 24-item form in both populations. Conclusions Curtailment, stochastic curtailment, and the 12-item short form have potential to enhance the efficiency of the SOAPP-

Development of Short-Form Versions of the Screener and Opioid Assessment for Patients with Pain-Revised (SOAPP-R):A Proof-of-Principle Study

BACKGROUND: The Screener and Opioid Assessment for Patients with Pain-Revised (SOAPP-R) is a 24-item questionnaire designed to assess risk of aberrant medication-related behaviors in chronic pain patients. The introduction of short forms of the SOAPP-R may save time and increase utilization by practitioners. OBJECTIVE: To develop and evaluate candidate SOAPP-R short forms. DESIGN: Retrospective study. SETTING: Pain centers. SUBJECTS: Four hundred and twenty-eight patients with chronic noncancer pain. METHODS: Subjects had previously been administered the full-length version of the SOAPP-R and been categorized as positive or negative for aberrant medication-related behaviors via the Aberrant Drug Behavior Index (ADBI). Short forms of the SOAPP-R were developed using lasso logistic regression. Sensitivity, specificity, and area under the curve (AUC) of all forms were calculated with respect to the ADBI using the complete data set, training-test analysis, and 10-fold cross-validation. The coefficient alpha of each form was also calculated. An external set of 12 pain practitioners reviewed the forms for content. RESULTS: In the complete data set analysis, a form of 12 items exhibited sensitivity, specificity, and AUC greater than or equal to those of the full-length SOAPP-R (which were 0.74, 0.67, and 0.76, respectively). The short form had a coefficient alpha of 0.76. In the training-test analysis and 10-fold cross-validation, it exhibited an AUC value within 0.01 of that of the full-length SOAPP-R. The majority of external practitioners reported a preference for this short form. CONCLUSIONS: The 12-item version of the SOAPP-R has potential as a short risk screener and should be tested prospectively

Shortening the PHQ-9: a proof-of-principle study of utilizing Stochastic Curtailment as a method for constructing ultrashort screening instruments

Objective: In primary care, screening instruments for mental health should be ultrashort to allow for routine usage. In this paper, Stochastic Curtailment is introduced as a method for constructing ultrashort screeners. Method: In a post hoc diagnostic accuracy study using the item scores on the Patient Health Questionnaire depression module (PHQ-9) of a large sample (N= 20, 685), Stochastic Curtailment was compared with two existing ultrashort versions of PHQ-9. The first was PHQ-2 (which includes the first two items of PHQ-9), and the second was a ‘two-step’ method (only if the PHQ-2 screened positive were all nine PHQ items administered). For PHQ-2 and two-step, both cut scores 2 and 3 were evaluated. Results: PHQ-2 showed the lowest and Stochastic Curtailment the highest diagnostic accuracy with reference to the classifications based on the full PHQ-9. To do so, Stochastic Curtailment used 3.08 items on average (S.D.= 1.98), which was slightly less than two-step (M= 3.18, S.D.= 2.62) under its most accurate cut score (≥ 2). Conclusions: It was concluded that Stochastic Curtailment is a promising method for constructing ultrashort screeners. Keywords: Screening; Mental health; Respondent burden; Curtailmen

Stochastic curtailment of questionnaires for three-level classification: Shortening the CES-D for assessing low, moderate, and high risk of depression

In clinical assessment, efficient screeners are needed to ensure low respondent burden. In this article, Stochastic Curtailment (SC), a method for efficient computerized testing for classification into two classes for observable outcomes, was extended to three classes. In a post hoc simulation study using the item scores on the Center for Epidemiologic Studies-Depression Scale (CES-D) of a large sample, three versions of SC, SC via Empirical Proportions (SC-EP), SC via Simple Ordinal Regression (SC-SOR), and SC via Multiple Ordinal Regression (SC-MOR) were compared at both respondent burden and classification accuracy. All methods were applied under the regular item order of the CES-D and under an ordering that was optimal in terms of the predictive power of the items. Under the regular item ordering, the three methods were equally accurate, but SC-SOR and SC-MOR needed less items. Under the optimal ordering, additional gains in efficiency were found, but SC-MOR suffered from capitalization on chance substantially. It was concluded that SC-SOR is an efficient and accurate method for clinical screening. Strengths and weaknesses of the methods are discussed