An Exploration of Alternative Scoring Methods Using Curriculum-Based Measurement in Early Writing

This manuscript describes two empirical studies of alternative scoring procedures used with curriculum-based measurement in writing (CBM-W). Study 1 explored the technical adequacy of a trait-based rubric in first grade. Study 2 explored the technical adequacy of a trait-based rubric, production-dependent, and production-independent scores in third grade. Results of Study 1 suggest that the rubric holds promise as a valid measure of sentence writing ability in first grade and has utility as a supplemental scoring procedure when using CBM-W as a screening tool. Results of Study 2 show that correct word sequences maintained the highest correlation coefficients across time with the trait-based rubric, but the other scoring procedures might offer promise as reliable alternative scoring methods. However, high internal correlations among the text features of the rubric along with highly variable interrater reliability suggest that caution must be taken in interpreting results

Scoring measures of word dictation curriculum-based measurement in writing: Effects of incremental administration

Spelling has been identified as a key transcription skill that emerges during the elementary years as students learn how to write and subsequently develop fluency with writing, making the assessment of spelling a critical component of evaluation systems within schools. This includes the use of curriculum-based measures of writing (CBM-W). This study examined the extent to which word dictation CBM-W administered during the Fall, Winter, and Spring of an academic year maintained technical adequacy across 1-min time intervals in grades 1–3. Results revealed moderate predictive and concurrent validity estimates with the Spelling subtest of the Weschler Individual Achievement Test-III. Statistically significant differences existed between and within grade levels across each minute of administration and across Fall, Winter, and Spring time points for all scoring procedures

Chapter 11: Teaching mathematics to students with disabilities from diverse backgrounds

According to the Organization for Economic Cooperation and Development (OECD), the literacy and numeracy skills that are distributed across a population will have significant impact on economic and social outcomes. In particular, the “higher the levels of inequality in literacy and numeracy skills … the greater the inequality of distribution of income” (OECD, 2013, p. 26). However, although literacy skills are important, it has been suggested that poor mathematical skills may be more of a handicap than poor literacy skills, especially in the workplace (Butterworth, 2005; McCloskey, 2007). As McCloskey (2007) notes, “… quantitative concepts and information are involved in many facets of home, work and community life …” (p. 421). Poor quantitative skills are likely to pose significant problems in everyday life, even into adulthood (Dougherty, 2003; McCloskey, 2007)

Using Data-Based Instruction to Support Struggling Elementary Writers

Writing is a critical academic and life skill, but many school-age children struggle with the complexity of written expression. Given the importance of writing, there is a clear need for a systematic approach to identifying and supporting struggling writers, including writers with learning and emotional disabilities. One such approach is known as data-based instruction (DBI). This article presents an overview of DBI and guidance on how educators can use the DBI steps with assessment data to inform their classroom writing instruction. Additional resources are shared to support teachers in using DBI with their struggling writers and writers with learning and emotional disabilities

Exploration of New Complexity Metrics for Curriculum-Based Measures of Writing

Researchers and practitioners have questioned whether scoring procedures used with curriculum-based measures of writing (CBM-W) capture growth in complexity of writing. We analyzed data from six independent samples to examine two potential scoring metrics for picture word CBM-W (PW), a sentence-level CBM task. Correct word sequences per response (CWSR) and words written per response (WWR) were compared with the current standard metric of correct word sequences (CWS). Linear regression analyses indicated that CWSR predicted scores on standardized norm-referenced criterion measures in more samples than did WWR or CWS. Future studies should explore the capacity of CWSR and WWR to show growth over time, stability, diagnostic accuracy, and utility for instructional decision making

Getting More From Your Maze: Examining Differences in Distractors

The present study examined the technical adequacy of maze-selection tasks constructed in 2 different ways: typical versus novel. We selected distractors for each measure systematically based on rules related to the content of the passage and the part of speech of the correct choice. Participants included 262 middle school students who were randomly assigned to 1 of the 2 maze formats. Scoring of the maze included both correct and correct-minus-incorrect scores. Students completed 3 criterion-reading tests: the Scholastic Reading Inventory, the AIMSweb R-Maze, and a high-stakes state assessment (the Missouri Assessment Program). Alternate-forms reliability was similar across maze formats; however, with regard to scoring procedure, reliability coefficients were consistently higher for correct than for correct-minus-incorrect scores. Validity coefficients were also similar across format with 1 exception: The coefficients for typical maze scores were stronger when compared with the Missouri Assessment Program scores than the coefficients for novel maze scores

Technical Adequacy of Curriculum-Based Measures in Writing in Grades 1–3

The purpose of this study was to investigate evidence of reliability, criterion validity, and grade-level differences of curriculum-based measures of writing (CBM-W) with 612 students in grades 1–3. Four scoring procedures (words written, words spelled correctly, correct word sequences, and correct minus incorrect word sequences) were used with two CBM-W tasks (picture–word and story prompt) during fall, winter, and spring of one academic year. A subsample of participants (n = 244) were given a criterion measure in spring of the academic year. Pearson’s r coefficients were calculated to determine evidence of alternate form reliability and criterion validity, and a MANOVA was used to detect significant growth within and across grade levels. Results indicated that scores on both CBM-W tasks had adequate reliability and validity coefficients in grades 2–3 and mixed results in grade 1. Significant growth was detected within and across all grades at each time point on each task. Implications for research and practice are discussed

Supporting teachers’ use of data-based instruction to improve students’ early writing skills

We examined the effects of a professional development (PD) system designed to support teachers’ use of data-based instruction (DBI) to improve early writing outcomes for children with intensive needs. The PD system, called DBI-TLC, provided tools for early writing assessment and intervention, learning modules including face-to-face workshops followed by classroom application, and ongoing coaching to support DBI implementation. Special education teachers in 19 classrooms in 2 Midwestern districts were assigned randomly to receive DBI-TLC or to a business-as-usual control group. All teachers completed pre- and posttests of DBI knowledge and skills and self-efficacy, and DBI-TLC teachers’ fidelity to DBI was assessed. Fifty-three students (2 to 3 from each classroom) completed pre- and posttests of early writing using curriculum-based measures (CBM) and the Test of Early Written Language-3 (TEWL-3). DBI-TLC teachers outperformed controls at posttest on DBI knowledge and skills (Hedge’s g = 2.88) and reported a more explicit writing instruction orientation compared to controls (g = 1.63). DBI fidelity varied (on average, 84% for assessment, 79% for intervention, and 52% for decision-making). Students whose teachers implemented DBI showed a pattern of stronger early writing performance compared to control students on CBM, with effect sizes of 0.23 to 0.40, but not on the TEWL-3 (0.02 to 0.13). We discuss the promise of DBI-TLC to improve teacher practice and student outcomes, as well as the need to continue to explore ways to support teachers’ implementation of DBI with fidelity. (PsycINFO Database Record (c) 2019 APA, all rights reserved