Diagnostic Assessments in Mathematics to Support Instructional Decision Making

Diagnosis is an integral part of instructional decision-making. As the bridge between identification of students who may be at-risk for failure and delivery of carefully designed supplemental interventions, diagnosis provides valuable information about students’ persistent misconceptions in the targeted domain. In this paper, we discuss current approaches to diagnosis in mathematics and highlight the strengths and limitations of each approach for making instructional decisions. We point to cognitive diagnostic assessments as an emerging solution for providing detailed and precise information about students’ thinking that is needed to provide appropriate educational opportunities for students struggling in mathematics. Accessed 41,983 times on https://pareonline.net from October 12, 2009 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Policies and Practices in Support of Mathematics Learning in Global Contexts

This issue of Global Education Review focuses on policies and practices in support of mathematics learning in global contexts. The issue highlights relevant research findings, illuminates areas that are in need of additional research, and examines the implications of these findings on educational outcomes. The ten manuscripts in this issue focus on three primary topics: efforts to improve student achievement and factors that impact achievement; implications of assessment frameworks, scoring, and analyses; and methods to improve teacher preparation and teacher knowledge. The manuscripts are organized in this issue by these topics

Policies and Practices in Support of Mathematics Learning in Global Contexts

This issue of Global Education Review focuses on policies and practices in support of mathematics learning in global contexts. The issue highlights relevant research findings, illuminates areas that are in need of additional research, and examines the implications of these findings on educational outcomes. The ten manuscripts in this issue focus on three primary topics: efforts to improve student achievement and factors that impact achievement; implications of assessment frameworks, scoring, and analyses; and methods to improve teacher preparation and teacher knowledge. The manuscripts are organized in this issue by these topics

Capturing Children’s Mathematical Knowledge: An Assessment Framework

This paper explores an innovative assessment framework for measuring children’s formal and informal mathematical knowledge. Many existing standardized measures, such as the Early Grade Mathematics Assessment, measure children’s performance in early primary grade skills that have been identified by researchers and policy makers as foundational and predictive of later academic achievement (Platas, Ketterlin-Geller, & Sitabkhan, 2016; RTI International, 2014). However, these standardized assessments only provide information on children’s mathematical ability as it pertains to skills and concepts that are a focus of school instruction, referred to as formal mathematics. While valuable, they leave unmeasured the mathematics that children use and develop as part of their everyday life, such as the strategies they use to solve simple arithmetical problems that arise as they move through their day (Khan, 1999; Saxe, 1991; Taylor, 2009).  In this article, we draw from mixed methods studies which focus on capturing the informal mathematical skills that children develop outside of school in various contexts (Guberman, 1996; Nasir, 2000; Sitabkhan, 2009; Sitabkhan, 2015). We describe how the use of observations of children’s mathematical activities in natural settings and subsequent cognitive interviews using mathematical tasks derived from those observations can illuminate mathematical knowledge and skills that may otherwise remain hidden. We found that an assessment framework that focuses on both standardized measures of formal mathematical learning and contextualized measures of children’s everyday mathematics can provide a more complete and nuanced picture of children’s knowledge, and taken together can inform the development of curricular materials and teacher training focused on early learning

A Framework for Evaluating Stopping Rules for Fixed-Form Formative Assessments: Balancing Efficiency and Reliability

Stopping rules for fixed-form tests with graduated item difficulty are intended to stop administration of a test at the point where students are sufficiently unlikely to provide a correct response following a pattern of incorrect responses. Although widely employed in fixed-form tests in education, little research has been done to empirically evaluate the stopping rules in these tests that often have important instructional and/or placement implications for students. In this manuscript, we propose and research a framework for evaluating stopping rules with respect to two important and sometimes conflicting criteria: (1) efficiency, and (2) reliability. Using this framework, we provide an example in which we apply three increasingly complex methods for evaluating efficiency and two methods for examining reliability

Iterative Cognitive Interview Design to Uncover Children\u27s Spatial Reasoning

Cognitive interviews play an important role in articulating the intended construct of educational assessments. This paper describes the iterative development of protocols for cognitive interviews with kindergarten through second-grade children to understand how their spatial reasoning skill development aligns with intended constructs. We describe the procedures used to gather evidence of construct relevance and improved alignment to task-based interview items through multiple pilot rounds before conducting cognitive interviews. We found improved alignment and reduced construct irrelevant variance after protocol revisions

Teacher Outcomes of an Intensive STEM-Focused Professional Learning Initiative: An Examination of Their Beliefs, Practices, and Perceptions

The purpose of this paper is to describe teacher outcomes from participating in an intensive science, technology, engineering, and mathematics (STEM) professional learning (PL) initiative for middle school science teachers in the United States. The initiative included intensive summer coursework and ongoing support (e.g., individual coaching, professional learning communities), and focused on enhancing teachers’ STEM instruction and their pedagogical content knowledge (PCK), with the ultimate goal of improving student outcomes. In this mixed method study, we examine change across time in teachers’ beliefs, use of STEM instruction, and PCK. In general, we did not observe statistically significant change in teachers’ beliefs, use of STEM instruction, or PCK from the beginning to the end of Year 1. In follow-up focus groups, ten teachers described their perceptions of student outcomes of STEM instruction and identified tensions related to implementation. We offer implications intended to inform future work in the area of STEM PL

Capturing Children’s Mathematical Knowledge: An Assessment Framework

This paper explores an innovative assessment framework for measuring children’s formal and informal mathematical knowledge. Many existing standardized measures, such as the Early Grade Mathematics Assessment, measure children’s performance in early primary grade skills that have been identified by researchers and policy makers as foundational and predictive of later academic achievement (Platas, Ketterlin-Geller, & Sitabkhan, 2016; RTI International, 2014). However, these standardized assessments only provide information on children’s mathematical ability as it pertains to skills and concepts that are a focus of school instruction, referred to as formal mathematics. While valuable, they leave unmeasured the mathematics that children use and develop as part of their everyday life, such as the strategies they use to solve simple arithmetical problems that arise as they move through their day (Khan, 1999; Saxe, 1991; Taylor, 2009). In this article, we draw from mixed methods studies which focus on capturing the informal mathematical skills that children develop outside of school in various contexts (Guberman, 1996; Nasir, 2000; Sitabkhan, 2009; Sitabkhan, 2015). We describe how the use of observations of children’s mathematical activities in natural settings and subsequent cognitive interviews using mathematical tasks derived from those observations can illuminate mathematical knowledge and skills that may otherwise remain hidden. We found that an assessment framework that focuses on both standardized measures of formal mathematical learning and contextualized measures of children’s everyday mathematics can provide a more complete and nuanced picture of children’s knowledge, and taken together can inform the development of curricular materials and teacher training focused on early learning

Teaching writing to at-risk students: The quality of evidence for self regulated strategy development

ABSTRACT: Thís studj evuluates the quality of the research and evidence base for a writing intervention called Self-Regulated Strategy Development (SRSD; . Five experimental and quasi-experimental studies and 16 single-subject studies investigating SRSD were analyzed on numerous methodological dimensions. Both tbe group design and single-subject studies also met proposed standards for an evidence-based practice. Tbe potential value of analyzing approaches and interventions using tbe proposed quality indicators and standards for evidence-based practices is discussed, as are implications for research and practice. ore than any other aca-tion of skills and knowledge including organizdemie domain, writing ing information and ideas, using established offers students the oppor-writing conventions (e.g., grammar, punctuatunity to both express their tion); writing legibly; identifying and implefeelings and opinions on a menting rhetorical structures; and writing in a particular topic as well as demonstrate their way that engages a specific audience. Any of knowledge of specific content. Becoming an these elements can present challenges for typical effective writer involves developing a constella-writers, and many are poorly developed in Exceptional Children 3O3 students with learning disabilities (LD; Few educators question the value of directly teaching students to write effectively. Yet factors such as the amount of time students spend being taught systematically how to write seem to conflict with the importance educators attach to writing In the past, it was common for educators to think of writing instruction somewhat passively, consisting mainly of having students read extensively and encouraging them to apply to their own writing what they observed in the writing of others. Research on these types of exposure methods indicates that they do not help students becorrie better writers, leading to an era of advocacy for more explicit approaches CONSEQUENCES OF POOR WRITING Although the importance of fostering effective writing skills among students is unquestioned, there is clear evidence from the National Assessment of Educational Progress (NAEP) that these efforts are insufficient (Graham &C Perin, 2007). On the NAEP writing assessment for 2002, students in Grades 4, 8, and 12 wrote narrative, informative, and persuasive essays, and their performance was categorized as Below Basic, Basic, Proficient, or Advanced. Basic is defined as "partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade" (Institute of Education Sciences, 2004). In other words, students who score at Below Basic and Basic levels are not meeting minimum standards for competent writing. On the 2002 NAEP, in Grades 4, 8, and 12, 72%, 69%, and 77% of students respectively wrote at Below Basic and Basic levels On the NAEP 2007 report, which provides writing results for Grades 8 and 12 (Institute of Education Sciences, 2007), there were slight increases overall in the percentage of students in the proficient category and above, and for each demographic subgroup. However, for students with disabilities, the outcomes are troubling. Ninetyfour percent of students with disabilities scored in the Basic and Below Basic categories. In other words, only 6% of students with disabilities were considered to have proficient writing skills. In summarizing research on writing instruction for students with LD, Gersten and Baker The value of writing effectively is not confined to accomplishing academic tasks in school settings. Recent reports by the National Gommission on Writing (2004Writing ( , 2005 indicated that the majority of public and private employers state that writing proficiency is critical in the workplace and directly influences their hiring and promotion decisions. Writing is no longer a requirement limited to the daily tasks of professionals, but it is a workplace demand that extends to nearly all living-wage jobs. The lack of fundamental writing skills among new employees is such that the National Gommission on Writing estimated that 30% of employers require on-thejob training in basic writing skills. The financial cost of this workplace remediation is considerable. Private companies spend approximately $3.1 billion annually on writing remediation, and state governments spend about$221 million annually 3O4 Spring 2009 RESEARCH ON WRITING INSTRUCTION The consequences of illiteracy, including problems students experience with fundamental writing skills, has spurred research on writing instruction in K to 12 settings. Although this research base is not as extensive as the research on reading instruction, many studies have been conducted and special education researchers have played leading roles in these efforts (see The research on writing instruction that has blossomed in the last 20 years has increasingly focused on the quality of writing content rather than writing mechanics (Cersten, Baker, Pugach, Scanlon, & Chard, 2001). A catalyst for this increase of studies on writing quality was the metaanalysis on writing instruction by Since Hillocks' (1984) meta-analysis, researchers have increasingly focused on ways to improve writing content and have embedded methods within wtiting instruction across multiple stages of the writing process. Different researchers have identified different numbers of stages, but essentially the stages are: (a) instruction in planning to write (e.g., Englert et al. Recently, Craham and Perin (2007) conducted a meta-analysis on writing interventions for students in Crades 4 to 12 and found 142 studies that met their inclusion criteria. From these studies, they calculated 176 effect sizes. This magnitude of research is in sharp contrast to the 29 effect-size calculations Hillocks (1984) derived for similar types of studies targeting students in Crades 4 to 12. Based on their findings, Craham and Perin described 11 elements of effective instruction in Crades 4 to 12. Although the elements were separated in the analysis, many of the elements overlapped and included multiple stages of the wtiting process. For example, in studies on collaborative writing approaches, students are taught to work together to plan, draft, and revise their writing samples. Studies on teaching students writing strategies also focus on these stages of writing. Other studies honed in on specific stages of writing, such as approaches that teach sentence combining, whete students are taught how to write more complex sentences. Craham and Perin encouraged readers not to consider the elements "as isolated but rather as interlinked" (p. 11). It is the linkage of the elements that leads to comprehensiveness in writing instruction. Less research has been conducted on students with LD specifically. Examining primarily published studies (Craham and Perin, 2007, reviewed a much broader range of studies), Cersten and Baker (2001) conducted a meta-analysis involving 13 experimental and quasi-experimental studies of interventions designed to improve writing content in a variety of genres with students with LD. The analysis focused specifically on how the writing process was taught, and the results indicated that teaching writing strategies to students with LD could result in considerable improvements in writing quality. Although single-subject studies were not targeted in this meta-analysis formally, the informal examination of single-subject studies also supported this conclusion about the positive benefit of teaching writing strategies directly to students with LD. A comprehensive approach to writing instruction that has been used with students with and without disabilities has been developed by Craham and Harris and their colleagues (Harris 1. Students are explicitly taught background knowledge needed to use a strategy successfully. 2. The strategy-as well as its purpose and benefits-is described and discussed. 3. The teacher models how to use the strategy. 4. Students memorize the steps of the strategy and any mnemonic associated with it. 5. The teacher supports or scaffolds student mastery of the strategy. 6. Students use the strategy with few or no supports. Students are also taught a number of self-regulation skills including goal setting, self-monitoring, self-instruction, and self-reinforcement. These skills help students manage the writing strategies, the writing process, and their bebavior during instruction. Studies of SRSD in writing represent one of the most consistent efforts to explore the specific features of an instructional intervention, including systematic replications of research. Although previous reviews have been conducted on SRSD METHOD Our research team carried out its work in four phases: (a) identifying intervention studies on SRSD in writing with students with LD or at risk for LD; (b) screening the studies to ensure they met inclusion criteria; (c) development, refinement, and application of a quality indicator rubric, based on published standards, for evaluating the methodological quality of the studies; and (d) application of published quality indicators and standards to determine whether the studies were of sufficient quality to deem SRSD an evidence-based practice. Each phase is described here in detail. PHASE I-IDENTIFYING THE LITERATURE To conduct a thorough search of literature focused on SRSD in writing instruction, we completed a three-step process. 3O6 Spring 2009 Second, we conducted an ancestral search using the reference lists from three secondary sources that focused specifically on research on writing instructional interventions for students with LD or students struggling with writing. These sources included