Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

The Road Ahead for State Assessments

The adoption of the Common Core State Standards offers an opportunity to make significant improvements to the large-scale statewide student assessments that exist today, and the two US DOE-funded assessment consortia -- the Partnership for the Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC) -- are making big strides forward. But to take full advantage of this opportunity the states must focus squarely on making assessments both fair and accurate.A new report commissioned by the Rennie Center for Education Research & Policy and Policy Analysis for California Education (PACE), The Road Ahead for State Assessments, offers a blueprint for strengthening assessment policy, pointing out how new technologies are opening up new possibilities for fairer, more accurate evaluations of what students know and are able to do. Not all of the promises can yet be delivered, but the report provides a clear set of assessment-policy recommendations. The Road Ahead for State Assessments includes three papers on assessment policy.The first, by Mark Reckase of Michigan State University, provides an overview of computer adaptive assessment. Computer adaptive assessment is an established technology that offers detailed information on where students are on a learning continuum rather than a summary judgment about whether or not they have reached an arbitrary standard of "proficiency" or "readiness." Computer adaptivity will support the fair and accurate assessment of English learners (ELs) and lead to a serious engagement with the multiple dimensions of "readiness" for college and careers.The second and third papers give specific attention to two areas in which we know that current assessments are inadequate: assessments in science and assessments for English learners.In science, paper-and-pencil, multiple choice tests provide only weak and superficial information about students' knowledge and skills -- most specifically about their abilities to think scientifically and actually do science. In their paper, Chris Dede and Jody Clarke-Midura of Harvard University illustrate the potential for richer, more authentic assessments of students' scientific understanding with a case study of a virtual performance assessment now under development at Harvard. With regard to English learners, administering tests in English to students who are learning the language, or to speakers of non-standard dialects, inevitably confounds students' content knowledge with their fluency in Standard English, to the detriment of many students. In his paper, Robert Linquanti of WestEd reviews key problems in the assessment of ELs, and identifies the essential features of an assessment system equipped to provide fair and accurate measures of their academic performance.The report's contributors offer deeply informed recommendations for assessment policy, but three are especially urgent.Build a system that ensures continued development and increased reliance on computer adaptive testing. Computer adaptive assessment provides the essential foundation for a system that can produce fair and accurate measurement of English learners' knowledge and of all students' knowledge and skills in science and other subjects. Developing computer adaptive assessments is a necessary intermediate step toward a system that makes assessment more authentic by tightly linking its tasks and instructional activities and ultimately embedding assessment in instruction. It is vital for both consortia to keep these goals in mind, even in light of current technological and resource constraints.Integrate the development of new assessments with assessments of English language proficiency (ELP). The next generation of ELP assessments should take into consideration an English learners' specific level of proficiency in English. They will need to be based on ELP standards that sufficiently specify the target academic language competencies that English learners need to progress in and gain mastery of the Common Core Standards. One of the report's authors, Robert Linquanti, states: "Acknowledging and overcoming the challenges involved in fairly and accurately assessing ELs is integral and not peripheral to the task of developing an assessment system that serves all students well. Treating the assessment of ELs as a separate problem -- or, worse yet, as one that can be left for later -- calls into question the basic legitimacy of assessment systems that drive high-stakes decisions about students, teachers, and schools." Include virtual performance assessments as part of comprehensive state assessment systems. Virtual performance assessments have considerable promise for measuring students' inquiry and problem-solving skills in science and in other subject areas, because authentic assessment can be closely tied to or even embedded in instruction. The simulation of authentic practices in settings similar to the real world opens the way to assessment of students' deeper learning and their mastery of 21st century skills across the curriculum. We are just setting out on the road toward assessments that ensure fair and accurate measurement of performance for all students, and support for sustained improvements in teaching and learning. Developing assessments that realize these goals will take time, resources and long-term policy commitment. PARCC and SBAC are taking the essential first steps down a long road, and new technologies have begun to illuminate what's possible. This report seeks to keep policymakers' attention focused on the road ahead, to ensure that the choices they make now move us further toward the goal of college and career success for all students. This publication was released at an event on May 16, 2011

Working memory in children with developmental disorders

The aim of the present study was to directly compare working memory skills across students with different developmental disorders to investigate whether the uniqueness of their diagnosis would impact memory skills. The authors report findings confirming differential memory profiles on the basis of the following developmental disorders: Specific Language Impairment, Developmental Coordination Disorder (DCD), Attention-Deficit/Hyperactivity Disorder, and Asperger syndrome(AS). Specifically, language impairments were associated with selective deficits in verbal short-term and working memory, whereas motor impairments (DCD) were associated with selective deficits in visuospatial short-term and working memory. Children with attention problems were impaired in working memory in both verbal and visuospatial domains, whereas the children with AS had deficits in verbal short-term memory but not in any other memory component. The implications of these findings are discussed in light of support for learning

Display Enhanced Testing For Concussions And Mild Traumatic Brain Injury

Cognitive assessment systems and methods that provide an integrated solution for evaluating the presence or absence of cognitive impairment. The present invention is used to test cognitive functions of an individual including information processing speed, working memory, work list learning and recall, along with variations of these tasks. Immersive and non-immersive systems and methods are disclosed. Testing and results feedback using the present invention may be completed in real time, typically in less than 15 minutes.Emory UniversityGeorgia Tech Research Corporatio

Distinguishing schemes and tasks in children's development of multiplicative reasoning

We present a synthesis of findings from constructivist teaching experiments regarding six schemes children construct for reasoning multiplicatively and tasks to promote them. We provide a task-generating platform game, depictions of each scheme, and supporting tasks. Tasks must be distinguished from children’s thinking, and learning situations must be organized to (a) build on children’s available schemes, (b) promote the next scheme in the sequence, and (c) link to intended mathematical concepts

Mitigating risk in computerized bureaucracy

This paper presents an important aspect of the pragmatic dimensions of mitigating the risks that stem from computerized bureaucracy, and thereby, preserving the organizational integrity of a firm. A case study is used to provide valuable insights into the mechanics of such mitigation. The case refers to the problematic implementation and use of a computerized reservation system in a large budget hotel in London, United Kingdom. Following the empirical findings, Ciborra’s notions of bricolage, improvisation and tinkering are examined as practical and useful ways of addressing the downsides of computerized bureaucracy

Adaptive Guidance: Enhancing Self-Regulation, Knowledge, and Performance in Technology-Based Training

Considerable research has examined the effects of giving trainees control over their learning (Steinberg, 1977, 1989; Williams, 1993). The most consistent finding of this research has been that trainees do not make good instructional use of the control they are given. Yet, today’s technologically based training systems often provide individuals with significant control over their learning (Brown, 2001). This creates a dilemma that must be addressed if technology is going to be used to create more effective training systems. The current study extended past research that has examined the effects of providing trainees with some form of advisement or guidance in addition to learner control and examined the impact of an instructional strategy, adaptive guidance, on learning and performance in a complex training environment. Overall, it was found that adaptive guidance had a substantial effect on the nature of trainees’ study and practice, self-regulation, knowledge acquired, and performance

A Perspective on the Potential Role of Neuroscience in the Court

This Article presents some lessons learned while offering expert testimony on neuroscience in courts. As a biomedical investigator participating in cutting-edge research with clinical and mentoring responsibilities, Dr. Ruben Gur, Ph.D., became involved in court proceedings rather late in his career. Based on the success of Dr. Gur and other research investigators of his generation, who developed and validated advanced methods for linking brain structure and function to behavior, neuroscience findings and procedures became relevant to multiple legal issues, especially related to culpability and mitigation. Dr. Gur found himself being asked to opine in cases where he could contribute expertise on neuropsychological testing and structural and functional neuroimaging. Most of his medical-legal consulting experience has been in capital cases because of the elevated legal requirement for thorough mitigation investigations in such cases, and his limited availability due to his busy schedule as a full-time professor and research investigator who runs the Brain and Behavior Lab at the University of Pennsylvania (“Penn”). Courtroom testimony, however, has not been a topic of his research and so he has not published extensively on the issues in peer-reviewed literature