The Road Ahead for State Assessments

The adoption of the Common Core State Standards offers an opportunity to make significant improvements to the large-scale statewide student assessments that exist today, and the two US DOE-funded assessment consortia -- the Partnership for the Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC) -- are making big strides forward. But to take full advantage of this opportunity the states must focus squarely on making assessments both fair and accurate.A new report commissioned by the Rennie Center for Education Research & Policy and Policy Analysis for California Education (PACE), The Road Ahead for State Assessments, offers a blueprint for strengthening assessment policy, pointing out how new technologies are opening up new possibilities for fairer, more accurate evaluations of what students know and are able to do. Not all of the promises can yet be delivered, but the report provides a clear set of assessment-policy recommendations. The Road Ahead for State Assessments includes three papers on assessment policy.The first, by Mark Reckase of Michigan State University, provides an overview of computer adaptive assessment. Computer adaptive assessment is an established technology that offers detailed information on where students are on a learning continuum rather than a summary judgment about whether or not they have reached an arbitrary standard of "proficiency" or "readiness." Computer adaptivity will support the fair and accurate assessment of English learners (ELs) and lead to a serious engagement with the multiple dimensions of "readiness" for college and careers.The second and third papers give specific attention to two areas in which we know that current assessments are inadequate: assessments in science and assessments for English learners.In science, paper-and-pencil, multiple choice tests provide only weak and superficial information about students' knowledge and skills -- most specifically about their abilities to think scientifically and actually do science. In their paper, Chris Dede and Jody Clarke-Midura of Harvard University illustrate the potential for richer, more authentic assessments of students' scientific understanding with a case study of a virtual performance assessment now under development at Harvard. With regard to English learners, administering tests in English to students who are learning the language, or to speakers of non-standard dialects, inevitably confounds students' content knowledge with their fluency in Standard English, to the detriment of many students. In his paper, Robert Linquanti of WestEd reviews key problems in the assessment of ELs, and identifies the essential features of an assessment system equipped to provide fair and accurate measures of their academic performance.The report's contributors offer deeply informed recommendations for assessment policy, but three are especially urgent.Build a system that ensures continued development and increased reliance on computer adaptive testing. Computer adaptive assessment provides the essential foundation for a system that can produce fair and accurate measurement of English learners' knowledge and of all students' knowledge and skills in science and other subjects. Developing computer adaptive assessments is a necessary intermediate step toward a system that makes assessment more authentic by tightly linking its tasks and instructional activities and ultimately embedding assessment in instruction. It is vital for both consortia to keep these goals in mind, even in light of current technological and resource constraints.Integrate the development of new assessments with assessments of English language proficiency (ELP). The next generation of ELP assessments should take into consideration an English learners' specific level of proficiency in English. They will need to be based on ELP standards that sufficiently specify the target academic language competencies that English learners need to progress in and gain mastery of the Common Core Standards. One of the report's authors, Robert Linquanti, states: "Acknowledging and overcoming the challenges involved in fairly and accurately assessing ELs is integral and not peripheral to the task of developing an assessment system that serves all students well. Treating the assessment of ELs as a separate problem -- or, worse yet, as one that can be left for later -- calls into question the basic legitimacy of assessment systems that drive high-stakes decisions about students, teachers, and schools." Include virtual performance assessments as part of comprehensive state assessment systems. Virtual performance assessments have considerable promise for measuring students' inquiry and problem-solving skills in science and in other subject areas, because authentic assessment can be closely tied to or even embedded in instruction. The simulation of authentic practices in settings similar to the real world opens the way to assessment of students' deeper learning and their mastery of 21st century skills across the curriculum. We are just setting out on the road toward assessments that ensure fair and accurate measurement of performance for all students, and support for sustained improvements in teaching and learning. Developing assessments that realize these goals will take time, resources and long-term policy commitment. PARCC and SBAC are taking the essential first steps down a long road, and new technologies have begun to illuminate what's possible. This report seeks to keep policymakers' attention focused on the road ahead, to ensure that the choices they make now move us further toward the goal of college and career success for all students. This publication was released at an event on May 16, 2011

System upgrade: realising the vision for UK education

A report summarising the findings of the TEL programme in the wider context of technology-enhanced learning and offering recommendations for future strategy in the area was launched on 13th June at the House of Lords to a group of policymakers, technologists and practitioners chaired by Lord Knight. The report – a major outcome of the programme – is written by TEL director Professor Richard Noss and a team of experts in various fields of technology-enhanced learning. The report features the programme’s 12 recommendations for using technology-enhanced learning to upgrade UK education

Shipboard Crisis Management: A Case Study.

The loss of the "Green Lily" in 1997 is used as a case study to highlight the characteristics of escalating crises. As in similar safety critical industries, these situations are unpredictable events that may require co-ordinated but flexible and creative responses from individuals and teams working in stressful conditions. Fundamental skill requirements for crisis management are situational awareness and decision making. This paper reviews the naturalistic decision making (NDM) model for insights into the nature of these skills and considers the optimal training regimes to cultivate them. The paper concludes with a review of the issues regarding the assessment of crisis management skills and current research into the determination of behavioural markers for measuring competence

Monitoring What Matters About Context and Instruction in Science Education: A NAEP Data Analysis Report

This report explores background variables in the National Assessment of Educational Progress (NAEP) to examine key context and instructional factors behind science learning for eighth grade students. Science education is examined from five perspectives: student engagement in science, science teachers' credentials and professional development, availability and use of science resources, approaches to science instruction, and methods and uses of science assessment

Using evaluation to inform the development of a user-focused assessment engine

This paper reports on the evaluation of a new assessment system, Technologies for Online Interoperability (TOIA). TOIA was built from a user-focussed specification of an assessment system. The formative evaluation of the project complemented this initial specification by ensuring that user feedback on the development and use of the system was iteratively fed back into the development process. The paper begins by summarising some of the key barriers and enablers to the use of assessment systems and the uptake of Computer-Assisted Assessment (CAA). It goes on to provide a critique of the impact of technology on assessment and considers whether innovative uses of information and communication technology (ICT) might result in new e-pedagogies and practices in assessment. The paper then reports on the findings of the TOIA evaluation and discusses how these were used to inform the development of the system

A review of ten years of implementation and research in aligning learning design with learning analytics at the Open University UK

There is an increased recognition that learning design drives both student learning experience and quality enhancements of teaching and learning. The Open University UK (OU) has been one of few institutions that have explicitly and systematically captured the designs for learning at a large scale. By applying advanced analytical techniques on large and fine-grained datasets, the OU has been unpacking the complexity of instructional practices, as well as providing conceptual and empirical evidence of how learning design influences student behaviour, satisfaction, and performance. This study discusses the implementation of learning design at the OU in the last ten years, and critically reviews empirical evidence from eight recent large-scale studies that have linked learning design with learning analytics. Four future research themes are identified to support future adoptions of learning design approaches

Improving Student Learning Through Strategic Compensation

Reviews the debate over teachers' pay; outlines the TeacherSolutions compensation reform model, developed by teachers in dialogue with experts and activists; and presents the Teacher Leaders Network Kansas' discussions of the reforms and their challenges

Integrating Technology With Student-Centered Learning

Reviews research on technology's role in personalizing learning, its integration into curriculum-based and school- or district-wide initiatives, and the potential of emerging digital technologies to expand student-centered learning. Outlines implications

Development of Self-Regulated Learning Skills Within Open-Ended Computer-Based Learning Environments for Science

Over the past decade, open-ended computer-based learning environments have been increasingly used to facilitate students’ learning of complex scientific topics. The non-linearity and open-endedness of these environments create learning opportunities for students, but can also impose challenges in terms of extraneous cognitive load and greater requirements for self-regulated learning (SRL). SRL is crucial for academic success in various educational settings. This dissertation explores how self-regulatory skills develop and the role of gender in the development of SRL skills in Virtual Performance Assessments (VPA), an immersive, open-ended virtual environment designed to assess middle school students’ science inquiry skills. Findings from three analyses combining educational data mining techniques with multilevel modeling indicated that students developed self-regulatory behaviors and strategies as they used VPA. For example, experience with VPA prepared students to adopt more efficient note-taking and note-reviewing strategies. Students who used VPA for the second time engaged in note-taking more frequently, noted a significantly higher quantity of unique information, used the control of variables strategy more frequently in note-taking, and reproduced more domain-specific declarative information in notes than students who used VPA for the first time, all of which have been found to be positively associated with science inquiry performance. Students also learned to exploit more available sources of information by applying learning strategies, in order to either solve inquiry problems, or to monitor and evaluate their solutions. Compared to the second-time users who focused primarily on answering the core inquiry question and selectively collected data, the first-time users’ behaviors showed the repetition and combination of exploratory actions such as talking with non-player characters and collecting data. In addition, consistent gender differences in SRL were observed in this study. Female students were more likely to take notes than male students; they took notes and reviewed notes more frequently and recorded a higher quantity of information in notes, especially information from the research kiosk. Females were also more likely to review notes or read research pages to assist them with the problem-solving and decision-making process than their male counterparts. Possibly due to the higher quantity of information recorded by female note-takers and their tendency to review notes over males, female students’ performance on science inquiry tasks improved across the course of using the two scenarios of VPA, whereas the male students’ science inquiry skills did not show improvement. Results from this dissertation study provide insights into the instructional design of personalized open-ended learning environments to facilitate self-regulated learning for both male and female students