The power of PISA – limitations and possibilities for educational research

On 6th December 2016, the Programme for International Student Assessment (PISA) releases its report on the achievements of 15-year-olds from 72 countries and economies around the world. This triennial international survey aims to evaluate education systems across 72 contexts by testing skills in Mathematics, Science and Reading Literacy. This is the sixth cycle of PISA and the OECD suggests countries and economies now have the capability to compare the results over time to ‘assess the impact of education policy decisions’ . Compared to other education studies, the media coverage of PISA must be described as massive (Meyer and Benavot, 2013, Baird et al., 2016) and, as with previous years, it is expected that PISA will attract considerable discussion among policy makers, educators and researchers (Wiseman, 2014). It is therefore timely to present a thematic issue of Assessment in Education, where we publish four articles that have analysed previous datasets from the PISA studies each commenting upon the challenges, limitations and potential future assessment research on the PISA data. The articles touch upon issues regarding sampling, language, item difficulty and demands, as well as the secondary analyses of students’ reported experiences of formative assessment in the classroom. One important message from the authors in this thematic Special Issue is the need for a more complex discussion around the use and misuse of PISA data, and the importance of pointing to the limitations of how the results are presented to policy makers and the public. In an area where the media produces narratives on schools and education systems based upon rankings in PISA, researchers in the field of large-scale assessment studies have a particularly important role in stepping up and advising on how to interpret and understand these studies, while warning against potential misuse. </p

Never Mind the Gap: Formative Assessment Confronted with Dewey’s and Gadamer’s Concept of Experience

The notion of “closing the learning gap” is widely used in the conceptualisation of formative assessment. It builds on an unarticulated assumption that students' learning can and should be controlled towards predefined outcomes. This article discusses this control assumption in the light of the concept of the American philosopher John Dewey and the German philosopher Hans‐Georg Gadamer. Their conceptualisation challenges the idea of learning as a linear and controllable process that results in stable and predictable outcomes. Using the concept of experience, we argue that learning follows a continuous circular movement where previous experiences condition future interpretations and that every experience changes the subject. This process of change is both unpredictable and diverse and requires that attention is paid to the uniqueness of each situation and to students as subjects. Following the discussion, we propose a model for considering the extensiveness and rigidity of formative assessment practices and that authors pay attention to whether they conceptualise formative assessment in a way that promotes student and teacher “gap closing” and control.publishedVersio

‘[It] isn’t designed to be assessed how we assess’:Rethinking assessment for qualification in the context of the implementation of the Curriculum for Wales

This paper reports teacher and learner perspectives on how assessment and reform influences pedagogical practices and behaviours. The research was conducted in a context of policy reform, at a time when Wales’ revised General Certificate of Secondary Education (GCSE) specifications had been implemented, and learners were preparing for their assessments; but, also during the period of debate on the development of Wales’ new curriculum, which has taken a distinct and contrasting position on assessment to the assumptions underlying the reform of Welsh GCSEs implemented from 2015. These data, therefore, offer unique insights into the affordances and limitations of two sharply contrasting systems at a time of considerable change, offering reflections on the current curriculum and its attendant assessment practices, and also a prospective analysis of how the principles embedded in the new curriculum could challenge these existing assumptions and conventions. Findings suggest that teachers and learners currently inhabit an assessment‐driven system, which encourages performative practices in pedagogy and is governed by external accountability; and that these practices are at odds with the principles of assessment articulated in Successful Futures. Consequently, teachers in this study expressed uncertainty about how assessment for certification purposes at GCSE could be compatible with the principles of the Curriculum for Wales

Students’ test motivation in PISA: the case of Norway

Do students make their best effort in large-scale assessment studies such as the Programme for International Student Assessment (PISA)? Despite six cycles of PISA surveys from 2000 to 2015, empirical studies regarding students’ test motivation and experience of the tests are sparse. The present study examines students’ test motivation in PISA, and how closely students’ achievement correlates with students’ reported test motivation. A total of 40 students from eight schools that participated in the PISA study in 2006, 2009 and 2012 were interviewed. In addition, questionnaire data from a total of 9400 students who participated in PISA 2009 and 2012 were collected. The findings of this study indicate that students overall were motivated to do their best in the PISA study, despite the fact that these low-stakes tests have no impact on students’ grades or future school entrance. In contrast to claims in the media, where it has been suggested that Norwegian students are more relaxed towards the PISA test than students in other countries, students report that they are motivated to do their best, but girls report significantly higher test motivation than boys. Despite the policy influence of PISA and the large number of countries participating, very few studies have reported findings from students’ perspectives regarding their test motivation and experience of the test. The present study aims to fill that gap