Rusman, E., Boon, J., Martínez-Monés, A., Rodríguez-Triana, M. J., & Retalis, S. (2013, 17-21 September). Towards the Use of New Methods for Formative e-Assessment of 21st Century Skills in Schools. Presentation at the Technology Enhanced Formative Assessment (TEFA) workshop (http://www.kbs.uni-hannover.de/tefa2013.html) at  EC-TEL 2013 (http://www.ec-tel.eu/index.php), Paphos, Cyprus.Schools are in the process of reorienting their curricula to the development
of 21st century skills and competences; however, their assessment
methods have not yet been updated. The PREATY project aims to make teachers
in primary and secondary schools aware of the use and benefits of modern
e-assessment methods and tools. This presentation presents the preliminary work developed in the project.We would like to gratefully acknowledge the contribution of the
PREATY Project, that is funded by the European Commission's Lifelong
Learning Programme, Project Number 526965-LLP-1-2012-1-
GR-COMENIUS-CMP, which has part-funded this work

Boon, Jo

Martínez-Monés, Alejandra

Retalis, Simeos

Rodríguez-Triana, María-Jesús

Rusman, Ellen

Schools are in the process of reorienting their curricula to the development of 21st century skills and competences; however, their assessment methods have not yet been updated. The PREATY project aims to make teachers in primary and secondary schools aware of the use and benefits of modern e-assessment methods and tools. This paper presents the preliminary work developed in the project

Rodríguez-Triana, María Jesús

Open University of the Netherlands Research Portal

Open Universiteit www.ou.nl Towards the Use of New Methods for Formative e-Assessment of 21st Century Skills in SchoolsCitation for published version (APA):Rusman, E., Boon, J., Martínez-Monés, A., Rodríguez-Triana, M. J., & Retalis, S. (2013). Towards the Use ofNew Methods for Formative e-Assessment of 21st Century Skills in Schools. Paper presented at 8th EuropeanConference on Technology Enhanced Learning, Paphos, Cyprus.Document status and date:Published: 01/09/2013Document Version:Peer reviewed versionDocument license:CC BY-NC-NDPlease check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences betweenthe submitted version and the official published version of record. People interested in the research are advised to contact the author for thefinal version of the publication, or visit the DOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and page numbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.• You may not further distribute the material or use it for any profit-making activity or commercial gain• You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:https://www.ou.nl/taverne-agreementTake down policyIf you believe that this document breaches copyright please contact us at:pure-support@ou.nlproviding details and we will investigate your claim.Downloaded from https://research.ou.nl/ on date: 08 Sep. 2023Towards the Use of New Methods for Formative  e-Assessment of  21st Century Skills in Schools Ellen Rusman1, Jo Boon1 , Alejandra Martínez-Monés2, María Jesús Rodríguez-Triana2, Simeos Retalis3 1 Open University of the Netherlands, CELSTEC, Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands  {ellen.rusman, jo.boon}ou.nl 2 University of Valladolid, GSIC-EMIC, Paseo de Belén, 12, 47014 Valladolid, Spain amartine@infor.uva.es; chus@gsic.uva.es 3  University of Piraeus, Department of Digital Systems, 80 Karaoli & Dimitriou 185 34 Piraeus, Greece retal@unipi.gr Abstract. Schools are in the process of reorienting their curricula to the devel-opment of 21st century skills and competences; however, their assessment methods have not yet been updated. The PREATY project aims to make teach-ers in primary and secondary schools aware of the use and benefits of modern e-assessment methods and tools. This paper presents the preliminary work de-veloped in the project. Keywords. Formative assessment, e-assessment, 21st century skills, key compe-tences, primary and secondary school, teacher training 1 Introduction  There has been a recent trend across the EU towards competence-based teaching and learning [5]. Nowadays, school curricula pay special at-tention to the development of 21st century skills (e.g. creativity, critical thinking, collaboration, or problem solving skills) and key competen-cies that are seen as needed for global citizens and workers. However, in many countries, the assessment system has not changed and is still primarily focused on summative assessment of static knowledge and end-results, without attention on the learning process and the acquisi-tion of competences and skills. Though there are alternative assessment methods -more process and formative orientated such as portfolios or rubrics-, they are not wide-spread and they are often seen as additional activities rather than an integral part of the curriculum. Also ICT technologies enable alterna-tive assessment forms, also called e-assessment [6]. A number of e-assessment tools are already available, and have proved their usefulness to meet assessment needs and promote learning. However, teachers, especially in primary and secondary education, have not adopted them yet. Therefore, there is a need to change the current practices and to support teachers with integrating modern assessment approaches and tools for learning as part of the curriculum. Aware of this need, the PREATY (PRoposing modern E-assessment Approaches and Tools to Young and experienced in-service teachers) project aims to equip teachers in primary and secondary schools with e-assessment strategies and tools to evaluate a number of key 21st century skills and competencies. Opposite to the traditional test and output-focussed perspective on assessment, the project pursues to promote assessment for learning, therefore focussing mainly on the formative assessment of these skills and competences.  This paper presents initial work carried out in the project. Section 2 describes our perspective on formative assessment and the potential benefits and requirements of supportive technology. Section 3 presents the modern e-assessment instruments that will be studied, with a de-scription of their main characteristics and supporting tools. Section 4 points out the challenges of implementing ‘assessment for learning’ and formative assessment in schools. Finally, Section 5 concludes with the approach and further steps taken in the PREATY project to introduce formative e-assessment of 21st century skills in schools. 2 Formative e-assessment: Characteristics and Potential Surplus Value of Technology The purpose of formative assessment is to support teaching and learn-ing processes by providing developmental feedback (on an item, topic and/or assessment level by means of different sources, such as peers, experts or teachers) to a learner (and their teacher) on his/her current understanding or skills [3][5][1][2] during a period of instruction [11][15]. Formative assessment differs from summative assessment in that it is a continuing process of feedback; therefore the information on performances is gathered continuously, mirrored against a set of prede-fined criteria or good practices, and is used to shape improvements and promote an individual's learning, rather than serve as a final formal summary of learners’ achievement [8][17]. The responsibility for learn-ing is with both learners as teachers and eventually their peers [2][9]. They determine (jointly) where a learner is going, where (s)he is now and how a learner can get where (s)he wants [19]. Formative assess-ment strategies address and integrate both role as well as reflective questioning aspects.  Technology can offer different functions that potentially can facilitate and enhance formative assessment aspects [6][10]. It can improve ac-cess to assessments by different actors (e.g. also by peers, experts and teachers) at anytime, anyplace and anywhere, enabling learners to measure their understanding when and how often they want and allow them more control of their learning. Feedback times can be shortened and this can help to change misconceptions rapidly, or feedback may be given from different perspectives, within a group or adopted to a learn-er. Thus, technology can affect feedback quality and the motivation of a learner. Also, technology can track, trace store, process and visualize learners’ results as well as actions, which makes them visi-ble and available for various learning purposes, such as individual or group reflection or to evaluate and visualize a learners progress. Technology can also affect teacher efficiency, as teachers can be sup-ported with various tools helping to reduce assessment time, thus sav-ing time that can be spent otherwise. Also, as technology enables rapid updating and combining material, it can also contribute to more varied and authentic assessment designs. 3 Identification of Available Formative e-assessment Approaches and Tools As a first step towards the introduction of formative e-assessment in primary and secondary schools, PREATY makes an inventory of inno-vative e-assessment instruments, starting with: e-Portfolio, learning analytics and enriched rubrics. These instruments may help teachers to have a better understanding of students’ performance and offer valuable feedback to students and parents. They can support formative assess-ment methods that have proven to be effective for learning [17], such as self- or peer-assessment, instructional dialogues, questioning and re-flective lessons. This section describes the main characteristics of the mentioned e-assessment instruments. The first instrument examined is an ePortfolio. They are an electron-ic version of traditional portfolios, meaning that the work samples in-side them will be of digital nature; coming from different sources and tools [18]. The ePortfolio facilitates the capabilities of the learning portfolio to show the process and results of a learning path, proving the quality and level of achievement of the targeted competences [4]. The ePortfolio should include basic storing capabilities, with interoperabil-ity support, as well the management of permissions and mechanisms to provide feedback on the shared artefacts. Finally, different views of the work samples should be possible, in order to highlight the competences of interest in each case.  A second e-assessment instrument of interest is Learning Analytics (LA), which can be defined as the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs [16]. LA allows people who are responsible for the teaching and learning process to identify possible learning risks, and make the necessary changes, depending on the situations encountered. Many LA systems and approaches are being developed, like the ones based on Social Network Analysis. In spite of the recent interest in these tech-niques, still a major challenge is to adapt the output and visualizations to the teacher’s and learner’s needs. Thirdly, learning analytics can be combined with rubrics in the so-called enriched rubrics [12]. These instruments have the form of a traditional rubric, but (at least part of) the criteria is assessed by indica-tors automatically computed from the interaction of the students with the system. They have been tested combined with the analytical capa-bilities offered by learning management systems, such as Moodle. In spite of the potential benefits of the reviewed instruments and associated tools, their uptake by the target group is very small. Next section goes into some explanations for this lack of interest.  4 Challenges and Approaches Towards Implementing Formative e-assessment in Schools Different bottlenecks can be mentioned that prevent easy implemen-tation of formative assessment in schools and the technological support of it. As the authors in [17] mention, based on an extensive review study, still a large amount of conceptual confusion exists on the defini-tion of formative assessment, and scientific research about the condi-tions of effectiveness of formative assessment is still in an early phase of development.   Implementing formative e-assessment in schools also requires some essential preparatory steps. In the first place more information and communication about formative assessment and possible approaches herein is necessary, as they are not yet widely known and accepted.  The tools mentioned in section 3 are seen as powerful, although only partially applicable for the target groups of the PREATY project. Suc-cess – and acceptance on the longer term by teachers- depends on some other crucial conditions. Reflection about formative assessment and the goals one can achieve belongs to a school culture where teachers – and management- are willing to think about innovation and come loose of a culture where they alone are in total control [17]. Looking at the con-ceptualization of [19], described earlier in this paper, it is also clear that different target groups are involved in the implementation process, not only teachers, but also learners and their peers. Thus, implementation of formative assessment in schools requires a willingness to completely (re)think (about) assessment. A second issue at stake in the PREATY project is the introduction and training related to the technological tools and the formative e-assessment aspects. Regarding the training about technological aspects it is important to focus on acceptation by giving attention on how to present the objective of technology; are tools presented as ‘doing things better’ or as ‘ doing better things’ [14]. These distinctions are important as they include implicit views on the kind of enhancement that is at stake.  From the project results thus far it is clear that very few tools in this field are available for the target group of the project. Most tools are developed for use in higher education. Teacher professionalization in the field of formative assessment and the related technological tools implies the training of complex skills. Trainings therefore will have to include [13]: 1. information and training for in service teachers on formative as-sessment – enhancing: assessment for learning, learn how to reflect, how to use criteria, how to handle feedback, how to work in teams etc.   2. information and training on technological tools;  as very few tools are available training could be used to gather information about desirability and requirements for tools to be developed in this field. 5 Conclusions The adoption of formative assessment in schools requires a change of mind, from a focus on summative testing, end results and teacher re-sponsibility towards a process and learning oriented perspective, with a joint responsibility of learners, teachers and peers. It is important to learn to see formative assessment as a process, rather than a particular kind of assessment [9], with different formative assessment strategies that can support that process. Next to teacher training material and training strategies that help to adopt and develop this mind-set, the availability of effective (for learning) formative e-assessment methods and tools can help to facilitate adoption within schools. The vision of the PREATY project is to advance in these directions, by setting up a set of teacher training workshops implementing these principles. We aim to explore the ways in which the e-assessment tools identified in the project can (or cannot) be adopted by primary and secondary school teachers to assist them with applying these principles in practice.   Acknowledgements We would like to gratefully acknowledge the contribution of the  PREATY Project, that is funded by the European Commission's Life-long Learning Programme, Project Number 526965-LLP-1-2012-1-GR-COMENIUS-CMP, which  has part-funded this work. References 1. Black, P. J., Wiliam, D.: Inside the Black Box: Raising standards through class-room assessment. Phi Delta Kappa, 80, 139-48 (1998) 2. Black, P.J., Wiliam, D.: Developing the theory of formative assessment. Educa-tional Assessment, Evaluation and Accountability (formerly the Journal of Per-sonal Evaluation in Education), 21 (1), 5-31 (2009). 3. Busuttil-Reynaud, G., Winkley, J. : JISC e-Assessment Glossary, Bristol (2006) 4. Butler, P.: A Review Of The Literature On Portfolios And Electronic Portfolios, Massey University College of Education, New Zealand and eCDF ePortfolio Pro-ject Steering Committee (2006) 5. Gordon, J., Halasz, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., et al.: Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education. Warsaw (2009) 6. JISC: Effective Practice with e-Assessment. An overview of technologies, poli-cies and practice in further and higher education. Bristol (2007) 7. Looney, J. : Making it Happen: Formative Assessment and Educational Technol-ogies. Promethean Education Strategy Group: Thinking Deeper Research Paper No.1. – Part 3. Online available at: http://www.prometheanworld.com/rx_content/files/PDF/ MakingitHappenForma-tiveAssessmentEducationalTechnologies-169721.pdf (2010) 8. Looney, J., Siemens, G.: Assessment Competency: Knowing What You Know and Learning Analytics. It is Time for a Breakthrough. Promethean Thinking Deeper Research Paper,  No.3 (2011) 9. McManus, S.: Attributes of effective formative assessment. Washington DC: CCSSO  (2008) 10. Pachler, N., Daly, C., Mor, Y., Mellar, H.: Formative e-assessment: practitioner cases. Computers & Education, 54, 715-721 (2010) 11. Pepper, D.: Literature Review: Assessment for key competences. KeyCoNet Re-port (2012) 12. Petropoulou, O., Vassilikopoulou, M., Retalis, S.: Enriched assessment rubrics: a new medium for enabling teachers to easily assess student's performance when participating in complex interactive learning scenarios, Operational Research, Volume 11 (2), 171-186 (2011) 13. Pinock, N. Brandt, W.C.: Connecting Formative Assessment Research to prac-tice. Learning Point Associates.  Online available at: http://www.learningpt.org/pdfs/FormativeAssessment.pdf (2009) 14. Reilly, R..: Guest Editorial-Web Based Instruction: Doing Things Better and Do-ing Better Things. IEEEE Transactions on Education, 48 (4), 565-566 (2005) 15. Ridgway, J., McCusker, S., Pead, D.: Literature Review of E-assessment, Bristol (2004) 16. Siemens, G., Baker, R.: Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK '12), Simon Buckingham Shum, Dragan Gasevic, and Rebecca Ferguson (Eds.). ACM, New York, NY, USA, 252-254 (2012) 17. Sluijsmans, J.T.,  Brinke, D., Vleuten, van der, C.: Toetsen met leerwaarde. Sa-menvatting van de reviewstudie naar de effectieve kenmerken van formatief toet-sen. NWO-PROO (2013) 18. Subrahmanyam, V.V., Kanjilal, U., Sharma, V.K., Dudeja, A.K.: ePortfolio: An eWay of showcasing the artifacts. In: IEEE International Conference on Technol-ogy Enhanced Education (ICTEE), pp. 1-4. IEEE Press, Kerala, India  (2012) 19. Wiliam, D., Thompson, M.: Integrating assessment with instruction: What will it take to make it work? In C.A. Dwyer (Ed.), The future of assessment: Shaping teaching and learning (pp. 53–82). Mahwah, NJ: Erlbaum (2007)  

English

Towards the Use of New Methods for Formative e-Assessment of 21st Century Skills in Schools

Open Universiteit www.ou.nl Towards the Use of New Methods for Formative e-Assessment of 21st Century Skills in SchoolsCitation for published version (APA):Rusman, E., Boon, J., Martínez-Monés, A., Rodríguez-Triana, M. J., & Retalis, S. (2013). Towards the Use ofNew Methods for Formative e-Assessment of 21st Century Skills in Schools. Paper presented at 8th EuropeanConference on Technology Enhanced Learning, Paphos, Cyprus.Document status and date:Published: 01/09/2013Document Version:Peer reviewed versionDocument license:CC BY-NC-NDPlease check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences betweenthe submitted version and the official published version of record. People interested in the research are advised to contact the author for thefinal version of the publication, or visit the DOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and page numbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.• You may not further distribute the material or use it for any profit-making activity or commercial gain• You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:https://www.ou.nl/taverne-agreementTake down policyIf you believe that this document breaches copyright please contact us at:pure-support@ou.nlproviding details and we will investigate your claim.Downloaded from https://research.ou.nl/ on date: 12 Oct. 2022Towards the Use of New Methods for Formative  e-Assessment of  21st Century Skills in Schools Ellen Rusman1, Jo Boon1 , Alejandra Martínez-Monés2, María Jesús Rodríguez-Triana2, Simeos Retalis3 1 Open University of the Netherlands, CELSTEC, Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands  {ellen.rusman, jo.boon}ou.nl 2 University of Valladolid, GSIC-EMIC, Paseo de Belén, 12, 47014 Valladolid, Spain amartine@infor.uva.es; chus@gsic.uva.es 3  University of Piraeus, Department of Digital Systems, 80 Karaoli & Dimitriou 185 34 Piraeus, Greece retal@unipi.gr Abstract. Schools are in the process of reorienting their curricula to the devel-opment of 21st century skills and competences; however, their assessment methods have not yet been updated. The PREATY project aims to make teach-ers in primary and secondary schools aware of the use and benefits of modern e-assessment methods and tools. This paper presents the preliminary work de-veloped in the project. Keywords. Formative assessment, e-assessment, 21st century skills, key compe-tences, primary and secondary school, teacher training 1 Introduction  There has been a recent trend across the EU towards competence-based teaching and learning [5]. Nowadays, school curricula pay special at-tention to the development of 21st century skills (e.g. creativity, critical thinking, collaboration, or problem solving skills) and key competen-cies that are seen as needed for global citizens and workers. However, in many countries, the assessment system has not changed and is still primarily focused on summative assessment of static knowledge and end-results, without attention on the learning process and the acquisi-tion of competences and skills. Though there are alternative assessment methods -more process and formative orientated such as portfolios or rubrics-, they are not wide-spread and they are often seen as additional activities rather than an integral part of the curriculum. Also ICT technologies enable alterna-tive assessment forms, also called e-assessment [6]. A number of e-assessment tools are already available, and have proved their usefulness to meet assessment needs and promote learning. However, teachers, especially in primary and secondary education, have not adopted them yet. Therefore, there is a need to change the current practices and to support teachers with integrating modern assessment approaches and tools for learning as part of the curriculum. Aware of this need, the PREATY (PRoposing modern E-assessment Approaches and Tools to Young and experienced in-service teachers) project aims to equip teachers in primary and secondary schools with e-assessment strategies and tools to evaluate a number of key 21st century skills and competencies. Opposite to the traditional test and output-focussed perspective on assessment, the project pursues to promote assessment for learning, therefore focussing mainly on the formative assessment of these skills and competences.  This paper presents initial work carried out in the project. Section 2 describes our perspective on formative assessment and the potential benefits and requirements of supportive technology. Section 3 presents the modern e-assessment instruments that will be studied, with a de-scription of their main characteristics and supporting tools. Section 4 points out the challenges of implementing ‘assessment for learning’ and formative assessment in schools. Finally, Section 5 concludes with the approach and further steps taken in the PREATY project to introduce formative e-assessment of 21st century skills in schools. 2 Formative e-assessment: Characteristics and Potential Surplus Value of Technology The purpose of formative assessment is to support teaching and learn-ing processes by providing developmental feedback (on an item, topic and/or assessment level by means of different sources, such as peers, experts or teachers) to a learner (and their teacher) on his/her current understanding or skills [3][5][1][2] during a period of instruction [11][15]. Formative assessment differs from summative assessment in that it is a continuing process of feedback; therefore the information on performances is gathered continuously, mirrored against a set of prede-fined criteria or good practices, and is used to shape improvements and promote an individual's learning, rather than serve as a final formal summary of learners’ achievement [8][17]. The responsibility for learn-ing is with both learners as teachers and eventually their peers [2][9]. They determine (jointly) where a learner is going, where (s)he is now and how a learner can get where (s)he wants [19]. Formative assess-ment strategies address and integrate both role as well as reflective questioning aspects.  Technology can offer different functions that potentially can facilitate and enhance formative assessment aspects [6][10]. It can improve ac-cess to assessments by different actors (e.g. also by peers, experts and teachers) at anytime, anyplace and anywhere, enabling learners to measure their understanding when and how often they want and allow them more control of their learning. Feedback times can be shortened and this can help to change misconceptions rapidly, or feedback may be given from different perspectives, within a group or adopted to a learn-er. Thus, technology can affect feedback quality and the motivation of a learner. Also, technology can track, trace store, process and visualize learners’ results as well as actions, which makes them visi-ble and available for various learning purposes, such as individual or group reflection or to evaluate and visualize a learners progress. Technology can also affect teacher efficiency, as teachers can be sup-ported with various tools helping to reduce assessment time, thus sav-ing time that can be spent otherwise. Also, as technology enables rapid updating and combining material, it can also contribute to more varied and authentic assessment designs. 3 Identification of Available Formative e-assessment Approaches and Tools As a first step towards the introduction of formative e-assessment in primary and secondary schools, PREATY makes an inventory of inno-vative e-assessment instruments, starting with: e-Portfolio, learning analytics and enriched rubrics. These instruments may help teachers to have a better understanding of students’ performance and offer valuable feedback to students and parents. They can support formative assess-ment methods that have proven to be effective for learning [17], such as self- or peer-assessment, instructional dialogues, questioning and re-flective lessons. This section describes the main characteristics of the mentioned e-assessment instruments. The first instrument examined is an ePortfolio. They are an electron-ic version of traditional portfolios, meaning that the work samples in-side them will be of digital nature; coming from different sources and tools [18]. The ePortfolio facilitates the capabilities of the learning portfolio to show the process and results of a learning path, proving the quality and level of achievement of the targeted competences [4]. The ePortfolio should include basic storing capabilities, with interoperabil-ity support, as well the management of permissions and mechanisms to provide feedback on the shared artefacts. Finally, different views of the work samples should be possible, in order to highlight the competences of interest in each case.  A second e-assessment instrument of interest is Learning Analytics (LA), which can be defined as the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs [16]. LA allows people who are responsible for the teaching and learning process to identify possible learning risks, and make the necessary changes, depending on the situations encountered. Many LA systems and approaches are being developed, like the ones based on Social Network Analysis. In spite of the recent interest in these tech-niques, still a major challenge is to adapt the output and visualizations to the teacher’s and learner’s needs. Thirdly, learning analytics can be combined with rubrics in the so-called enriched rubrics [12]. These instruments have the form of a traditional rubric, but (at least part of) the criteria is assessed by indica-tors automatically computed from the interaction of the students with the system. They have been tested combined with the analytical capa-bilities offered by learning management systems, such as Moodle. In spite of the potential benefits of the reviewed instruments and associated tools, their uptake by the target group is very small. Next section goes into some explanations for this lack of interest.  4 Challenges and Approaches Towards Implementing Formative e-assessment in Schools Different bottlenecks can be mentioned that prevent easy implemen-tation of formative assessment in schools and the technological support of it. As the authors in [17] mention, based on an extensive review study, still a large amount of conceptual confusion exists on the defini-tion of formative assessment, and scientific research about the condi-tions of effectiveness of formative assessment is still in an early phase of development.   Implementing formative e-assessment in schools also requires some essential preparatory steps. In the first place more information and communication about formative assessment and possible approaches herein is necessary, as they are not yet widely known and accepted.  The tools mentioned in section 3 are seen as powerful, although only partially applicable for the target groups of the PREATY project. Suc-cess – and acceptance on the longer term by teachers- depends on some other crucial conditions. Reflection about formative assessment and the goals one can achieve belongs to a school culture where teachers – and management- are willing to think about innovation and come loose of a culture where they alone are in total control [17]. Looking at the con-ceptualization of [19], described earlier in this paper, it is also clear that different target groups are involved in the implementation process, not only teachers, but also learners and their peers. Thus, implementation of formative assessment in schools requires a willingness to completely (re)think (about) assessment. A second issue at stake in the PREATY project is the introduction and training related to the technological tools and the formative e-assessment aspects. Regarding the training about technological aspects it is important to focus on acceptation by giving attention on how to present the objective of technology; are tools presented as ‘doing things better’ or as ‘ doing better things’ [14]. These distinctions are important as they include implicit views on the kind of enhancement that is at stake.  From the project results thus far it is clear that very few tools in this field are available for the target group of the project. Most tools are developed for use in higher education. Teacher professionalization in the field of formative assessment and the related technological tools implies the training of complex skills. Trainings therefore will have to include [13]: 1. information and training for in service teachers on formative as-sessment – enhancing: assessment for learning, learn how to reflect, how to use criteria, how to handle feedback, how to work in teams etc.   2. information and training on technological tools;  as very few tools are available training could be used to gather information about desirability and requirements for tools to be developed in this field. 5 Conclusions The adoption of formative assessment in schools requires a change of mind, from a focus on summative testing, end results and teacher re-sponsibility towards a process and learning oriented perspective, with a joint responsibility of learners, teachers and peers. It is important to learn to see formative assessment as a process, rather than a particular kind of assessment [9], with different formative assessment strategies that can support that process. Next to teacher training material and training strategies that help to adopt and develop this mind-set, the availability of effective (for learning) formative e-assessment methods and tools can help to facilitate adoption within schools. The vision of the PREATY project is to advance in these directions, by setting up a set of teacher training workshops implementing these principles. We aim to explore the ways in which the e-assessment tools identified in the project can (or cannot) be adopted by primary and secondary school teachers to assist them with applying these principles in practice.   Acknowledgements We would like to gratefully acknowledge the contribution of the  PREATY Project, that is funded by the European Commission's Life-long Learning Programme, Project Number 526965-LLP-1-2012-1-GR-COMENIUS-CMP, which  has part-funded this work. References 1. Black, P. J., Wiliam, D.: Inside the Black Box: Raising standards through class-room assessment. Phi Delta Kappa, 80, 139-48 (1998) 2. Black, P.J., Wiliam, D.: Developing the theory of formative assessment. Educa-tional Assessment, Evaluation and Accountability (formerly the Journal of Per-sonal Evaluation in Education), 21 (1), 5-31 (2009). 3. Busuttil-Reynaud, G., Winkley, J. : JISC e-Assessment Glossary, Bristol (2006) 4. Butler, P.: A Review Of The Literature On Portfolios And Electronic Portfolios, Massey University College of Education, New Zealand and eCDF ePortfolio Pro-ject Steering Committee (2006) 5. Gordon, J., Halasz, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., et al.: Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education. Warsaw (2009) 6. JISC: Effective Practice with e-Assessment. An overview of technologies, poli-cies and practice in further and higher education. Bristol (2007) 7. Looney, J. : Making it Happen: Formative Assessment and Educational Technol-ogies. Promethean Education Strategy Group: Thinking Deeper Research Paper No.1. – Part 3. Online available at: http://www.prometheanworld.com/rx_content/files/PDF/ MakingitHappenForma-tiveAssessmentEducationalTechnologies-169721.pdf (2010) 8. Looney, J., Siemens, G.: Assessment Competency: Knowing What You Know and Learning Analytics. It is Time for a Breakthrough. Promethean Thinking Deeper Research Paper,  No.3 (2011) 9. McManus, S.: Attributes of effective formative assessment. Washington DC: CCSSO  (2008) 10. Pachler, N., Daly, C., Mor, Y., Mellar, H.: Formative e-assessment: practitioner cases. Computers & Education, 54, 715-721 (2010) 11. Pepper, D.: Literature Review: Assessment for key competences. KeyCoNet Re-port (2012) 12. Petropoulou, O., Vassilikopoulou, M., Retalis, S.: Enriched assessment rubrics: a new medium for enabling teachers to easily assess student's performance when participating in complex interactive learning scenarios, Operational Research, Volume 11 (2), 171-186 (2011) 13. Pinock, N. Brandt, W.C.: Connecting Formative Assessment Research to prac-tice. Learning Point Associates.  Online available at: http://www.learningpt.org/pdfs/FormativeAssessment.pdf (2009) 14. Reilly, R..: Guest Editorial-Web Based Instruction: Doing Things Better and Do-ing Better Things. IEEEE Transactions on Education, 48 (4), 565-566 (2005) 15. Ridgway, J., McCusker, S., Pead, D.: Literature Review of E-assessment, Bristol (2004) 16. Siemens, G., Baker, R.: Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK '12), Simon Buckingham Shum, Dragan Gasevic, and Rebecca Ferguson (Eds.). ACM, New York, NY, USA, 252-254 (2012) 17. Sluijsmans, J.T.,  Brinke, D., Vleuten, van der, C.: Toetsen met leerwaarde. Sa-menvatting van de reviewstudie naar de effectieve kenmerken van formatief toet-sen. NWO-PROO (2013) 18. Subrahmanyam, V.V., Kanjilal, U., Sharma, V.K., Dudeja, A.K.: ePortfolio: An eWay of showcasing the artifacts. In: IEEE International Conference on Technol-ogy Enhanced Education (ICTEE), pp. 1-4. IEEE Press, Kerala, India  (2012) 19. Wiliam, D., Thompson, M.: Integrating assessment with instruction: What will it take to make it work? In C.A. Dwyer (Ed.), The future of assessment: Shaping teaching and learning (pp. 53–82). Mahwah, NJ: Erlbaum (2007)  

DSpace at Open Universiteit Nederland

Towards	  the	  Use	  of	  New	  Methods	  
for	  Forma2ve	  	  
e-­‐Assessment	  of	  21st	  Century	  Skills	  
in	  Schools	  
	  
	  
Ellen	  Rusman,	  Jo	  Boon,	  Alejandra	  MarBnez-­‐Monés,	  
María	  Jesús	  Rodríguez-­‐Triana,	  Simeon	  Retalis	  
	  TEFA	  WS@	  ECTEL	  2013,	  Paphos,	  Cyprus,	  18/09/2013	  	  
Outline	  
•  IntroducOon	  	  
•  The	  vision	  of	  Preaty	  
•  A	  review	  of	  e-­‐assessment	  methods	  and	  tools	  
•  Conclusions	  and	  further	  work	  	  
IntroducOon	  
•  School	  curricula	  pay	  special	  aTenOon	  to	  21st	  
century	  skills	  and	  key	  competences	  	  
…	  however	  …	  
•  In	  many	  countries,	  the	  assessment	  system	  is	  sOll	  
primarily	  focused	  on	  summaOve	  assessment	  of	  
staOc	  knowledge	  and	  end-­‐results	  
IntroducOon	  
•  ICT	  technologies	  enable	  alternaOve	  
assessment	  forms	  (e-­‐assessment)	  and	  poses	  
potenOal	  beneﬁts	  	  
…	  however	  …	  
•  Teachers	  (and	  schools)	  have	  not	  adopted	  
these	  technologies	  
IntroducOon	  
•  PREATY	  (PRoposing	  modern	  E-­‐assessment	  Approaches	  
and	  Tools	  to	  Young	  and	  experienced	  in-­‐service	  teachers)	  	  
•  Goal:	  	  
•  Equip	  teachers	  in	  primary	  and	  secondary	  schools	  with	  
e-­‐assessment	  strategies	  and	  tools	  to	  formaOvely	  
assess	  key	  21st	  century	  skills	  and	  competencies.	  	  
The	  vision	  of	  Preaty	  
•  Technology	  as	  a	  motor	  of	  change	  	  
à	  Explore	  the	  exisOng	  proposals	  of	  ICT	  technologies	  
applied	  to	  assessment	  
•  Need	  for	  a	  change	  in	  the	  teachers’	  mind	  set	  
à Design	  and	  organize	  training	  workshops	  
•  Technology	  pull	  and	  user	  push	  
	  	  
e-­‐Assessment	  methods	  and	  tools	  
•  Two	  approaches	  (at	  least)	  of	  ICT	  support	  to	  
assessment	  
1.  ICT	  as	  an	  “oﬀ-­‐the-­‐shell”	  tool	  for	  in-­‐class	  assessment	  
methods	  
e.g:	  gForms	  to	  manage	  surveys,	  PowerPoint	  to	  make	  a	  
presentaOon	  
2.  Speciﬁc	  methods	  and	  tools	  especially	  thought	  for	  
formaOve	  assessment	  
	  
•  Main	  methods	  (and	  tools)	  reviewed	  
– E-­‐porfolio	  
– Learning	  analyOcs	  	  
– Rubrics	  &	  Enriched	  rubrics	  
e-­‐Assessment	  methods	  and	  tools	  
e-­‐Porfolio	  
•  Electronic	  version	  of	  the	  porfolio	  
•  Show	  the	  process	  and	  results	  of	  a	  learning	  
path	  
•  Data-­‐managing	  capabiliOes,	  oﬀering	  diﬀerent	  
views	  of	  the	  data,	  feedback	  faciliOes	  …	  	  
e-­‐Porfolio	  
•  Aspects	  considered:	  
– Standalone	  tool	  or	  Integrated	  in	  a	  VLE?	  
– Already	  used	  in	  schools?	  
– Language	  
– Cost,	  licence	  
– Technical	  requirements	  	  	  
e-­‐Porfolio	  systems	  	  
Tool Name Integrated 
with LMS 
Educational 
level 
Languages 
supported 
Cost License  Technical 
requirements 
Desire2Learn 
ePorfolio 
Desire2Learn Schools and 
Higher 
education 
9 Contact 
Desire2Learng 
Open source/ 
Institution 
Hosted 
Elgg  Moodle Schools and 
Higher 
education 
7 Free when self-
hosted and 
maintained. 
Open source/ 
Institution 
Hosted or server 
Epsilen 
Portfolio 
Epsilen Schools and 
Higher 
education 
English 
  
Free for 
educational sites 
FTE basis / 
Institution 
Hosted 
Exabis  Moodle Schools English, Dutch Free Open source / 
Institution 
Server 
Mahara  Moodle Schools and 
Higher 
education 
21 (Greek, 
Italian, Dutch, 
Spanish)  
Free Open source/
Institution 
Server  
PebblePad  Moodle or 
Blackboard 
Higher 
education 
English Requires 
payment 
Comercial licence / 
Institution 
Either 
Sakai 
Portfolio  
Sakai Higher 
education 
English, 
Spanish 
Free ECL License / 
Owner or institution 
Either 
Learning	  analyOcs	  	  
•  CollecOon,	  analysis,	  and	  reporOng	  of	  data	  
about	  learners	  and	  their	  contexts,	  
•  Purposes:	  understanding	  and	  opOmizing	  
learning	  
•  Several	  approaches,	  from	  the	  interacOon	  
analysis	  methods	  to	  insOtuOonal	  approaches	  
Learning	  analyOcs 	  -­‐	  integrated	  in	  VLEs	  
Name	   Primary	  users	   Output	   Integra2on	  
level	  	  
Language	   Free	   User	   help	   and	  
support	  
AnalyOcs	  and	  
RecommendaOo
ns	  	  
Teachers	  and	  
students	  
StaOsOcal	  tables	  and	  
graphs	  
	  (Moodle	  plugin	   English	   YES	   NO	  
Engagement	  
AnalyOcs	  
Teachers	  and	  
administrators	  	  
StaOsOcs	  	   Moodle	  plugin	  	   English 	   YES	   YES	  
Excel	  Pivot	  
Tables	  	  
Teachers,	  
administrators,	  
researchers	  
StaOsOcal	  tables,	  
diagrams,	  export	  ﬁles	  
	  (Part	  of	  
Microsoj	  Oﬃce	  
package)	  	  
Several (as many as 
MS office tools) 	  
NO	   YES.	  	  
GISMO	  	   Teachers	   Enriched	  staOsOcal	  
tables,	  bar	  diagrams,	  
export	  ﬁles	  	  
Moodle	  plugin	   English 	   YES	   YES	  	  
MOCLog	  	   Students,	  teachers,	  
educaOonal	  
designers,	  and	  
administrators	  	  
StaOsOcs	  adapted	  to	  
the	  user	  (CSV	  ﬁles.	  	  
Moodle	  
package	  
English	   YES	   YES	  	  	  
	  
Learning	  analyOcs	  -­‐	  Social	  network	  analysis	  tools	  
Name	   Primary	  users	   Output	   Integra2on	  level	  	   Language	   Cost	   User	  help	  
and	  
support	  
SAMSA	  	   Teachers	  and	  
researchers	  
VisualizaOons	  and	  
SNA	  analysis.	  	  
Medium	  (BSCW	  /
Synergeia.	  Forums...)	  
English 	   -­‐	   NO	  
SNAPP	  	   Teachers	  and	  students	  	   Sociograms	  with	  the	  
interacOons	  in	  forum	  
Integrated	  with	  several	  
LMS	  (Blackboard,	  
ANGEL,	  Desire2Learn,	  
and	  Moodle)	  
English 	   Free	   NO	  	  
NodeXL	  	   SNA	  researchers	  and	  
parOally,	  pracOOoners.	  	  
	  	  
.csv	  (text),	  .txt,	  .xls	  
(Excel),	  .xslt	  (Excel	  
2007),	  .dl	  (UCINet),	  
GraphML	  
Medium	  (Excel	  ﬁles).	  	   English 	   Free	  (Lite	  
version)	  
YES	  
Enriched	  Rubrics	  
•  A	  tradiOonal	  rubric,	  where	  part	  of	  the	  criteria	  
are	  automaOcally	  computed	  by	  the	  system	  
•  LAe-­‐R:	  A	  Moodle	  plug-­‐in	  
•  LAe-­‐R	  allows	  a	  teacher	  to	  add	  types	  of	  criteria	  
associated	  to	  learning	  analysis	  indicators	  
Enriched	  rubrics	  
!
Enriched	  rubrics	  
•  Tested	  with	  12	  in-­‐service	  primary	  and	  
secondary	  school	  teachers	  with	  posiOve	  
results,	  but	  not	  tested	  in	  real	  pracOce	  
•  One	  example	  of	  a	  tool	  that	  could	  help	  idenOfy	  
the	  opportuniOes	  and	  requirements	  of	  e-­‐
assessment	  tools	  at	  the	  target	  school	  levels	  	  
Conclusion	  and	  next	  steps	  
•  Introducing	  formaOve	  e-­‐assessment	  in	  the	  
schools	  requires	  a	  change	  in	  the	  mentality	  of	  
teachers	  (and	  other	  stakeholders)	  and	  more	  
adapted	  tools	  
•  Teacher	  workshops	  will	  be	  designed	  with	  the	  
aim	  of	  contribuOng	  to	  both	  lines	  of	  work	  	  
Conclusion	  and	  next	  steps	  
•  Principles	  for	  the	  teacher	  workshops	  
1.  Training	  on	  e-­‐assessment	  	  
2.  Training	  on	  tools	  	  	  	  
–  Sketch	  scenarios	  using	  tools	  used	  at	  higher	  educaOon	  
(o,	  if	  any,	  at	  primary	  or	  secondary	  schools)	  
–  IdenOfy	  requirements	  for	  primary	  and	  secondary	  
educaOon	  



Rusman, E., Boon, J., Martínez-Monés, A., Rodríguez-Triana, M. J., & Retalis, S. (2013). Towards the Use of New Methods for Formative e-Assessment of 21st Century Skills in Schools. In Technology Enhanced Formative Assessment (TEFA) workshop (http://www.kbs.uni-hannover.de/fileadmin/institut/pdf/TEFA2013-Paper_1_Rusman-et_al.pdf ) at EC-TEL 2013 (http://www.ec-tel.eu/index.php). September, 17-21, 2013, Paphos, Cyprus.Schools are in the process of reorienting their curricula to the development of 21st century skills and competences; however, their assessment methods have not yet been updated. The PREATY project aims to make teachers in primary and secondary schools aware of the use and benefits of modern e-assessment methods and tools. This paper presents the preliminary work developed in the project.We would like to gratefully acknowledge the contribution of the  PREATY Project, that is funded by the European Commission's Life-long Learning Programme, Project Number 526965-LLP-1-2012-1-GR-COMENIUS-CMP, which  has part-funded this work

Towards the Use of New Methods for Formative  
e-Assessment of  21st Century Skills in Schools 
Ellen Rusman1, Jo Boon1 , Alejandra Martínez-Monés2, María Jesús 
Rodríguez-Triana2, Simeos Retalis3 
1 Open University of the Netherlands, CELSTEC, Valkenburgerweg 177, 6419 AT Heerlen, 
The Netherlands  
{ellen.rusman, jo.boon}ou.nl 
2 University of Valladolid, GSIC-EMIC, Paseo de Belén, 12, 47014 Valladolid, Spain 
amartine@infor.uva.es; chus@gsic.uva.es 
3  University of Piraeus, Department of Digital Systems, 80 Karaoli & Dimitriou 
185 34 Piraeus, Greece 
retal@unipi.gr 
Abstract. Schools are in the process of reorienting their curricula to the devel-
opment of 21st century skills and competences; however, their assessment 
methods have not yet been updated. The PREATY project aims to make teach-
ers in primary and secondary schools aware of the use and benefits of modern 
e-assessment methods and tools. This paper presents the preliminary work de-
veloped in the project. 
Keywords. Formative assessment, e-assessment, 21st century skills, key compe-
tences, primary and secondary school, teacher training 
1 Introduction  
There has been a recent trend across the EU towards competence-based 
teaching and learning [5]. Nowadays, school curricula pay special at-
tention to the development of 21st century skills (e.g. creativity, critical 
thinking, collaboration, or problem solving skills) and key competen-
cies that are seen as needed for global citizens and workers. However, 
in many countries, the assessment system has not changed and is still 
primarily focused on summative assessment of static knowledge and 
end-results, without attention on the learning process and the acquisi-
tion of competences and skills. 
Though there are alternative assessment methods -more process and 
formative orientated such as portfolios or rubrics-, they are not wide-
spread and they are often seen as additional activities rather than an 
integral part of the curriculum. Also ICT technologies enable alterna-
tive assessment forms, also called e-assessment [6]. A number of e-
assessment tools are already available, and have proved their usefulness 
to meet assessment needs and promote learning. However, teachers, 
especially in primary and secondary education, have not adopted them 
yet. Therefore, there is a need to change the current practices and to 
support teachers with integrating modern assessment approaches and 
tools for learning as part of the curriculum. 
Aware of this need, the PREATY (PRoposing modern E-assessment 
Approaches and Tools to Young and experienced in-service teachers) 
project aims to equip teachers in primary and secondary schools with e-
assessment strategies and tools to evaluate a number of key 21st century 
skills and competencies. Opposite to the traditional test and output-
focussed perspective on assessment, the project pursues to promote 
assessment for learning, therefore focussing mainly on the formative 
assessment of these skills and competences.  
This paper presents initial work carried out in the project. Section 2 
describes our perspective on formative assessment and the potential 
benefits and requirements of supportive technology. Section 3 presents 
the modern e-assessment instruments that will be studied, with a de-
scription of their main characteristics and supporting tools. Section 4 
points out the challenges of implementing ‘assessment for learning’ and 
formative assessment in schools. Finally, Section 5 concludes with the 
approach and further steps taken in the PREATY project to introduce 
formative e-assessment of 21st century skills in schools. 
2 Formative e-assessment: Characteristics and Potential 
Surplus Value of Technology 
The purpose of formative assessment is to support teaching and learn-
ing processes by providing developmental feedback (on an item, topic 
and/or assessment level by means of different sources, such as peers, 
experts or teachers) to a learner (and their teacher) on his/her current 
understanding or skills [3][5][1][2] during a period of instruction 
[11][15]. Formative assessment differs from summative assessment in 
that it is a continuing process of feedback; therefore the information on 
performances is gathered continuously, mirrored against a set of prede-
fined criteria or good practices, and is used to shape improvements and 
promote an individual's learning, rather than serve as a final formal 
summary of learners’ achievement [8][17]. The responsibility for learn-
ing is with both learners as teachers and eventually their peers [2][9]. 
They determine (jointly) where a learner is going, where (s)he is now 
and how a learner can get where (s)he wants [19]. Formative assess-
ment strategies address and integrate both role as well as reflective 
questioning aspects.  
Technology can offer different functions that potentially can facilitate 
and enhance formative assessment aspects [6][10]. It can improve ac-
cess to assessments by different actors (e.g. also by peers, experts and 
teachers) at anytime, anyplace and anywhere, enabling learners to 
measure their understanding when and how often they want and allow 
them more control of their learning. Feedback times can be shortened 
and this can help to change misconceptions rapidly, or feedback may be 
given from different perspectives, within a group or adopted to a learn-
er. Thus, technology can affect feedback quality and the motivation 
of a learner. Also, technology can track, trace store, process and 
visualize learners’ results as well as actions, which makes them visi-
ble and available for various learning purposes, such as individual or 
group reflection or to evaluate and visualize a learners progress. 
Technology can also affect teacher efficiency, as teachers can be sup-
ported with various tools helping to reduce assessment time, thus sav-
ing time that can be spent otherwise. Also, as technology enables rapid 
updating and combining material, it can also contribute to more varied 
and authentic assessment designs. 
3 Identification of Available Formative e-assessment 
Approaches and Tools 
As a first step towards the introduction of formative e-assessment in 
primary and secondary schools, PREATY makes an inventory of inno-
vative e-assessment instruments, starting with: e-Portfolio, learning 
analytics and enriched rubrics. These instruments may help teachers to 
have a better understanding of students’ performance and offer valuable 
feedback to students and parents. They can support formative assess-
ment methods that have proven to be effective for learning [17], such as 
self- or peer-assessment, instructional dialogues, questioning and re-
flective lessons. This section describes the main characteristics of the 
mentioned e-assessment instruments. 
The first instrument examined is an ePortfolio. They are an electron-
ic version of traditional portfolios, meaning that the work samples in-
side them will be of digital nature; coming from different sources and 
tools [18]. The ePortfolio facilitates the capabilities of the learning 
portfolio to show the process and results of a learning path, proving the 
quality and level of achievement of the targeted competences [4]. The 
ePortfolio should include basic storing capabilities, with interoperabil-
ity support, as well the management of permissions and mechanisms to 
provide feedback on the shared artefacts. Finally, different views of the 
work samples should be possible, in order to highlight the competences 
of interest in each case.  
A second e-assessment instrument of interest is Learning Analytics 
(LA), which can be defined as the measurement, collection, analysis, 
and reporting of data about learners and their contexts, for purposes of 
understanding and optimizing learning and the environments in which 
it occurs [16]. LA allows people who are responsible for the teaching 
and learning process to identify possible learning risks, and make the 
necessary changes, depending on the situations encountered. Many LA 
systems and approaches are being developed, like the ones based on 
Social Network Analysis. In spite of the recent interest in these tech-
niques, still a major challenge is to adapt the output and visualizations 
to the teacher’s and learner’s needs. 
Thirdly, learning analytics can be combined with rubrics in the 
so-called enriched rubrics [12]. These instruments have the form of a 
traditional rubric, but (at least part of) the criteria is assessed by indica-
tors automatically computed from the interaction of the students with 
the system. They have been tested combined with the analytical capa-
bilities offered by learning management systems, such as Moodle. 
In spite of the potential benefits of the reviewed instruments and 
associated tools, their uptake by the target group is very small. Next 
section goes into some explanations for this lack of interest.  
4 Challenges and Approaches Towards Implementing 
Formative e-assessment in Schools 
Different bottlenecks can be mentioned that prevent easy implemen-
tation of formative assessment in schools and the technological support 
of it. As the authors in [17] mention, based on an extensive review 
study, still a large amount of conceptual confusion exists on the defini-
tion of formative assessment, and scientific research about the condi-
tions of effectiveness of formative assessment is still in an early phase 
of development.   
Implementing formative e-assessment in schools also requires some 
essential preparatory steps. In the first place more information and 
communication about formative assessment and possible approaches 
herein is necessary, as they are not yet widely known and accepted.  
The tools mentioned in section 3 are seen as powerful, although only 
partially applicable for the target groups of the PREATY project. Suc-
cess – and acceptance on the longer term by teachers- depends on some 
other crucial conditions. Reflection about formative assessment and the 
goals one can achieve belongs to a school culture where teachers – and 
management- are willing to think about innovation and come loose of a 
culture where they alone are in total control [17]. Looking at the con-
ceptualization of [19], described earlier in this paper, it is also clear that 
different target groups are involved in the implementation process, not 
only teachers, but also learners and their peers. Thus, implementation 
of formative assessment in schools requires a willingness to completely 
(re)think (about) assessment. 
A second issue at stake in the PREATY project is the introduction 
and training related to the technological tools and the formative e-
assessment aspects. Regarding the training about technological aspects 
it is important to focus on acceptation by giving attention on how to 
present the objective of technology; are tools presented as ‘doing things 
better’ or as ‘ doing better things’ [14]. These distinctions are important 
as they include implicit views on the kind of enhancement that is at 
stake.  From the project results thus far it is clear that very few tools in 
this field are available for the target group of the project. Most tools are 
developed for use in higher education. 
Teacher professionalization in the field of formative assessment and 
the related technological tools implies the training of complex skills. 
Trainings therefore will have to include [13]: 
1. information and training for in service teachers on formative as-
sessment – enhancing: assessment for learning, learn how to reflect, 
how to use criteria, how to handle feedback, how to work in teams etc.   
2. information and training on technological tools;  as very few 
tools are available training could be used to gather information about 
desirability and requirements for tools to be developed in this field. 
5 Conclusions 
The adoption of formative assessment in schools requires a change of 
mind, from a focus on summative testing, end results and teacher re-
sponsibility towards a process and learning oriented perspective, with a 
joint responsibility of learners, teachers and peers. It is important to 
learn to see formative assessment as a process, rather than a particular 
kind of assessment [9], with different formative assessment strategies 
that can support that process. Next to teacher training material and 
training strategies that help to adopt and develop this mind-set, the 
availability of effective (for learning) formative e-assessment methods 
and tools can help to facilitate adoption within schools. The vision of 
the PREATY project is to advance in these directions, by setting up a 
set of teacher training workshops implementing these principles. We 
aim to explore the ways in which the e-assessment tools identified in 
the project can (or cannot) be adopted by primary and secondary school 
teachers to assist them with applying these principles in practice.  
 
Acknowledgements 
We would like to gratefully acknowledge the contribution of the  
PREATY Project, that is funded by the European Commission's Life-
long Learning Programme, Project Number 526965-LLP-1-2012-1-
GR-COMENIUS-CMP, which  has part-funded this work. 
References 
1. Black, P. J., Wiliam, D.: Inside the Black Box: Raising standards through class-
room assessment. Phi Delta Kappa, 80, 139-48 (1998) 
2. Black, P.J., Wiliam, D.: Developing the theory of formative assessment. Educa-
tional Assessment, Evaluation and Accountability (formerly the Journal of Per-
sonal Evaluation in Education), 21 (1), 5-31 (2009). 
3. Busuttil-Reynaud, G., Winkley, J. : JISC e-Assessment Glossary, Bristol (2006) 
4. Butler, P.: A Review Of The Literature On Portfolios And Electronic Portfolios, 
Massey University College of Education, New Zealand and eCDF ePortfolio Pro-
ject Steering Committee (2006) 
5. Gordon, J., Halasz, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., et al.: 
Key competences in Europe: Opening doors for lifelong learners across the 
school curriculum and teacher education. Warsaw (2009) 
6. JISC: Effective Practice with e-Assessment. An overview of technologies, poli-
cies and practice in further and higher education. Bristol (2007) 
7. Looney, J. : Making it Happen: Formative Assessment and Educational Technol-
ogies. Promethean Education Strategy Group: Thinking Deeper Research Paper 
No.1. – Part 3. Online available at: 
http://www.prometheanworld.com/rx_content/files/PDF/ MakingitHappenForma-
tiveAssessmentEducationalTechnologies-169721.pdf (2010) 
8. Looney, J., Siemens, G.: Assessment Competency: Knowing What You Know 
and Learning Analytics. It is Time for a Breakthrough. Promethean Thinking 
Deeper Research Paper,  No.3 (2011) 
9. McManus, S.: Attributes of effective formative assessment. Washington DC: 
CCSSO  (2008) 
10. Pachler, N., Daly, C., Mor, Y., Mellar, H.: Formative e-assessment: practitioner 
cases. Computers & Education, 54, 715-721 (2010) 
11. Pepper, D.: Literature Review: Assessment for key competences. KeyCoNet Re-
port (2012) 
12. Petropoulou, O., Vassilikopoulou, M., Retalis, S.: Enriched assessment rubrics: a 
new medium for enabling teachers to easily assess student's performance when 
participating in complex interactive learning scenarios, Operational Research, 
Volume 11 (2), 171-186 (2011) 
13. Pinock, N. Brandt, W.C.: Connecting Formative Assessment Research to prac-
tice. Learning Point Associates.  Online available at: 
http://www.learningpt.org/pdfs/FormativeAssessment.pdf (2009) 
14. Reilly, R..: Guest Editorial-Web Based Instruction: Doing Things Better and Do-
ing Better Things. IEEEE Transactions on Education, 48 (4), 565-566 (2005) 
15. Ridgway, J., McCusker, S., Pead, D.: Literature Review of E-assessment, Bristol 
(2004) 
16. Siemens, G., Baker, R.: Learning analytics and educational data mining: towards 
communication and collaboration. In Proceedings of the 2nd International 
Conference on Learning Analytics and Knowledge (LAK '12), Simon 
Buckingham Shum, Dragan Gasevic, and Rebecca Ferguson (Eds.). ACM, New 
York, NY, USA, 252-254 (2012) 
17. Sluijsmans, J.T.,  Brinke, D., Vleuten, van der, C.: Toetsen met leerwaarde. Sa-
menvatting van de reviewstudie naar de effectieve kenmerken van formatief toet-
sen. NWO-PROO (2013) 
18. Subrahmanyam, V.V., Kanjilal, U., Sharma, V.K., Dudeja, A.K.: ePortfolio: An 
eWay of showcasing the artifacts. In: IEEE International Conference on Technol-
ogy Enhanced Education (ICTEE), pp. 1-4. IEEE Press, Kerala, India  (2012) 
19. Wiliam, D., Thompson, M.: Integrating assessment with instruction: What will it 
take to make it work? In C.A. Dwyer (Ed.), The future of assessment: Shaping 
teaching and learning (pp. 53–82). Mahwah, NJ: Erlbaum (2007) 
 


https://research.ou.nl/files/958147/TEFA_manuscript.pdf

Towards the Use of New Methods for Formative e-Assessment of 21st Century Skills in Schools

Abstract

Similar works

Full text

Available Versions

Open University of the Netherlands Research Portal

Open University of the Netherlands Research Portal

DSpace at Open Universiteit Nederland

DSpace at Open Universiteit Nederland