Assessing with e-Ase

The Educational Development Unit have produced a guide to assist academic staff in transforming assessment practices underpinned by technology. 'Assessing with e-Ase' highlights how technology combined with sound pedagogy has the potential to enhance the student experience of learning, teaching and assessment. Drawing on the experiences, knowledge and understanding of the e-assessment project at Middlesex University, it offers a consideration of contextual, procedural and logistical factors that may impact when using e-assessment practices. Key issues for consideration when developing and implementing e-assessment are included along with the part played by policy and strategy. Examples of departmental case studies, an outline of the support available to staff and links to resources that may provide useful additional information are also included

Is AI changing learning and assessment as we know it? Evidence from a ChatGPT experiment and a conceptual framework

ChatGPT, a state-of-the-art chatbot built upon Open AI's generative pre-trained transformer, has generated a major public interest and caused quite a stir in the higher education sector, where reactions have ranged from excitement to consternation. This paper therefore examines the potential impact of ChatGPT on learning and assessment, using the example of academic essays, being a major form of assessment with widespread applications of ChatGPT. This provides an opportunity to unpack broader insights on the challenge of generative AI's to the relevance, quality and credibility of higher education learning in a rapidly changing 21st century knowledge economy. We conducted a quasi-experiment in which we deployed ChatGPT to generate academic essays in response to a typical assessment brief, and then subjected the essays to plagiarism checks and independent grading. The results indicate that ChatGPT is able to generate highly original, and high quality, contents from distinct individual accounts in response to the same assessment brief. However, it is unable to generate multiple original contents from the same account, and it struggled with referencing. The discussion highlights the need for higher education providers to rethink their approach to assessment, in response to disruption precipitated by artificial intelligence. Thus, following the discussion of empirical data, we propose a new conceptual framework for AI-assisted assessment for lifelong learning, in which the parameters of assessment extend beyond knowledge (know what) testing, to competence (know how) assessment and performance (show how) evaluation

Digital writing technologies in higher education : theory, research, and practice

This open access book serves as a comprehensive guide to digital writing technology, featuring contributions from over 20 renowned researchers from various disciplines around the world. The book is designed to provide a state-of-the-art synthesis of the developments in digital writing in higher education, making it an essential resource for anyone interested in this rapidly evolving field. In the first part of the book, the authors offer an overview of the impact that digitalization has had on writing, covering more than 25 key technological innovations and their implications for writing practices and pedagogical uses. Drawing on these chapters, the second part of the book explores the theoretical underpinnings of digital writing technology such as writing and learning, writing quality, formulation support, writing and thinking, and writing processes. The authors provide insightful analysis on the impact of these developments and offer valuable insights into the future of writing. Overall, this book provides a cohesive and consistent theoretical view of the new realities of digital writing, complementing existing literature on the digitalization of writing. It is an essential resource for scholars, educators, and practitioners interested in the intersection of technology and writing

Collaborative examinations in asyncronous learning networks : field experiments on collaborative learning through online assessments

With the proliferation of computer networks and the emergence of virtual teams, learning and knowledge sharing in the online environment has become an increasingly important topic. Applying constructivism and collaborative learning theories to assessment, the collaborative online exam is designed featuring students\u27 active participation in various phases of the exam process through small group activities online. A participatory online exam process is designed featuring similar procedures except that students\u27 involvement in each phase of the exam is individual. The collaborative online exam and the participatory online exam are investigated regarding student exam study strategies, group process, exam outcomes, faculty satisfaction, and exam efficiency. A 1*3 field experiment was conducted to compare three exam modes: the traditional exam, the participatory exam, and the collaborative exam. Results show that the collaborative examination significantly enhanced interaction and promoted higher order learning. In particular, small group activities in the online learning process significantly increased interactions among students which enhanced their sense of an online learning community. Active involvement in the online exams significantly reduced the use of surface learning in exam study. Overall, students reported significantly higher perceptions of learning in the collaborative exam than the other exam modes

E-assessment for learning? Exploring the potential of computer-marked assessment and computer-generated feedback, from short-answer questions to assessment analytics.

This submission draws on research from twelve publications, all addressing some aspect of the broad research question: “Can interactive computer-marked assessment improve the effectiveness of assessment for learning?” The work starts from a consideration of the conditions under which assessment of any sort is predicted to best support learning, and reviews the broader literature of assessment and feedback before considering the potential of computer-based assessment, focusing on relatively sophisticated constructed-response questions, and on the impact of instantaneous, tailored and increasing feedback. A range of qualitative and quantitative research methodologies are used to investigate factors which influence the engagement of distance learners of science with computer-marked assessment and computer-generated feedback. It is concluded that the strongest influence on engagement is the student’s understanding of what they are required to do, including their understanding of the wording of assessment tasks and feedback. Clarity of wording is thus important, as is an iterative design process that allows for improvements to be made. Factors such as cut-off dates can have considerable impact, pointing to the importance of good overall assessment design, and more generally to the power and responsibility that lie in the hands of remote developers of online assessment and teaching. Four of the publications describe research into the marking accuracy and effectiveness of questions to which students give their answer as a short phrase or sentence. Relatively simple pattern-matching software has been shown to give marking accuracy at least as good as that of human markers and more sophisticated computer-marked systems, provided questions are developed on the basis of responses from students at a similar level. However, educators continue to use selected-response questions in preference to constructed-response questions, despite concerns over the validity and authenticity of selected-response questions. Factors contributing to the low take-up of more sophisticated computer-marked tasks are discussed. E-assessment also has the potential to improve the learning experience indirectly by providing information to educators about student engagement and student errors, at either the cohort or individual student level. The effectiveness of these “assessment analytics” is also considered, concluding that they have the potential to provide deep general insight and an early warning of at-risk students

A reflective view of pedagogical teaching framework focused on experiential learning:achieving university teaching and learning enhancement strategy and graduate attributes

Within teacher education reflection has been shown to be of beneficial use for improving practice across all stages of education. This article is reflective of individual teaching practice within Higher Education at Abertay University, specifically to the field of Sport and Exercise Science. Efforts to reflect on current practice in comparison to experiential learning are made. These are then compared to the ‘Abertay Attributes’ in order to determine the validity of including experiential learning within this field at the University. There are also comparisons made between classes where experiential learning is encouraged and the ‘traditional’ academic style of delivery, the lecture. It is strongly suggested that experiential learning enhances the learning experience of the student. There are also direct links with the aims of the teaching and learning enhancement strategy as well as the ‘Abertay Attributes’, most significantly the ‘Intellectual’, ‘Professional’ and ‘Active Citizen’ attributes.Keywords: Experiential Learning, Teaching, University, Sport, Reflection. <br/

Lecturers’ reflections on the teaching of electrical systems and construction NQF L4 NCV curriculum at TVET college campuses in KwaZulu-Natal.

Masters Degree. University of KwaZulu-Natal, Durban.The purpose of this study was to explore reflections of seven lecturers, all teaching Electrical Infrastructure Construction NQF L4 NCV subjects. Moreover, three are teaching Electrical Systems and Constructions (ES&C) NQF L4 NCV curriculum elective subject and four teaching core subjects at two TVET College campuses in Kwa-Zulu Natal. The research was also meant to answer the following research questions: What are the lectures’ reflections of the teaching of ES&C NQF L4 NCV curriculum at TVET College campuses? What informs lecturers’ reflections of the teaching of ES&C NQF L4 curriculum at the TVET College campuses? (Why these reflections?) Mention study educational gains from the lecturers’ reflections as they teach ES&C NQF4 NCV curriculum at the TVET College campuses. This study presents qualitative method and framed in an interpretative paradigm. Purposive and convenience sampling were utilised to select the most accessible participants. Furthermore, this study was framed by the concepts of curricula spider-web concepts/learning signals to explore lecturers’ reflection on their teaching practice. Data was generated through reflective activity, one-on-one semi-structured interviews and focus group discussion to explore the lecturers’ reflections. Data was analysed using guided analysis. Moreover, lecturers’ reflections levels were categorised into professional, societal and personal rationale. However, the study findings indicated that lecturers were mostly influenced by personal and societal rationale in teaching of ES&C NQF L4 NCV curriculum. The findings of this study also revealed that lecturers are not aware of all the factors that underpin their teaching practices. Furthermore, the lack of the knowledge of the curricular spider-web concepts/ learning signals may result in disparities between intended, implemented and attained curriculum. The findings from the literature review identified three levels of reflections that are important for lecturers to understand the curriculum. These levels are technical, practical and critical reflection, and they were used to frame the responses of the teachers. In addition, this study finding revealed that training resources were a major challenge that created disjuncture between the intended, the implemented and the attained curriculum. Recommendations were made suggesting that lecturers be empowered with critical reflections knowledge, to be able to master curriculum implementation challenges. Moreover, the management must attend to the resources challenges. Key words: Electrical System & Construction NQF L4 NCV curriculum, dual-content, curricular spider-web concepts, learning resources, Lecturers’ reflections

Towards a Research Agenda on Computer-Based Assessment - Challenges and Needs for European Educational Measurement

In 2006 the European Parliament and the Council of Europe have passed recommendations on key competences for lifelong learning and the use of a common reference tool to observe and promote progress in terms of the achievement of goals formulated in ¿Lisbon strategy¿ in March 2000 (revised in 2006, see http://ec.europa.eu/growthandjobs/) and its follow-up declarations. For those areas which are not already covered by existing measurements (foreign languages and learning-to-learn skills), indicators for the identification of such skills are now needed, as well as effective instruments for carrying out large-scale assessments in Europe. In this context it is hoped that electronic testing could improve the effectiveness of the needed assessments, i.e. to improve identification of skills, by reducing costs of the whole operation (financial efforts, human resources etc.). The European Commission is asked to assist Member States to define the organisational and resource implications for them of the construction and administration of tests, including looking into the possibility of adopting e-testing as the means to administer the tests. In addition to traditional testing approaches carried out in a paper-pencil mode, there are a variety of aspects needed to be taken into account when computer-based testing is deployed, such as software quality, secure delivery, if Internet-based: reliable network capacities, support, maintenance, software costs for development and test delivery, including licences. Future European surveys are going to introduce new ways of assessing student achievements. Tests can be calibrated to the specific competence level of each student and become more stimulating, going much further than it can be achieved with traditional multiple choice questions. Simulations provide better means of contextualising skills to real life situations and providing a more complete picture of the actual competence to be assessed. However, a variety of challenges require more research into the barriers posed by the use of technologies, e.g. in terms of computer, performance and security. The ¿Quality of Scientific Information¿ Action (QSI) and the Centre for Research on Lifelong Learning (CRELL) are carrying out a research project on quality criteria of Open Source skills assessment tools. 2 workshops were carried out in previous years bringing together European key experts from assessment research and practice in order to identify and discuss quality criteria relevant for carrying out large-scale assessments at a European level. This report reflects the contributions made on experiences and key challenges for European skills assessment.JRC.G.9-Econometrics and statistical support to antifrau