Chapter 5: Evaluation

The OTiS (Online Teaching in Scotland) programme, run by the now defunct Scotcit programme, ran an International e-Workshop on Developing Online Tutoring Skills which was held between 8–12 May 2000. It was organised by Heriot–Watt University, Edinburgh and The Robert Gordon University, Aberdeen, UK. Out of this workshop came the seminal Online Tutoring E-Book, a generic primer on e-learning pedagogy and methodology, full of practical implementation guidelines. Although the Scotcit programme ended some years ago, the E-Book has been copied to the SONET site as a series of PDF files, which are now available via the ALT Open Access Repository. The editor, Carol Higgison, is currently working in e-learning at the University of Bradford (see her staff profile) and is the Chair of the Association for Learning Technology (ALT)

Thereʼs no ʻIʼ in ʻEmergency Management Team:ʼ designing and evaluating a serious game for training emergency managers in group decision making skills

Serious games are games that are designed to educate rather than entertain. The game outlined and evaluated here was commissioned and designed as a tool to improve the group decision making skills of people who manage real-world emergencies such as floods, fires, volcanoes and chemical spills. The game design exploits research on decision making groups and applies pedagogically sound games design principles. An evaluation of the game design was carried out based on a paper prototype. Eight participants were recruited and assigned to two groups of four participants each. These groups were video recorded while playing the game and the video was analysed in terms of game actions and member participation. Results indicate that the group who behaved in a more appropriate manner for a decision making group were rewarded with more positive feedback from the game state. These findings suggest that the game itself delivers appropriate feedback to players on their collaborative behaviour and is thus fit for the purposes intended in the current project

Charting Success: Using Practical Measures to Assess Information Literacy Skills in the First-Year Writing Course

Objective – The aim was to measure the impact of a peer-to-peer model on information literacy skill-building among first-year students at a small commuter college in the United States. The University of New Hampshire (UNH) is the state’s flagship public university and UNH Manchester is one of its seven colleges. This study contributed to a program evaluation of the Research Mentor Program at UNH Manchester whereby peer writing tutors are trained in basic library research skills to support first-year students throughout the research and writing process. Methods – The methodology employed a locally developed pre-test/post-test instrument with fixed-choice and open-ended questions to measure students’ knowledge of the library research process. Anonymized data was collected using an online survey with SurveyMonkey™ software. A rubric was developed to score the responses to open-ended questions. Results – The study indicated a positive progression toward increased learning for the three information literacy skills targeted: 1) using library resources correctly, 2) building effective search strategies, and 3) evaluating sources appropriately. Students scored higher in the fixed-choice questions than the open-ended ones, demonstrating their ability to more effectively identify the applicable information literacy skill than use the language of information literacy to describe their own research behavior. Conclusions – The assessment methodology used was an assortment of low-key, locally-developed instruments that provided timely data to measure students understanding of concepts taught and to apply those concepts correctly. Although the conclusions are not generalizable to other institutions, the findings were a valuable component of an ongoing program evaluation. Further assessment measuring student performance would strengthen the conclusions attained in this study

How can exploratory learning with games and simulations within the curriculum be most effectively evaluated?

There have been few attempts to introduce frameworks that can help support tutors evaluate educational games and simulations that can be most effective in their particular learning context and subject area. The lack of a dedicated framework has produced a significant impediment for uptake of games and simulations particularly in formal learning contexts. This paper aims to address this shortcoming by introducing a four-dimensional framework for helping tutors to evaluate the potential of using games- and simulation- based learning in their practice, and to support more critical approaches to this form of games and simulations. The four-dimensional framework is applied to two examples from practice to test its efficacy and structure critical reflection upon practice

Evaluation of a global MBA programme

E-learning continues to develop rapidly supported by increasing sophistication of information technology and by better understanding of how to make content and delivery more effective. Moreover, new forms of e-learning support system are being introduced to higher education institutions in an effort to meet the student-centred learning paradigms recommended by UNESCO (UNESCO, 1998). The creation and implementation of effective quality assurance for such learning processes has been identified as one of the most challenging tasks. Jara & Mellar (2010) and Martínez-Argüelles et al. (2010) point out that the collection of student feedback should be a central part of strategies to monitor the quality and standards of teaching and learning in higher education institutions for both conventional learning and e-learning. Jara & Mellar (2010) note also that while research into e-learning abounds, studies that focus on the effectiveness of the provision of e-learning are limited, and that this is a gap to be filled. This article reports on the evaluation of, and the consequent changes to, the global e-learning MBA programme from the University of Bedfordshire. The research was conducted by three members of the MBA team - two Senior Lecturers and the E-learning Development Manager. The aim was to investigate the learning experiences and perceptions of the students and to evaluate the effectiveness of the e-learning support system

Investigating Continuing Professional Development Provided for Egyptian Higher Education Online Tutors

Tutors confront great challenges in their teaching practices, including changes in subject content, new instructional methods, changes laws and procedures and students’ needs. In online learning (OL), more changes can be added, namely, the massive and accelerated advance in technology. Therefore, online tutors need to be provided with CPD that develops their skills and experience to improve the effectiveness of their distance learning courses. This paper investigates how Egyptian Higher Education (HE) online tutors are provided with Continuing Professional Development (CPD) to pursue their work. Data for this paper was collected form 20 online tutors from two major Egyptian universities. This paper explores the current situation of CPD provided for HE online tutors with its affordances, limitations, and proposed recommendations that can help to overcome these challenges

ITS for Teaching TOEFL

Abstract: An e-learning system is increasingly gaining popularity in the academic community because of several benefits of learning anywhere anyplace and anytime. An Intelligent Tutoring System (ITS) is a computer system that aims to provide immediate and customized instruction or feedback to learners, usually without requiring intervention from a human teacher.(ITSB) is the tutoring system Builder Which designed and improved to help teachers in building intelligent tutoring system in many fields .In this paper we have an example and an evaluating are presented of building an intelligent tutoring system for teaching TOEFL using ITSB tool