Chapter 5: Evaluation

The OTiS (Online Teaching in Scotland) programme, run by the now defunct Scotcit programme, ran an International e-Workshop on Developing Online Tutoring Skills which was held between 8–12 May 2000. It was organised by Heriot–Watt University, Edinburgh and The Robert Gordon University, Aberdeen, UK. Out of this workshop came the seminal Online Tutoring E-Book, a generic primer on e-learning pedagogy and methodology, full of practical implementation guidelines. Although the Scotcit programme ended some years ago, the E-Book has been copied to the SONET site as a series of PDF files, which are now available via the ALT Open Access Repository. The editor, Carol Higgison, is currently working in e-learning at the University of Bradford (see her staff profile) and is the Chair of the Association for Learning Technology (ALT)

Threshold concepts: Impacts on teaching and learning at tertiary level

This project explored teaching and learning of hard-to-learn threshold concepts in first-year English, an electrical engineering course, leadership courses, and in doctoral writing. The project was envisioned to produce disciplinary case studies that lecturers could use to reflect on and refine their curriculum and pedagogy, thereby contributing to discussion about the relationship between theory and methodology in higher education research (Shay, Ashwin, & Case, 2009). A team of seven academics investigated lecturers’ awareness and emergent knowledge of threshold concepts and associated pedagogies and how such pedagogies can afford opportunities for learning. As part of this examination the lecturers also explored the role of threshold concept theory in designing curricula and sought to find the commonalities in threshold concepts and their teaching and learning across the four disciplines. The research highlights new ways of teaching threshold concepts to help students learn concepts that are fundamental to the disciplines they are studying and expand their educational experiences. Given that much of the international research in this field focuses on the identification of threshold concepts and debates their characteristics (Barradell, 2013; Flanagan, 2014; Knight, Callaghan, Baldock, & Meyer, 2013), our exploration of what happens when lecturers use threshold concept theory to re-envision their curriculum and teaching helps to address a gap within the field. By addressing an important theoretical and practical approach the project makes a considerable contribution to teaching and learning at the tertiary level in general and to each discipline in particular

Learning to Teach Argumentation: Research and development in the science classroom

The research reported in this study focuses on an investigation into the teaching of argumentation in secondary science classrooms. Over a one-year period, a group of 12 teachers from schools in the greater London area attended a series of workshops to develop materials and strategies to support the teaching of argumentation in scientific contexts. Data were collected at the beginning and end of the year by audio and video recording lessons where the teachers attempted to implement argumentation. To assess the quality of argumentation, analytical tools derived from Toulmin's argument pattern (TAP) were developed and applied to classroom transcripts. Analysis shows there was development in teachers' use of argumentation across the year. Results indicate that the pattern of use of argumentation is teacher-specific, as is the nature of change. To inform future professional development programmes, transcripts of five teachers, three showing a significant change and two no change, were analysed in more detail to identify features of teachers' oral contributions that facilitated and supported argumentation. The analysis showed that all teachers attempted to encourage a variety of processes involved in argumentation and that the teachers whose lessons included the highest quality of argumentation (TAP analysis) also encouraged higher order processes in their teaching. The analysis of teachers' facilitation of argumentation has helped to guide the development of in-service materials and to identify the barriers to learning in the professional development of less experienced teachers

Classtalk: A Classroom Communication System for Active Learning

This pdf file is an article describing the advantages of using Classtalk technology in the classroom to enhance classroom communication. Classtalk technology cab facilitate the presentation of questions for small group work, collec the student answers and then display histograms showing how the class answered. This new communication technology can help instructors create a more interactive, student centered classroom, especially when teaching large courses. The article describes Classtalk as a very useful tool not only for engaging students in active learning, but also for enhancing the overall communication within the classroom. This article is a selection from the electronic Journal for Computing in Higher Education. Educational levels: Graduate or professional

Enhancing postgraduate supervision through a process of conversational inquiry

This paper outlines and begins to evaluate a process to build a critical and reflective community of postgraduate supervisors who can develop their supervision practice through reflective conversations, with the sharing of best practice and reference to research-based evidence. In 2009, the initiative of the Postgraduate Supervisors’ Conversations was set up through the collaboration of the Pro-Vice Chancellor (Postgraduate) and the Teaching Development Unit at the University of Waikato, New Zealand. We designed this initiative to complement the compulsory workshops for postgraduate supervisors that are intended to provide foundation skills. We aimed to create a professional development opportunity that could enhance supervisors’ capacity to manage the ongoing interpersonal and academic complexity of the supervision process as well as its dynamic character. This paper outlines the rationale for the Postgraduate Supervisors’ Conversations, describes its implementation and discusses the implications of an initial evaluative focus group discussion with attendees

Qualitative analysis of academic group and discussion forum on Facebook

In the present study, data was triangulated and two methods of data analysis were used. Qualitative analysis was undertaken of free-text data from students’ reflective essaysto extract socially-related themes. Heuristic evaluation was conducted by expert evaluators, who investigated forum contributions and discourse in line with contemporary learning theory and considered the social\ud culture of participation. Findings of the qualitative analysis of students’ perceptions and results of the\ud heuristic evaluation of forum participation confirmed each other, indicating a warm social climate and a conducive, well-facilitated environment that supported individual styles of participation. It fostered interpersonal relationships between distance learners, as well as study-related benefits enhanced by peer teaching and insights acquired in a culture of social negotiation. The environment was effectively moderated, while supporting student-initiative.\u

A framework for developing and implementing an online learning community

Developing online learning communities is a promising pedagogical approach in online learning contexts for adult tertiary learners, but it is no easy task. Understanding how learning communities are formed and evaluating their efficacy in supporting learning involves a complex set of issues that have a bearing on the design and facilitation of successful online learning experiences. This paper describes the development of a framework for understanding and developing an online learning community for adult tertiary learners in a New Zealand tertiary institution. In accord with sociocultural views of learning and practices, the framework depicts learning as a mediated, situated, distributed, goal-directed, and participatory activity within a socially and culturally determined learning community. Evidence for the value of the framework is grounded in the findings of a case study of a semester-long fully online asynchronous graduate course. The framework informs our understanding of appropriate conditions for the development and conduct of online learning communities. Implications are presented for the design and facilitation of learning in such contexts

The complexities of teaching 'inclusion' in higher education

This article considers how action research can support the teaching of 'Inclusion' in Higher Education. As a professional committed to improving educational practices, action research was identified as a practical research approach to study the relationship between theories and practices of inclusive education. This article will report on a short action research project that focussed on an Applied Social Science undergraduate degree unit which is taken in students' final year: 'Contemporary Issues in Exclusion and Inclusion in Education'. Student's own understanding, expectations and reflections on their learning were captured via questionnaires to directly inform teaching and assessment practices. Action research facilitated a critical lens which enabled lecturers to reconsider teaching and assessment strategies in a collaborative, participative manner and early findings indicate an improvement in student learning

Report on the Evaluation of EVS Usage and Trends at the University of Hertfordshire : February to June 2014

The Electronic Voting Systems (EVS) evaluation project for iTEAM has investigated the current level of engagement in the use of EVS across the institution in 2014. It has built on the work and outputs of the JISC supported Evaluating Electronic Voting Systems (EEVS) project in 2011-12 and the work of the iTEAM project through 2011-2013. It offers an up-to-date examination of the trends in EVS adoption and the breadth and nature of EVS use across the different academic schools. The project adopted a mixed-methods approach to evaluate usage and engagement. The starting point was a desk study to examine the existing data on numbers of EVS handsets purchased by academic schools in 2011, 2012 and 2013 and registered across the University and to explore the details from the School reports previously submitted to iTEAM. Sources of data included Information Hertfordshire and the iTEAM archive. Quantitative surveys were drawn up and information requests for student numbers were made to Senior Administrative Managers (SAM). A series of interviews were held with School-based academics including EVS Champions and Associate Deans for Learning and Teaching. Three purchasing trends for EVS handsets by different Schools were found:- slow decrease in HUM, LAW and PAM, moderate increase in BS, EDU and HSK and rapid increase in CS, ET and LMS. In terms of levels of EVS usage in 2013 -14 four different patterns emerged among the schools. These showed: slow increase (CS, LMS and PAM), slow decrease (BS, ET, EDU and HUM), rapid decrease (LAW) and no change (CA and HSK). The EVS purchasing and usage trends comply with the figures given by Rogers for his technology adoption model. Some schools are characterised by successful ongoing EVS use over several years while other school strategies for EVS, which had showed promise early on, have faltered in their use. There was some evidence that academics in STEMM subjects are more likely to engage willingly with EVS use where larger groups are taught, but this is not yet in evidence across all the STEMM groups at this university. Furthermore good practice exists and flourishes across non-STEMM subjects as well. The strategies for successful School-based EVS embedding and continued use include the following three hallmarks:- •Top-down management support for purchasing of handsets and including training for academics and administrators, and alignment with the School teaching and learning strategy. •The existence of a core of innovators and early adopters of technology including the local EVS champions, who are willing to actively engage with their fellow colleagues in sharing the potential of EVS technology. •An engagement with the pedagogical implications for changing and developing practice that the greater use of formative or summative polling and questioning requires. The immediate future of classroom technologies such as EVS offers two main directions. Firstly, there is the continuation of adopting ‘institutionally provided’ handheld devices. This is a low-cost method that can be used easily and flexibly. The other options for classroom polling rely on sufficient wifi availability in the teaching rooms and/or mobile phone signal strength/network availability and capacity. It is anticipated that the capacity issue will present fewer barriers for adoption in future, and that the future of the classroom response systems is inevitably linked to the widespread use of mobile technologies by students