Benchmarks for MOOCs: the OpenupEd quality label

This paper reports on the OpenupEd Quality Label, a quality assurance process for MOOCs (massive open online courses) developed for the European OpenupEd portal (http://www.openuped.eu). The OpenupEd Quality Label is offered as a way of ensuring that MOOCs offer a good quality educational experience. It adopts a quality enhancement approach, based on initial self-assessment against benchmark indicators, followed by a review leading to an improvement action plan. It is designed to complement both an institutional course approval process, and ongoing evaluation and monitoring of courses in presentation. The overall approach and the benchmarks are derived from the E-xcellence e-learning quality projects (E-xcellencelabel.eadtu.eu). The benchmarks are divided into two groups: course-level and institutional. There are 11 benchmarks that apply at course-level and which should be checked for each MOOC. There are 21 institutional benchmarks grouped into six areas: Strategic Management, Curriculum Design, Course Design, Course Delivery, Staff Support and Student Support. The institutional benchmarks are only reviewed periodically. The benchmarks are supported by the E-xcellence manual which provides background material and more detailed indicators of good practice, with a supplement for MOOC-specific aspects. A ‘quick scan’ checklist is provided to capture the extent to which each benchmark is achieved. Other templates are provided to gather evidence and prepare an action plan. OpenupEd has further distinctive features intended to enhance the quality of MOOCs. The initial providers are members of EADTU (European Association of Distance Teaching Universities) and will bring extensive experience of open and distance learning to bear on MOOCs. Their commitment to ‘open education’ acts to remove unnecessary barriers to learning, and OpenupEd MOOCs should demonstrate specific values: openness to learners, digital openness, a learner-centred approach, support for independent learning, media-supported interactions, recognition options, a focus on quality, and embracing a spectrum of diversity. The OpenupEd label should benefit all stakeholders in MOOCs. Students can be reassured about the experience they are committing to. Employers can recognise the content and skills demonstrated by a MOOC certificate. MOOC authors can achieve recognition for their input. Institutions can protect their brand reputation. Funders can be reassured that products are worthwhile. Quality agencies, who work on behalf of all the above parties, may find their task eased

E-xcellence methodology: lessons learned over ten years of development and implementation

The E xcellence methodology for quality assurance of e learning in higher education has developed into a flexible and effective approach to quality assurance. It has proved suitable to meet the needs of a diverse range of institutions and national quality assurance regimes, and is applicable to both distance and blended modes. The E xcellence resources (http://e-xcellencelabel.eadtu.eu/) include a quality manual, guidance for institutions and assessors, and an online Quick Scan self-evaluation tool. The manual was updated in 2012 and 2016 to reflect evolving changes in practice. The E-xcellence methodology provides 35 benchmark statements, grouped under six headings: Strategic Management, Curriculum Design, Course Design, Course Delivery, Staff Support and Student Support. Higher education institutions self-assess their capabilities against each of the benchmark statements on a four-point scale. They also prepare a roadmap of future actions which can be mapped to benchmark statements. A visit by external reviewers enriches and complements the self-assessment. This study is based on an analysis of E xcellence self-evaluations and roadmaps at twenty higher education institutions. Tabulating those benchmarks that are rated as not yet adequate, and those which attract the most planned actions, highlights the aspects that institutions have found most challenging as they develop and implement online and blended learning programmes. This profiling exercise indicates that institutions regard issues of strategy, curriculum design and staff support as presenting the greatest challenges. Particular problems include staff workload and developing an online academic community for students. In contrast, the provision of reliable IT systems and hardware is unproblematic

Learning and peer feedback in shared online spaces

Peer feedback, where students evaluate and give comments on each other's work, is a valued form of learning activity. Giving and receiving peer feedback, particularly if assessed, encourages students to engage with the assessment criteria and reflect on their work, promoting a deeper understanding. This paper introduces a project to investigate approaches to using peer feedback in shared online spaces, as part of distance learning courses. One approach uses a wiki, where students each contribute their work and other students provide evaluative comments and advice. The other approach uses OpenStudio, a 'studio' environment where students upload self-produced media objects, such as photographs and presentations, and comment on each other's work. Both approaches have produced high levels of student engagement. An initial examination of peer feedback in a wiki indicates that the quality of feedback is high, but students do not make as much use of it as they might. Survey results for a course using a studio (with feedback from peers but not tutors) reveals some students who, while they enjoy the sharing activities, believe they need 'expert' opinion. This project will evaluate these two approaches to online peer feedback in order to identify successful features in terms of (a) engaging students and (b) promoting student learning. The project will also identify areas for improvement. The first stage will map the uses of peer feedback in Computing and IT courses of the UK Open University. This stage will document: the educational objectives and intended learning outcomes of the activities; what students share and discuss in the activities; and, if the activities are assessed, how this is done. The second stage will evaluate the peer feedback activities in three courses which use online spaces for sharing and peer feedback. Data will be collected from students and tutors through focus groups and online surveys. The course forums will also be used to invite students to give their views. A qualitative analysis will be undertaken of online artefacts and interactions. Skills in evaluating the work of others and giving constructive feedback are important, both in education and beyond. This project will help develop these skills in online learning students. It will do this by generating case studies, investigating learning designs and proposing best practice guidelines for using and assessing online peer feedback

Learning and peer feedback in shared online spaces

Peer feedback, where students evaluate and give comments on each other's work, is a valued form of learning activity. Giving and receiving peer feedback, particularly if assessed, encourages students to engage with the assessment criteria and reflect on their work, promoting a deeper understanding. This paper introduces a project to investigate approaches to using peer feedback in shared online spaces, as part of distance learning courses. One approach uses a wiki, where students each contribute their work and other students provide evaluative comments and advice. The other approach uses OpenStudio, a 'studio' environment where students upload self-produced media objects, such as photographs and presentations, and comment on each other's work. Both approaches have produced high levels of student engagement. An initial examination of peer feedback in a wiki indicates that the quality of feedback is high, but students do not make as much use of it as they might. Survey results for a course using a studio (with feedback from peers but not tutors) reveals some students who, while they enjoy the sharing activities, believe they need 'expert' opinion. This project will evaluate these two approaches to online peer feedback in order to identify successful features in terms of (a) engaging students and (b) promoting student learning. The project will also identify areas for improvement. The first stage will map the uses of peer feedback in Computing and IT courses of the UK Open University. This stage will document: the educational objectives and intended learning outcomes of the activities; what students share and discuss in the activities; and, if the activities are assessed, how this is done. The second stage will evaluate the peer feedback activities in three courses which use online spaces for sharing and peer feedback. Data will be collected from students and tutors through focus groups and online surveys. The course forums will also be used to invite students to give their views. A qualitative analysis will be undertaken of online artefacts and interactions. Skills in evaluating the work of others and giving constructive feedback are important, both in education and beyond. This project will help develop these skills in online learning students. It will do this by generating case studies, investigating learning designs and proposing best practice guidelines for using and assessing online peer feedback

Students' feelings in social and collaborative learning: some case studies

In HE institutions within Europe and beyond, there is considerable interest in adopting collaborative approaches to online learning. These approaches give students opportunities to learn from each other as they study online, and also to develop employability skills in working with others. Many educators are therefore keen to adopt these new methods in their courses; however not all students are enthusiastic about them. For example, students may be anxious about the prospect of taking part in online collaborative learning activities, particularly if they do not know the other students, and if their grades are dependent upon the success of the collaboration. Students experience a range of different feelings in relation to online collaborative learning (Webster & Hadwin, 2013). Some feelings (anxiety or frustration) may be a largely negative experience, while others (sense of achievement or enjoyment) are largely positive. Recent research shows the effects of these different emotions on students’ engagement and success are not as obvious as might be assumed (Hilliard, 2017). We present two case studies of modules from the UK Open University which have integrated collaborative activities: a second-year (9-month part-time) undergraduate module on Information Technology; and a short (10-week part-time) module on Digital Photography. We consider how the feelings of students about the collaborative activities change from before they start, to during the collaboration, and finally after the activities have finished. The findings are used to identify how educators can best support their students in undertaking online collaboration with confidence and maximising the benefits gained. Hilliard, J. (2017). Students’ Perceptions And Experiences Of Anxiety In An Online Collaborative Project. MRes thesis The Open University. [http://oro.open.ac.uk/52546/] Webster, E. A., & Hadwin, A. F. (2013). Regulating emotions during computer-supported collaborative problem solving. Annual Conference of the Canadian Society for the Study of Education, Victoria, British Columbia

Quality Assessment for E-learning: a Benchmarking Approach (Third edition)

The primary purpose of this manual is to provide a set of benchmarks, quality criteria and notes for guidance against which e-learning programmes and their support systems may be judged. The manual should therefore be seen primarily as a reference tool for the assessment or review of e-learning programmes and the systems which support them. However, the manual should also prove to be useful to staff in institutions concerned with the design, development, teaching, assessment and support of e-learning programmes. It is hoped that course developers, teachers and other stakeholders will see the manual as a useful development and/or improvement tool for incorporation in their own institutional systems of monitoring, evaluation and enhancement

Online conversations around digital artefacts: the studio approach to learning in STEM subjects

Studio-based learning provides a model that can be adapted for online learning. In conventional teaching settings, studio-based learning follows an apprenticeship model where students work independently or in groups, under the guidance of a tutor, using real-world activities. The ‘Using OpenStudio in STEM learning’ project has been established to evaluate the use of online studio-based learning in the Open University (UK). This paper reports our findings from the first two phases of the project which gathered data from educators who present the modules and also from a survey of students. Educators representing distance learning modules from a range of STEM disciplines including Computing and IT, Design, Engineering and Environmental Technology participated in a workshop to share information about the use of OpenStudio on their modules. A simple model of OpenStudio activities was derived from the workshop to illustrate the process of 'showing and sharing', viewing and reviewing', commenting and critiquing', and 'reviewing and reflecting' involved. Two Computing and IT undergraduate modules were then selected for more detailed analysis, one at level 1 (first year) and another at level 3 (third year). Both quantitative and qualitative data were gathered from samples of students on these modules and analysed. Comparisons between the OpenStudio model, the survey findings and Kolb’s Experiential Learning model (1984) revealed the range of student views and the diversity of students’ experiences of the learning activities, and provided some thought-provoking insights into student behaviour in carrying out the OpenStudio activities. The data suggest that students enjoy the OpenStudio activities, especially the visual nature of artefacts and the idea that shorter comments may be made, rather than longer more discursive pieces of writing. In addition to learning about their subject area, students are also learning how to give feedback to their peers and how to use the feedback they receive, both of which are important skills. Many students are confident in their own ability and are able to evaluate the feedback they receive. However, some students may lack confidence in their own ability to give feedback on the work of their peers, particularly at level 1. Importantly, there needs to be an opportunity to complete the cycle of the experiential learning model in the activity by allowing students to produce another artefact. The experiential nature of the online studio activity presents an opportunity for students to reflect-in-action as well as reflect on their actions (Schön, 1983)

Active participation in synchronous online learning

Online synchronous education has recently been vital in replacing face-to-face teaching during the pandemic. Although there is increasing use of synchronous technologies in higher education, there are still challenges which need to be addressed. The most significant of these is lack of active participation by students. Even where educators are experienced in running synchronous online learning sessions, previous research has shown that these sessions tend to be didactic in nature, and interactive elements are often not used. Students’ unwillingness to use audio or video channels is one factor, which denies educators the non-verbal feedback intrinsic to face-to-face teaching. In general, there may be a lack of student engagement, with students attending but in a passive capacity, with a reluctance to contribute. There is a need to address two related challenges: designing synchronous online sessions which have active learning built in; and overcoming students’ inhibitions regarding active participation. This article discusses these topics, based on prior research and practice. It then briefly presents selected data from a student survey at the UK Open University. The survey collected quantitative and qualitative data to gain an understanding of students’ experiences and views about online synchronous sessions: for example, what value do students place on active participation, and why? The aim of this investigation is to make online synchronous learning more interactive, engaging and effective

The Open Networking Lab: an open online course for experiential learning of computer networking

The Open Networking Lab project (https://onl.kmi.open.ac.uk/) aims to provide open online resources to enable anyone to learn the basics of computer networking. The project is hosted at The UK Open University and is supported by funding from UfI (www.ufi.co.uk) as part of their ‘VocTech Impact 2017’ funding initiative for vocational learning using digital technologies. Central to the project is the PT Anywhere network simulation software (Mikroyannidis et al. 2017) based on Cisco’s powerful Packet Tracer simulator. Learners can use PT Anywhere to develop their skills in solving computer networking problems. The ultimate aim is to enable as many learners as possible, regardless of prior educational background, to access employment in computer networking - an area which is in high demand from industry. The Open Networking Lab project is developing a Badged Open Course, which will be hosted on the Open University’s OpenLearn platform, where it will be accessible without cost to any learner or educator worldwide. Initial development is being carried out using the ‘sister’ platform OpenLearn Create so that the learning resources can be iteratively developed, piloted and improved prior to launch on OpenLearn itself. The Open Networking Lab course and resources will be evaluated with learners and teachers from Further Education colleges within the Cisco Networking Academy (www.netacad.com). The evaluation will involve hundreds of learners at different colleges within the UK. Data will be gathered from learners using surveys and observation, and from teachers via interviews. PT Anywhere and OpenLearn also provide various kinds of learning analytics. This data, with appropriate ethical considerations, will form a key part of the evaluation. Using PT Anywhere enables an experiential and practical approach to learning (Kolb, 1984; Brown et al., 1989). Learners will be shown, primarily via videos, screencasts and animations, how computer networks are set up and configured. They will then try out these ideas for themselves using the PT Anywhere simulator. Quizzes and other forms of assessment will enable learners to demonstrate that they have gained specific skills; they will then be able to claim corresponding digital badges. This package of activity-based learning should help learners feel motivated and engaged, and enable them to gain a sense of achievement as they progress through the online course. References: Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18 (1), 32-42. Kolb, D.A. (1984). Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall. Mikroyannidis, A., Gomez-Goiri, A.; Smith, A. and Domingue, J. (2017). Online experimentation and interactive learning resources for teaching network engineering. In: 2017 IEEE Global Engineering Education Conference (EDUCON), 25-28 Apr 2017, Athens, Greece, IEEE, pp. 181–188

Remote Pair Programming

Pair programming is routinely used in industry and taught in face-to-face programming classes. Research indicates that it improves self-confidence and students’ programming, communication and team working skills. We investigate the technology, barriers, and soft-skill benefits for distance-learning students who pair program with a remote partner online. In one study, students watched two tutors pair programming and then performed a pair programming task remotely with a student partner. Students felt significantly more positive with the latter compared to the former. As universities strive to provide a good student experience during a pandemic, these findings highlight the value of active remote pair programming using standard online communication tools