Pragmatic meta analytic studies: learning the lessons from naturalistic evaluations of multiple cases

This paper explores the concept of pragmatic meta‐analytic studies in eLearning. Much educational technology literature focuses on developers and teachers describing and reflecting on their experiences. Few connections are made between these experiential ‘stories’. The data set is fragmented and offers few generalisable lessons. The field needs guidelines about what can be learnt from such single‐case reports. The pragmatic meta‐analytic studies described in this paper have two common aspects: (1) the cases are related in some way, and (2) the data are authentic, that is, the evaluations have followed a naturalistic approach. We suggest that examining a number of such cases is best done by a mixed‐methods approach with an emphasis on qualitative strategies. In the paper, we overview 63 eLearning cases. Three main meta‐analytic strategies were used: (1) meta‐analysis of the perception of usefulness across all cases, (2) meta‐analysis of recorded benefits and challenges across all cases, and (3) meta‐analysis of smaller groups of cases where the learning design and/or use of technology are similar. This study indicated that in Hong Kong the basic and non‐interactive eLearning strategies are often valued by students, while their perceptions of interactive strategies that are potentially more beneficial fluctuate. One possible explanation relates to the level of risk that teachers and students are willing to take in venturing into more innovative teaching and learning strategies

On-line assessment for e-learning: options and opportunities

The desire to produce educational multi-media packages of ever greater sophistication is such that other, more problematic, elements of e- or on- line learning receive less attention by academics and courseware developers alike. One such problematic area is that of assessment, which is surprising, as e-learning assessment procedures are more critical in defining the learning that takes place. However, because e-learning can create a much richer, more varied active learning experience than would normally occur via the passive didactic teaching mode currently utilised in most universities and centred on the use of the lecture, it also has the potential to provide new and innovative assessments modes and systems. The extent to which the potential of innovatory assessment is realised via e-learning depends on two factors. First, the level of computer component and interactivity utilised in elearning. Second, the attitude of academic staff towards their teaching role, and, more specifically, how they operate within an elearning environment. The speed of the development in online technologies and techniques is such that the information given here will not provide all the answers, it should, nevertheless, enable some of the correct questions to be pose

FORGE: An eLearning Framework for Remote Laboratory Experimentation on FIRE Testbed Infrastructure

The Forging Online Education through FIRE (FORGE) initiative provides educators and learners in higher education with access to world-class FIRE testbed infrastructure. FORGE supports experimentally driven research in an eLearning environment by complementing traditional classroom and online courses with interactive remote laboratory experiments. The project has achieved its objectives by defining and implementing a framework called FORGEBox. This framework offers the methodology, environment, tools and resources to support the creation of HTML-based online educational material capable accessing virtualized and physical FIRE testbed infrastruc- ture easily. FORGEBox also captures valuable quantitative and qualitative learning analytic information using questionnaires and Learning Analytics that can help optimise and support student learning. To date, FORGE has produced courses covering a wide range of networking and communication domains. These are freely available from FORGEBox.eu and have resulted in over 24,000 experiments undertaken by more than 1,800 students across 10 countries worldwide. This work has shown that the use of remote high- performance testbed facilities for hands-on remote experimentation can have a valuable impact on the learning experience for both educators and learners. Additionally, certain challenges in developing FIRE-based courseware have been identified, which has led to a set of recommendations in order to support the use of FIRE facilities for teaching and learning purposes

Interoperability Between ELearning Systems

Online assessments are an integral part of eLearning systems that enhance both distance and continuous education. Although over two-hundred and fifty eLearning applications exist, most educational institutions are trapped with a particular vendor primarily due to lack of in test-question sharing features. This paper highlights the evolution of eLearning systems whilst detailing the two most prominent objective test question standards, namely, QML and QTI. An analysis conducted amongst software houses which are involved in the development of eLearning systems confirms the fact that most applications make use of proprietary formats and clearly shows a lack of import and export options amongst other features

An evaluation of constructivism for learners with ADHD: Development of a constructivist pedagogy for special needs

We examine whether constructivist eLearning tools can be used to help learners cope with special educational needs, such as difficulties with attention and concentration. Preliminary work is reported here, in which we seek to determine the reasons why a constructivist approach is difficult for learners with ADHD. This work is intended to lead to recommendations of how learners with ADHD could benefit from constructivist eLearning systems, e.g. through the managed use of multimedia technology. A preliminary model has been developed that illustrates the areas in which constructivist pedagogies need to address the limitations of ADHD learners. Further work will expand this model and eventually test it in a real environment (e.g. in a school with ADHD learners). The outcome will encourage a reconsideration of existing multimedia theories as they relate to learners with special needs, and provide new directions in order to support learners with ADHD

Learning from the past and looking at the future. Closing the evaluation-revision-implementation cycle in an elearning module

This paper will outline the importance of implementing an instructional design model which incorporates a continuous cycle of evaluation, revision and implementation. The context is a case study of a distance learning module which was first presented in 1993 as an introductory module for students taking an Oscail BA programme. As the majority of Oscail students are adults returning to education after a long period of absence from formal education, providing learners with a supportive learning environment which would ease re-entering the educational world has always been one of Oscail’s key goals. Between 1993 and 2002, the Introductory Module was offered in the traditional distance education format and was subject to regular evaluation and revision (Lorenzi, MacKeogh and Fox, 2004; MacKeogh and Lorenzi, 2005). By early 2000, the wider accessibility of technology encouraged Oscail to combine the need to prepare students for university study with the development of e-learning skills. In 2002 the Introductory Module was reconstituted as the SPEL (Student Passport for E-learning) module and was presented for three years. A review of the module in 2006 indicated shortfalls in what is described in the literature as the bolt-on ‘skills approach’ which makes use of ‘a set of atomized skills – removed from subject specific domains’ (Lea & Street, 1998: 158). This prompted a rethink of the overall approach to preparing students for study, and resulted in a decision to adopt an embedded approach to e-skills development. It was hoped that a task-based approach to applying skills to subject specific activities would improve the quality of the learning experience and ultimately result in better retention rates. A review of the implementation of the embedded approach in the first year reveals that while students reported an increase in skills development over the module, and identified a number of positive features, the approach did not succeed in increasing retention. Detailed evaluation including surveys of tutors and students as well as online focus groups have identified a number of factors which militated against the achievement of some of the goals of the module. These included the scheduling of activities, workload, a shorter than usual academic year, insufficient tutor training and student induction, and administrative support. This paper will start with a brief overview of instructional design systems and the models used by Oscail in developing its elearning programmes. We will then outline the way in which through a cycle of implementation, evaluation and revision, the original introductory module evolved into a full year-long module comprising an embedded portfolio of study skills tasks. We will describe the outcomes of the embedded SPEL programme and will outline the evaluation process that has led to further revisions. We will discuss the specific issues that have emerged from the evaluations and will show what measures have been put in place to overcome the difficulties encountered. We will conclude with some observations on the changing roles of stakeholders in the context of new elearning approaches

Technology in learning

An Association for Learning Technology and Technology Enhanced Learning Research Programme response to some questions from the Department of Business Innovation and Skills, with a foreword by John Cook and Richard Noss

Teaching and learning in virtual worlds: is it worth the effort?

Educators have been quick to spot the enormous potential afforded by virtual worlds for situated and authentic learning, practising tasks with potentially serious consequences in the real world and for bringing geographically dispersed faculty and students together in the same space (Gee, 2007; Johnson and Levine, 2008). Though this potential has largely been realised, it generally isn’t without cost in terms of lack of institutional buy-in, steep learning curves for all participants, and lack of a sound theoretical framework to support learning activities (Campbell, 2009; Cheal, 2007; Kluge & Riley, 2008). This symposium will explore the affordances and issues associated with teaching and learning in virtual worlds, all the time considering the question: is it worth the effort

Transforming pre-service teacher curriculum: observation through a TPACK lens

This paper will discuss an international online collaborative learning experience through the lens of the Technological Pedagogical Content Knowledge (TPACK) framework. The teacher knowledge required to effectively provide transformative learning experiences for 21st century learners in a digital world is complex, situated and changing. The discussion looks beyond the opportunity for knowledge development of content, pedagogy and technology as components of TPACK towards the interaction between those three components. Implications for practice are also discussed. In today’s technology infused classrooms it is within the realms of teacher educators, practising teaching and pre-service teachers explore and address effective practices using technology to enhance learning