Evaluation in a project life‐cycle: The hypermedia CAMILLE project

In the CAL literature, the issue of integrating evaluation into the life‐cycle of a project has often been recommended but less frequently reported, at least for large‐scale hypermedia environments. Indeed, CAL developers face a difficult problem because effective evaluation needs to satisfy the potentially conflicting demands of a variety of audiences (teachers, administrators, the research community, sponsors, etc.). This paper first examines some of the various forms of evaluation adopted by different kinds of audiences. It then reports on evaluations, formative as well as summative, set up by the European CAMILLE project teams in four countries during a large‐scale courseware development project. It stresses the advantages, despite drawbacks and pitfalls, for CAL developers to systematically undertake evaluation. Lastly, it points out some general outcomes concerning learning issues of interest to teachers, trainers and educational advisers. These include topics such as the impact of multimedia, of learner variability and learner autonomy on the effectiveness of learning with respect to language skills

The Effects of Multimedia Learning Materials Quality on Knowledge Acquisition

Different learning environments offer l earners various kinds of multimedia learning materials (MLMs). Although the usage of multimedia has proven to foster meaningful learning, a multimedia resource will not necessarily contribute to the teaching-learning process unless proper attention is paid to its quality. This paper explores issues regarding the assessment of multimedia learning materials (MLMs) quality as well as the relationship between MLM quality and knowledge acquisition. We present a research experiment that involves low-quality and high-quality MLMs implemented in the learning management system (LMS) Moodle of a polytechnic school course. The quality of MLMs was evaluated by means of the LORI assessment tool adapted for the purpose of this research. The analysis of research data shows that MLMs developed according to the principles of multimedia learning and principles for reducing cognitive load were perceived as being of higher quality than those not developed using multimedia principles. Furthermore, students’ usage of high-quality MLMs during treatment resulted in better knowledge acquisition indicated by significantly higher scores in knowledge assessment

Open educational resources and practices in China: a systematic literature review

The concepts of Open Educational Resources (OER) and Open Educational Practices (OEP), regarded as two pillars of the broader open education movement, have been evolving since the concept of OER was first coined in the 2012 Paris Declaration. Several research studies have been conducted to investigate the impacts of OER and OEP adoption and implementation in universities. However, most of those studies have focused on western and developed countries, and little information is known about developing countries, especially Asian ones. Particularly, China was one of the first Asian countries to adopt open education and its related strategies following the MIT OpenCourseWare conference in Beijing in 2003. This study conducts a systematic literature review to investigate the current state of the art of OER and OEP in China. The findings show that several governmental, organizational, and institutional initiatives have been launched to facilitate OER adoption in China. They also show that while several OEPs have been implemented, there is still a continuous need to work on these practices and further investigate their impacts on learning outcomes and behaviors, as no current reviewed study has done so. Finally, a generic framework of OER and OEP challenges is presented along with recommendations to further enhance the adoption of OER and OEP in Chin

Evaluating the introduction of CBA into the learning, teaching and assessment strategy of the diagnostic radiography course at Sheffield Hallam University

feedback have been introduced into the BSc (Hons) Diagnostic Radiography course at Sheffield Hallam University (SHU). CBA was selected for a number of reasons: it is a natural progression of the current use of information technology (IT) in learning and teaching; it further develops essential IT skills; it allows real medical images to be viewed; it tests students prior to placement, therefore helping to ensure appropriate levels of knowledge and understanding, and consequently reducing the burden of supervising radiographers; students requiring additional tutor support will be identified at this stage; it is thought to be an efficient use of lecturer time; it is anticipated that there will be a reduction in potential biases surrounding the marking process; an ‘easy’ to use CBA system is readily available in SHU’s adopted virtual learning environment, Blackboard 5.5; SHUS’s IT infrastructure is more robust than ever before. Although CBA is only one method within a diverse overall learning, teaching and assessment strategy, it is new and innovative and is therefore being evaluated to assess the appropriateness and effectiveness of implementation

Openness and education: a beginner’s guide

While recent high-profile developments such as Massive Open Online Courses (MOOCs) have placed renewed emphasis on the idea of openness in education, different notions of open in relation to education can be found dating back to the 1960s. This document builds on recent research undertaken to trace this history, acknowledging that there is no single root of ‘open’ in this context, but to map the different ways of thinking about open education that have come to bear on the field we see today. Mapping of themes across time aims to provides those new to the field with a useful overview of the history and introduction to the concept of openness, and ways to explore the literature further. Each section of this document will summarise the nature of one of the themes, and its relationship to the broader network. Additionally, the document provides an annotated bibliography, through summaries of five of the most influential publications across a range of perspectives in each theme

An Evaluation in China and the UK of a Virtual Laboratory in Materials Science

This thesis describes the design, development and evaluation of virtual technology-based courseware—Virtual Laboratory in Materials Science used in a tension test that forms part of the course of “Properties of Materials” taught to most first year engineering undergraduates. The effectiveness of this specially developed courseware for virtual laboratory work was evaluated in a pretest—posttest comparative study of the performance of designated subjects between two treatment groups that worked with the courseware and two control groups that worked with a real testing machine. All participants were engineering students studying either with the United Kingdom Open University (UKOU) or with the China Radio & TV University system (CRTVUs). The findings showed that most students enjoyed using the courseware because the simulated real experiment environment can make them feel personally on the scene. Among all the media used in the courseware, 3D and images were more favoured and more helpful to the students in terms of the usability of the courseware. Sounds were not seen as particularly helpful although some participants agreed sounds made the courseware more interesting. The result of the research indicated that using the Virtual Laboratory in Materials Science could make a contribution to students’ understanding of the tensile testing. This evaluation clearly revealed that virtual reality (VR) and virtual environments (VE) technology can facilitate and support engineering course learning or even make learning fun. But when used at times when there were steps that need lots of manipulation, the virtual experiment still had some difficulties that need to be resolved. The research showed that the potential of the virtual testing courseware in promoting concept teaching needs to be tapped further

Facilitating Computer-Supported Collaborative Learning with Question-Asking Scripting Activity and its Effects on Students’ Conceptual Understanding and Critical Thinking in Science

Computer-supported collaborative learning (CSCL) provides an environment that enhances social interaction and shared knowledge construction among students. However, limited research has examined CSCL reinforced by question-asking scripting activity. This research investigated the effects of CSCL with question-asking scripting activity on the development of conceptual understanding and critical thinking in science. Moreover, the research design was a three-group pre-test-post-test quasi-experimental study in which the research sample were 106 Grade 7 students. The only difference between the control and the experimental groups was the exposure to CSCL. The experimental groups were exposed to CSCL approaches: one without scripting while one was exposed with question-asking scripting activity. Results revealed that CSCL approaches significantly affected the development of students’ conceptual understanding and critical thinking. Specifically, CSCL with scripting stimulated more intellectual discussion which allowed learners to deepen lesson comprehension and improve their critical thinking skills. Insights on the innovations through technology integration, collaborative inquiry learning, and question-asking activity to enhance science education were also discussed. The findings of this study have important implications for future practice

Self-Regulation and Cognitive Load as Mediating Factors for Tailored Interactive Multimedia Instruction

The primary purpose of this dissertation was to explore whether self-regulation or cognitive load have mediating effects on both learning experiences and learning effectiveness in tailored versus non-tailored interactive multimedia instructional (IMI) training. Although, there is a plethora of literature looking at the impact of cognitive load in IMI (Clark, 2008; Mayer, 2005; Mayer, 2008; Mayer, Griffith, Jurkowitz, & Rothman, 2008; Sweller, 2011) or looking at self-regulation (Pintrich, 2000a, 2000b; Schunk, Meece, & Pintrich, 2012; Zimmerman et al., 2000) separately, there is limited literature that looks at self-regulation and cognitive load in tailored IMI instruction, and even less literature examining these variables within the military population. Participants were soldiers both junior and senior in their military career attending a leadership based course at two different Army installations. Several measures were used to collect data both prior to (MSLQ, demographics, pretest) and after (learning experiences survey, NASA-TLX, posttest) soldiers engaged in the IMI training. Data analysis involved the use of quantitative statistical procedures to test levels of significance, along with the magnitude of relationships between the different variables. Results indicate that individuals who came into the training with self-regulation skills tended to score better on the pretest but by the time they reached the posttest these differences did not appear to have a significant impact on learning. Additionally, self-regulation and cognitive load appeared to have different effects on participants depending on their learning experiences and career experience

Teacher and Student Perceptions of DynEd Multimedia Courseware: An Evaluation of CALL in an American Technical College

This study examines the perceptions of teachers and students using the DynEd Multimedia Courseware in adult ESL workshops in an American technical college setting. The goals of the study were to determine (a) the teachers’ perceptions of their training to facilitate DynEd, (b) the teachers’ and students’ perceptions of the facilitators’ role in supporting students, and (c) the teachers’ and students’ perceptions of DynEd’s appropriateness for adult learners at our institution. Data from questionnaires and focus group interviews were analyzed using Chapelle’s (2001) Criteria for CALL Task Appropriateness as the conceptual framework. Findings suggest that both teachers and students need training and support to use DynEd effectively. Findings also indicate that the students’ perceptions of DynEd are more positive than the teachers’ perceptions