Evaluating tutor training for online PBL teamwork courses in first year engineering

The use of Problem-based Learning and other collaborative pedagogies in undergraduate engineering courses is recommended by a plethora of learning theory and research on educational best-practice, particularly for Applied Sciences such as Medicine and Engineering. One barrier to implementing and sustaining these curricular and pedagogical approaches lies in the development of the appropriate knowledge and skills and a consistent and appropriate approach in the teaching team. A significant change from the traditional pedagogies employed by tutors and the training of tutors is required, if PBL and similar methods are to be effective in delivering their numerous affordances, especially in asynchronous online environments for distance learning. This paper describes the development of a strategy to train engineering tutors in online PBL facilitation, and the evaluation framework used to assess the effectiveness of this training. Results of the evaluation of training and subsequent behavioural changes of the tutors are given. The evaluation revealed a variance between the message of the training and subsequent practice. Recommendations are made about the need for ongoing tutor development and support, and the necessity of evaluation in the implementation of PBL pedagogies

Building a framework based on european quality standards for prevention and e-learning to evaluate online training courses in prevention

Prevention practitioners are required to develop their skills and competencies in response to recent advances in prevention science. It is thus important to develop training programmes which ensure the development of a prevention workforce. Aiming to respond to this need, the Specialised Training Course in Addictions Prevention is based on the European Universal Prevention Curriculum (EUPC) and delivered online. In view of this being the first online course based on the EUPC, the absence of training programmes for prevention practitioners in Portugal, and the significance of international guidelines in these two domains (prevention and online distance learning), it was considered important to create a framework which supports the evaluation and monitoring of the course. This paper presents a framework for the evaluation of the specialised training course in prevention based on two models: the European Drug Prevention Quality Standards, and the adaptation of the European Standards and Guidelines to digital programmes and courses – Considerations for Quality Assurance of E-learning Provision. Given the importance of quality assurance in contemporary society, the framework presented in this paper is a tool that may be useful for evaluating other online distance learning programmes which focus on the professional development of prevention practitioners.info:eu-repo/semantics/publishedVersio

A randomised control study about the effects of different assessment registration encouragements

Despite their potential to robustly evaluate the effects of educational interventions, Randomised Control Trials (RCTs) in education, particularly in assessment and evaluation in blended or distance learning studies, are scarce. This proposal reports the implementation of a RCT to evaluate the effectiveness of a short-scale intervention (i.e., motivational encouragement) which aimed to improve students' attendance of final exam, completion and pass rates in four online language courses. Results revealed that the specific intervention was not adequate to improve students' performance and attendance. More studies are needed to identify the types of interventions that would elicit positive learning outcomes. To assist practitioners with the implementation of RCTs in education, a learning analytics framework (Analytics4Action Evaluation Framework) is described

A Systematic Approach to Design of Distance Graduate Management Programmes

This study describes the systematic creation, application and evaluation of acomprehensive framework for the design of distance graduate programmes, the goal of which is to inform decision-making for sustainable curricula that suit the growing demand for flexible learning options. A wide range of challenges face educators, and existing models appear to be insufficient to guide such endeavours. Successful distance learning is rooted in the values of the institution and requires a significant amount of organizational support, needs assessment of stakeholders, strategic planning, implementation and evaluation. This first international study of distance masters degree programmes in Tourism and Hospitality Management (T&HM) employs an exploratory mixed method research design in a comprehensive investigation of the interrelated elements that contextualize and are part of the distance graduate curriculum. Director interviews and online surveys of alumni contribute insights into the graduate distance learning experience. A short case study within an Irish higher education institution pilots the draft framework; triangulating data by adding the perspective of traditionalinstructors transitioning into a blended learning format. This study provides a robust curriculum model linking new findings and rich eclectic sources that can assist distance programme planners in the selection of technologyenhancedapproaches to meet the unique needs and interests of learners whilebalancing change. Extending the academic plan of Stark and Lattuca (1997, 2009), this timely study offers a design framework to formatively stimulate quality interaction, foster high-level thinking and motivate both learners and instructors in a student-centred paradigm. Holistic design, not technology alone, opens the way to enhancing flexibility and programme competitiveness and resilience in a borderless academic community

An integrated approach for analysing and assessing the performance of virtual learning groups

Collaborative distance learning involves a variety of elements and factors that have to be considered and measured in order to analyse and assess group and individual performance more effectively and objectively. This paper presents an approach that integrates qualitative, social network analysis (SNA) and quantitative techniques for evaluating online collaborative learning interactions. Integration of various different data sources, tools and techniques provides a more complete and robust framework for group modelling and guarantees a more efficient evaluation of group effectiveness and individual competence. Our research relies on the analysis of a real, long-term, complex collaborative experience, which is initially evaluated in terms of principled criteria and a basic qualitative process. At the end of the experience, the coded student interactions are further analysed through the SNA technique to assess participatory aspects, identify the most effective groups and the most prominent actors. Finally, the approach is contrasted and completed through a statistical technique which sheds more light on the results obtained that far. The proposal draws a well-founded line toward the development of a principled framework for the monitoring and analysis of group interaction and group scaffolding which can be considered a major issue towards the actual application of the CSCL proposals to real classrooms.Peer ReviewedPostprint (author's final draft

Envisioning Online English Teaching in Indonesia: A Digital Autoethnographic Account

The onset of the global pandemic has become a radical turn of brick-and-mortar schooling to online distance learning. In this respect, continuous dialogue, and evaluation around the issue of online learning should be nurtured, particularly from actual pedagogical practices. Drawing on a digital autoethnographic account of the author, this article explores everyday online English teaching in tertiary education. I collected data using textual, visual, and aural experiences, corroborated by Zoom auto-recorded chats and screenshots as the artefacts of my online learning and teaching activities. The data were analyzed using the Community of Inquiry (CoI) framework that focused on social, cognitive, and teaching presence of online learning. It is found that social presence, mainly caused by unstable internet connections, has been severely impacted, which was detrimental to class interactivity, learning goals, and learning assessment. The autoethnographic account in the current study depicts micro-level pedagogical activities that were then reflected to identify some concerns at meso-(institutional), and macro-level (governmental/national) that need to be addressed. The study implies that collaborative works and assessments of online learning shall be sustained by teachers and pertinent stakeholders. Eventually, the significance of this article lies beyond the explication of everyday disrupted pedagogy, as it echoes wider socio-economic-political realities in Indonesia

Virtual world technologies provide platform for interaction between on-campus and online students: A case study

There has been a great deal of attention paid to efforts by educators to integrate teaching methodologies and strategies between face to face and online classrooms in an effort to maximize learning by combining delivery modalities. Recent studies point to students not only learning more when online capabilities were added to traditional courses, but also increasing their level of interaction thereby improving the students‟ sense of satisfaction with the courses taken. These studies tend to isolate deliveries to either all online classes and students or to all on-campus classes and students, without taking into account the more recent movement of blending teaching methods and crossing over the barriers between online and face to face students. So, what happens when online students are given the opportunity, through the use of virtual world technologies, to engage with students attending traditional on-campus sessions? The purpose of this case study is to evaluate the use of virtual world technologies as a platform for the conduct of synchronous and asynchronous classroom activities. A framework for conducting an undergraduate „Technology Project Management‟ course is presented that includes delivery approaches to students from both online (Distance Education) class offerings and on-campus (Face-to-face) class offerings. Stand-alone, asynchronous, or synchronous learning modalities are incorporated into the framework. To evaluate the framework, a composite evaluation of survey, responses, and assessments analysis are examined. Discussion includes the challenges of developing and presenting this framework as well as managing the instructional complexities involved in the planning and execution of virtual world interactions in the classroom setting. Further discussion includes use of virtual teaming sessions and self-paced online case studies; incorporation of in-world interactive learning modules; assessment of impromptu, in-world, e-learning sessions in the form of informal student interactions; and use of online text and voice chat capabilities. The data indicates, surprisingly, that the learning curve for students was not as steep as expected and that overall the students felt reasonably comfortable with the introduction of this technology into their learning environments. Finally, there is little evidence, beyond pedagogical preference, of adverse effects in using this framework while there were some initially positive small gains in the online students‟ performance related to learning objectives using the virtual world technologies

Reflecting on the Need to Evaluate Courses Using ICT within a Pedagogical Framework

In the rush by academics to incorporate online learning technologies into tertiary education courses, pedagogical imperatives are often ignored. This paper addresses the need to step back and evaluate courses holistically within a structured framework that allows for the determination of appropriateness of use of one or more technologies. This paper attempts to add to the body of knowledge on successful and appropriate integration of technology into courses by investigating how email technology can support learning. It uses Reeves’ fourteen dimensions of pedagogy for computer based learning to evaluate a course that uses email as the principle communication tool for teams of distance education and internal students. The outcomes of the evaluation indicated that use of a technology can be supportive of learning needs if used appropriately

Design guidelines for self-assessment support for adult academic distance learning

Menendez Blanco, M., Van der Veer, G., Benvenuti, L., & Kirschner, P. A. (2011). Design guidelines for self-assessment support for adult academic distance learning. In H-J Shalin (Ed.), Constructing self-discovery learning spaces online: scaffolding and decision making technologies (pp. 169-198). Hershey, PA: IGI Global.This contribution focuses on adult distance learning. Based on experiences at the Open University of the Netherlands we investigate specific problems that our students have with self-assessment and metacognition while studying. Starting from a literature review and complementing this with available student data from our teaching research center, we developed a conceptual framework that was iteratively adjusted and assessed by a questionnaire study and interviews. This allowed us to develop design guidelines for self-assessment support in distance learning environments. These guidelines were reviewed by experts. The input from the experts was used to modify the guidelines and iterate until they were considered complete. Tangible designs (i.e., mock-ups) for each of the self-assessment methods were proposed. These tangible designs were prototyped for later evaluation. Finally, we provide our conclusions and propose recommendations for actual application and systematic design

CHINA'S K12 ONLINE EDUCATION RESEARCH UNDER THE EPIDEMIC : Satisfaction Analysis on K12 Distance Learning

The value that the Internet brings to the K12 education industry is reflected in four aspects: supply chain, efficiency improvement, experience optimization and clear output. In this research, we first briefly introduce the K12 online education and its history of development. Additionally, we discuss the general features of the online educations according to analyzing the comments under the relevant applications. Then we analyze the market of K12 online education in China, including the dominant applications in the current market, the business model, and the supply chain of the online education. Next, we analyze the evaluation framework for online education which is proposed by Quality Matters as a model. Even though it is still very difficult to design a model that can perfectly evaluation the quality of online education, however, at least, according to the analysis, we see the possibility. To analyze the satisfaction of the distance learning in K12, we design a questionnaire and receive responses from 218 student and more than 100 responses from their parents. Based on the data, we find the result is relatively positive. Evidence shows the potential bright future of online education. Therefore, we use SWOT model to analyze and predict the future development of K12 online education from different dimensions. Finally conclude the research