Chapter 5: Evaluation

The OTiS (Online Teaching in Scotland) programme, run by the now defunct Scotcit programme, ran an International e-Workshop on Developing Online Tutoring Skills which was held between 8–12 May 2000. It was organised by Heriot–Watt University, Edinburgh and The Robert Gordon University, Aberdeen, UK. Out of this workshop came the seminal Online Tutoring E-Book, a generic primer on e-learning pedagogy and methodology, full of practical implementation guidelines. Although the Scotcit programme ended some years ago, the E-Book has been copied to the SONET site as a series of PDF files, which are now available via the ALT Open Access Repository. The editor, Carol Higgison, is currently working in e-learning at the University of Bradford (see her staff profile) and is the Chair of the Association for Learning Technology (ALT)

E-Learning in Postsecondary Education

Over the past decade postsecondary education has been moving increasingly from the class room to online. During the fall 2010 term 31 percent of U.S. college students took at least one online course. The primary reasons for the growth of e-learning in the nation\u27s colleges and universities include the desire of those institutions to generate new revenue streams, improve access, and offer students greater scheduling flexibility. Yet the growth of e-learning has been accompanied by a continuing debate about its effectiveness and by the recognition that a number of barriers impede its widespread adoption in higher education

Evaluating a virtual learning environment in the context of its community of practice

The evaluation of virtual learning environments (VLEs) and similar applications has, to date, largely consisted of checklists of system features, phenomenological studies or measures of specific forms of educational efficacy. Although these approaches offer some value, they are unable to capture the complex and holistic nature of a group of individuals using a common system to support the wide range of activities that make up a course or programme of study over time. This paper employs Wenger's theories of 'communities of practice' to provide a formal structure for looking at how a VLE supports a pre-existing course community. Wenger proposes a Learning Architecture Framework for a learning community of practice, which the authors have taken to provide an evaluation framework. This approach is complementary to both the holistic and complex natures of course environments, in that particular VLE affordances are less important than the activities of the course community in respect of the system. Thus, the VLE's efficacy in its context of use is the prime area of investigation rather than a reductionist analysis of its tools and components. An example of this approach in use is presented, evaluating the VLE that supports the undergraduate medical course at the University of Edinburgh. The paper provides a theoretical grounding, derives an evaluation instrument, analyses the efficacy and validity of the instrument in practice and draws conclusions as to how and where it may best be used

Reviews

Martin Oliver (ed.), Innovation in the Evaluation of Learning Technology, London: University of North London, 1998. ISBN: 1–85377–256–9. Softback, 242 pages, £15.00

Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies

A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that (a) contrasted an online to a face-to-face condition, (b) measured student learning outcomes, (c) used a rigorous research design, and (d) provided adequate information to calculate an effect size. As a result of this screening, 51 independent effects were identified that could be subjected to meta-analysis. The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction. The difference between student outcomes for online and face-to-face classes—measured as the difference between treatment and control means, divided by the pooled standard deviation—was larger in those studies contrasting conditions that blended elements of online and face-to-face instruction with conditions taught entirely face-to-face. Analysts noted that these blended conditions often included additional learning time and instructional elements not received by students in control conditions. This finding suggests that the positive effects associated with blended learning should not be attributed to the media, per se. An unexpected finding was the small number of rigorous published studies contrasting online and face-to-face learning conditions for K–12 students. In light of this small corpus, caution is required in generalizing to the K–12 population because the results are derived for the most part from studies in other settings (e.g., medical training, higher education)

Assessing the impact of “more-flexible” learning as part of a study program

With the increasing use of Flexible Learning approaches in Higher Education at the Zurich University of Applied Sciences (ZHAW), measuring their effectiveness, from both an educational and a participant's point of view, is of particular importance. In response to the limited scientific contributions on this topic, this article presents a possibility of how an assessment can take place: this study analyzes 62 undergrad-uate student responses to a Blended Learning task and compares the participant findings with a pre-existing educational competency framework

The Future of Professional Education in Natural Family Planning

Nurses and other health care professionals often have little knowledge of methods of natural family planning (NFP) and do not readily prescribe natural methods for their patients. One reason for this is that little or no information on NFP is provided in nursing or medical schools. The holistic, informational, and integrative nature of NFP fits well with professional nursing practice. A university online distance education NFP teacher training program, which offers academic credit and includes theory, practice, and the latest developments in fertility monitoring, has been developed for health care professionals. Professional NFP services in the United States need to meet worldwide standards and include documenting and assessing pregnancy outcomes, tailoring NFP services to the client or couple, and simplifying them for ease of use in a standard health care practice

Digital Dissemination Platform of Transportation Engineering Education Materials Founded in Adoption Research

INE/AUTC 14.0

Student Satisfaction and Performance in an Online Teacher Certification Program

The article presents a study which demonstrates the effectiveness of an online post baccalaureate teacher certification program developed by a Wisconsin university. The case method approach employing multiple methods and multiple data sources were used to investigate the degree to which pre-service teachers were prepared to teach. It was concluded that the study supports online delivery as an effective means of teacher preparation, but it was limited in the number of students followed into their first year of teaching