Evaluation in a project life‐cycle: The hypermedia CAMILLE project

In the CAL literature, the issue of integrating evaluation into the life‐cycle of a project has often been recommended but less frequently reported, at least for large‐scale hypermedia environments. Indeed, CAL developers face a difficult problem because effective evaluation needs to satisfy the potentially conflicting demands of a variety of audiences (teachers, administrators, the research community, sponsors, etc.). This paper first examines some of the various forms of evaluation adopted by different kinds of audiences. It then reports on evaluations, formative as well as summative, set up by the European CAMILLE project teams in four countries during a large‐scale courseware development project. It stresses the advantages, despite drawbacks and pitfalls, for CAL developers to systematically undertake evaluation. Lastly, it points out some general outcomes concerning learning issues of interest to teachers, trainers and educational advisers. These include topics such as the impact of multimedia, of learner variability and learner autonomy on the effectiveness of learning with respect to language skills

Online peer assessment: helping to facilitate learning through participation

The focus of this paper is on the combination of enquiry-based learning, information literacy and e-learning and how they are embedded in an online peer assessment exercise. What it shall present is a structure and strategy that aids student learning in the short and long-term. Ninety eight students completed a questionnaire before and after a three-week online peer assessment exercise during a first year undergraduate research and study skills module. The results demonstrate that a significant number of students valued the design of the exercise and the benefits it can have on their future learning and development. The paper concludes by suggesting that new and innovative ways of assessment are needed to keep engaging students and develop their learning in different ways

Champagne training on a beer budget

[Abstract]: When confronted by rapidly escalating costs for End User Computing (EUC) training and support, the Gold Coast City Council (GCCC) - the local government authority for Australia’s premier tourist destination - took a proactive stance by adopting a low-cost strategy. This strategy has now been in place for about six years and, as shown in this case study, is reaping rewards in terms of productivity increases and technology diffusion. After identifying an appropriate theoretical perspective for the study, this paper describes the EUC training approach taken at GCCC, and investigates its impact on the productivity of the 1500 person workforce. Both the trainees and the trainers were surveyed to evaluate fully this low-cost strategy. Related social issues of empowering the stakeholders are examined and recommendations are made to ensure that this approach will continue to provide champagne training on a beer budget

Online interactivity: best practice based on two case studies

The purpose of this paper is to identify and explore best practice in the effective support of online interactivity. Five key issues related to best practice are delineated, based on the experience derived from two case studies. The first case study involved online, collaborative work carried out by twenty-six conversion M.Sc. students following a module on‘Interactive Multimedia Systems'. The online group work was structured around the production of essay-style critiques and the development of prototype multimedia resources. The discussions were structured using the bulletin board facility in WebCT. The second study involved sixty-four second-year undergraduate students following a module on‘Communication via Multimedia’. These students were involved in assessed online discussion groups that aimed to foster a community in enquiry and provide an opportunity for vicarious learning. The assessed discussion groups were based on Netscape Collabra. A comparison of the experience of these two case studies led to the identification of a set of five key issues relating to best practice in the effective support of online interactivity. The first four issues concern the design and implementation of the online learning experiences. The fifth issue involves reflection and improvement on the interventions mad

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: N

Knowledge tree: Putting discourse into computer‐based learning

Most CBL materials currently in use model only the declarative aspects of the learning process. If such courseware is used without careful planning, this can be dangerous because one of the most fundamental aspects of education is the dialogue that occurs between teachers and the students. Traditionally, this has taken place in informal discussions as well as in formal small‐group learning sessions such as the conventional tutorial. However, as the student‐staff ratio increases, so does the opportunity for this type of personal dialogue decrease. Modern networking technology offers a huge potential to add discourse to CBL, but there are many pedagogical problems involved with the intrinsically ephemeral and anarchic nature both of the Internet and of most conferencing or bulletin‐board systems. In this paper we describe a software system called Knowledge Tree (KT) which we have developed to address some of these issues. KT combines a hierarchical concept‐oriented database functionality with that of a Usenet‐style bulletin board Using this, a knowledge garden may be developed for any subject area. These each contain a hypermedia database of frequently asked questions, together with answers provided by subject experts. There is provision for inter‐student discussions of problems and issues. When students ask new questions these are automatically emailed to a relevant subject expert (determined by a subject‐specific concept thesaurus). The answer is then placed in the database which eventually grows to become a valuable teaching resource. KT is discipline‐independent as the concept thesaurus can be changed to encapsulate any domain of knowledge. We have used it in support of conventional lecture courses, as an important component of a multimedia course, and for general IT support. These examples illustrate the role that this system can play both in basic information provision, and in facilitating the discussion of deep issues

Utilizing Computer-Assisted Vocabulary Learning Tools in English Language Teaching: Examining In-Service Teachers' Perceptions of the Usability of Digital Flashcards

This study explores five in-service teachers' perceptions with regards to the technical and pedagogical usability of digital flashcards in English language teaching. All the teachers were enrolled in a one-year Masters of Art in English Language Teaching program at Coventry University and had previous teaching experience ranging from elementary to university level. The study adopted a quan?QUAL mixed-method research design, combining elements of surveys and case studies, to examine the factors that affected the teachers' perceptions in addition to how they view three specific websites: Cram, Quizlet, and StudyStack. Participants explored these websites and created sets of flashcards in a computer lab, then completed a survey and participated in a focus group interview. Findings suggest that although the teachers were willing to integrate digital flashcards in their future teaching, it is dependent on several factors, including: learners' age, the quality of graphics in the websites, and the teachers' prior experience as students on their MA program. Nonetheless, the “wow” factor seemed to influence their perceptions of the usability of these websites, which can either be extreme positive or negative initial reactions as a result of the websites' presentational scheme.</jats:p

E/Valuating new media in language development

This paper addresses the need for a new approach to the educational evaluation of software that falls under the rubric "new media" or "multimedia" as distinct from previous generations of Computer-Assisted Language Learning (CALL) software. The authors argue that present approaches to CALL software evaluation are not appropriate for a new genre of CALL software distinguished by its shared assumptions about language learning and teaching as well as by its technical design. The paper sketches a research-based program called "E/Valuation" that aims to assist language educators to answer questions about the educational effectiveness of recent multimedia language learning software. The authors suggest that such program needs to take into account not only the nature of the new media and its potential to promote language learning in novel ways, but also current professional knowledge about language learning and teaching

The Regulation of E-learning: New National and International Policy Perspectives

The universe of postsecondary education is expanding. It is an era of rapid demographic and labor market changes, increased competition and shifts in institutional form (e.g., the rise of for profit degree granters, the hybrid form of nonprofit/for profit partnerships, corporate universities), and new forms of delivery driven by emerging technologies. In nearly all of these cases, the pace of innovation and establishment of new institutional forms outstrips the ability of regulators or policy makers to stay ahead of the curve.To better understand the complex interplay of public policy drivers regulating e-learning, the Center for Studies in Higher Education convened a meeting of experts in February 2006 for a preliminary examination of existing and emerging public policies that will shape its regulation both domestically and internationally. Three white papers were used as a point of departure for the discussions, which focused on the following areas: The Changing Regulatory Environment: Who, What, Why, and Where?; Finance, Investment, and the Flow of Capital; Student Access and Equity; Social Costs and Benefits from an International Perspective; and Consumer Protection and Cross-border Education. This report summarizes our conversations and recommendations for future research