Prospects for summative evaluation of CAL in higher education

Many developers and evaluators feel an external demand on them for summative evaluation of courseware. Problems soon emerge. One is that the CAL may not be used at all by students if it is not made compulsory. If one measures learning gains, how does one know that one is measuring the effect of the CAL or of the motivation in that situation? Such issues are the symptoms of the basic theoretical problem with summative evaluation, which is that CAL does not cause learning like turning on a tap, any more than a book does. Instead, it is one rather small factor in a complex situation. It is of course possible to do highly controlled experiments: for example to motivate the subjects in a standardized way. This should lead to measurements that are repeatable by other similar experiments. However they will be measurements that have little power to predict the outcome when the CAL is used in real courses. Hence the simple view of summative evaluation must be abandoned. Yet it is possible to gather useful information by studying how a piece of CAL is used in a real course and what the outcomes were. Although this does not guarantee the same outcomes for another purchaser, it is obviously useful to know that the CAL has been used successfully one or more times, and how it was used on those occasions. Such studies can also serve a different ‘integrative’ rather than summative function by pointing out failings of the CAL software and suggesting how to remedy them

A "BPS expansion" for B and D mesons

We analyze consequences of the approximation \mu_\pi^2 -\mu_G^2 << \mu_\pi^2 (a `BPS' limit) for B and D mesons. It is shown that neglecting perturbative effects many power corrections would vanish to all orders in 1/m_Q, in particular those violating heavy flavor symmetry. Among them are corrections to B->D formfactors. A number of relations receive corrections only to the second order expanding around the BPS limit to any order in 1/m_Q, including both f_+ and f_- at zero recoil. This allows an accurate evaluation of {\cal F}_+ for B->D l\nu. Its perturbative renormalization is computed analytically in the required Wilsonian scheme, yielding the dominant 3% enhancement.Comment: 13 pages, LaTeX, two figure

Program evaluation of Cal-SAFE: A program for pregnant and parenting teens

This program evaluated the effectiveness of the program interventions implemented during the 2002-2003 school year in the Riverside County Office of Education\u27s Cal-SAFE pregnant and parenting minor program. Two researchers divided the twelve school sites, gathered data and formulated information to assess the effectiveness of the program on continuance or completion of high school after teen pregnancy

CAL evaluation: Future directions

Formal, experimental methods have proved increasingly difficult to implement, and lack the capacity to generate detailed results when evaluating the impact of CAL on teaching and learning. The rigid nature of experimental design restricts the scope of investigations and the conditions in which studies can be conducted It has also consistently failed to account for all influences on learning. In innovative CAL environments, practical and theoretical development depends on the ability fully to investigate the wide range of such influences. Over the past five years, a customizable evaluation framework has been developed specifically for CAL research. The conceptual approach is defined as Situated Evaluation of CAL (SECAL), and the primary focus is on quality of learning outcomes. Two important principles underpin this development. First, the widely accepted need to evaluate in authentic contexts includes examination of the combined effects of CAL with other resources and influential aspects of the learning environment. Secondly, evaluation design is based on a critical approach and qualitative, case‐based research. Positive outcomes from applications of SECAL include the easy satisfaction of practical and situation‐specific requirements and the relatively low cost of evaluation studies. Although there is little scope to produce generalizable results in the short term, the difficulty of doing so in experimental studies suggests that this objective is difficult to achieve in educational research. A more realistic, longer‐term aim is the development of grounded theory based on common findings from individual cases

A modern course on digital design at the University of Twente

At the University of Twente, the Netherlands a modern course on basic digital design has been developed. This course, which is based on a text book in English (K. J. Breeding, 1992), is supported by a sophisticated computer-assisted learning program and life-like practical work with extensive use of computer-aided-design tools. These features largely reduce the need for human teachers. Major parts of the course are particularly suited to be copied by other institute

Teaching periodontal pocket charting to dental students: a comparison of computer assisted learning and traditional tutorials

AIM: The aim of this study was to compare the effectiveness of a computer assisted learning (CAL) programme with that of traditional small group tutorials in teaching theoretical and practical aspects of periodontal pocket charting. METHOD: Sixty-one third year undergraduate dental students were randomized to either receive a tutorial or to work through the CAL programme. Students using the CAL programme completed questionnaires relating to previous computer experience and the ease of use of the programme. All students were assessed immediately after the intervention by means of a confidence log, a practical exercise and a further confidence log. They were assessed again three weeks later by means of a confidence log and a multiple-choice written test. RESULTS: There were very few significant differences between groups for any of the assessments used. However, subjective comments indicated that students occasionally felt disadvantaged if they had not received a tutorial. CONCLUSION: CAL and traditional teaching methods are equally effective in teaching periodontal pocket charting to undergraduate dental students

Evaluating EuroMET ‐ a Web‐based resource for meteorologists

Euro MET has been created to address the education and training needs of professional meteorologists and students in tertiary education throughout Europe and beyond Two module libraries have been developed for the WWW which are available in English, French, Spanish and German. The modules have been evaluated during their development and after completion. All the project partners used them in courses and training programmes within their institutions. This paper describes the formative and sum‐mative evaluation strategies used and some of the results obtained The efficacy of the evaluation approach adopted with respect to the size of the project is also discusse

Enriching accounts of computer‐supported collaboration by using video data

This paper will discuss the approach to the evaluation of computer‐supported collaborative learning developed in our group over the past ten years. This approach depends on the collection of video data to allow the analysis of key features of problem‐solving behaviour within groups of students working on collaborative learning tasks. Our theoretical framework derives from two sources‐ the CIAOl framework for evaluating examples of CAL and an analysis of appropriate methods of evaluating computer‐supported collaboration. Our work in this area has been supported by developing the data capture facilities for the CALRG (Computers and Learning Research Group) at the Open University. We will draw on a number of studies to illustrate this approach and will present a brief case study from work done on a computer‐supported learning environment for statistics where we use video records of video‐mediated collaboration. This case study gives an example of the rich data that can be collected using video recording and analysed to increase understanding of computer‐supported collaboration