Toward a document evaluation methodology: What does research tell us about the validity and reliability of evaluation methods?

Although the usefulness of evaluating documents has become generally accepted among communication professionals, the supporting research that puts evaluation practices empirically to the test is only beginning to emerge. This article presents an overview of the available research on troubleshooting evaluation methods. Four lines of research are distinguished concerning the validity of evaluation methods, sample composition, sample size, and the implementation of evaluation results during revisio

Analysing the Role of Interactivity in User Experience

An experimental investigation into the role of interaction in user experience (UX) with a controlled manipulation of interactivity features (e.g. avatars, interactive video) in a university information website is reported. The more interactive version had better affect and hedonic ratings, even though its perceived usability was worse. Analysis of qualitative data showed users were attracted to the interactive features, although they complained about poor usability. The results of the experiments are discussed to consider the role of interactivity in user experience and the differences between users’ quantitative judgements of UX and their comments on interactive features which reveal different perspectives

The role of cognitive schemas in a web-based student evaluation of teaching system: usability issues of design and implementation

With the increase in the use of web-based student evaluations of teaching (SETs) by institutions of higher education, the importance of understanding the usability of such web-based systems has increased. Such understanding is needed to ensure that web-based SETs produce the best information possible. By using schema theory from cognitive psychology as a backdrop, this study seeks to identify the impact that the method of response and grouping of items displayed per page have on the usability of web-based SETs. Issues of user satisfaction, error rate, and time on task are examined. Participants were divided into one of six similarity groups based on the degree to which participant expectation and the design of the web-based SET system matched. Analyzing data from 791 university students at a large southern University, it was determined that the use of a radio-button response format with SET items grouped by area produces the best results from a usability perspective. The use of drop-down boxes and text boxes is discouraged as a response format. Limitations of this study and suggested directions for future research are discussed.Educational Psycholog

Chapter 5: Evaluation

The OTiS (Online Teaching in Scotland) programme, run by the now defunct Scotcit programme, ran an International e-Workshop on Developing Online Tutoring Skills which was held between 8–12 May 2000. It was organised by Heriot–Watt University, Edinburgh and The Robert Gordon University, Aberdeen, UK. Out of this workshop came the seminal Online Tutoring E-Book, a generic primer on e-learning pedagogy and methodology, full of practical implementation guidelines. Although the Scotcit programme ended some years ago, the E-Book has been copied to the SONET site as a series of PDF files, which are now available via the ALT Open Access Repository. The editor, Carol Higgison, is currently working in e-learning at the University of Bradford (see her staff profile) and is the Chair of the Association for Learning Technology (ALT)

A high school physics instructor\u27s website: Design, implementation, and evaluation

In order to test the ability of the Internet to supplement classroom instruction, an instructor-authored WWW site crashwhite.com was developed for two Berkeley High courses: Advanced Placement (AP) physics, and college-prep physics class. The website was intended to supplement classroom instruction by making classroom materials available to students and parents outside the classroom, and to facilitate increased teacher-parent, teacher-student, and student-student communication

A comparison of the comprehension of procedural information using computer and hard-copy media

Users of technical procedures must be able to understand the documents to use them to perform their work. As more companies contemplate putting their procedures on-line, it is important to know whether computer systems will be as effective as traditional hard-copy presentation in communicating procedures to the employees who must use them; To determine whether there is a relationship between computer usage and the comprehension of technical procedures, an experiment was conducted among employees of a scientific and technical company in Las Vegas, Nevada. A control group read and demonstrated its comprehension of hard-copy procedures only, while an experimental group read and demonstrated its comprehension of a hard-copy and then an on-line procedure; The experimental group selected fewer correct answers on a comprehension test for the on-line than for the hard-copy procedure. This suggests that when readers accustomed to the hard-copy medium switch to the computer medium, comprehension decreases