Participation and Performance on Paper- and Computer-Based Low-Stakes Assessments

High-stakes assessments, such the Graduate Records Examination, have transitioned from paper to computer administration. Low-stakes Research-Based Assessments (RBAs), such as the Force Concept Inventory, have only recently begun this transition to computer administration with online services. These online services can simplify administering, scoring, and interpreting assessments, thereby reducing barriers to instructors' use of RBAs. By supporting instructors' objective assessment of the efficacy of their courses, these services can stimulate instructors to transform their courses to improve student outcomes. We investigate the extent to which RBAs administered outside of class with the online Learning About STEM Student Outcomes (LASSO) platform provide equivalent data to tests administered on paper in class, in terms of both student participation and performance. We use an experimental design to investigate the differences between these two assessment conditions with 1,310 students in 25 sections of 3 college physics courses spanning 2 semesters. Analysis conducted using Hierarchical Linear Models indicates that student performance on low-stakes RBAs is equivalent for online (out-of-class) and paper-and-pencil (in-class) administrations. The models also show differences in participation rates across assessment conditions and student grades, but that instructors can achieve participation rates with online assessments equivalent to paper assessments by offering students credit for participating and by providing multiple reminders to complete the assessment. We conclude that online out-of-class administration of RBAs can save class and instructor time while providing participation rates and performance results equivalent to in-class paper-and-pencil tests

Bored with point and click?

Computers have the potential to be exploited as one of the most exciting examples of instructional media. Yet designers often fail to realize this potential. This is, in part, due to the limitations of hardware and software and, in part, due to the lack of good theory developed through conclusive research. Good examples of computer-based learning may owe more to the imaginative flair of the courseware designer than they do to the application of explicit design guidelines and good learning theory. This paper will therefore consider a variety of issues that may be blocking theoretical development and draw conclusions for future courses of action. This starts with a statement of the problem, first by considering the macro and micro issues, and then by looking at a recent call for help in ComputerBased Learning Environment (CBLE) design. Next, the contribution of instructional design theories will be presented together with a way forward for investigating the issues. Finally the implications for future progress are presented

Learning through online discussion: a case of triangulation in research

This paper reports on issues in carrying out research into online discussion. The context is a study of a distance learning module within an MBA program. The module required students to tackle problems based on real life scenarios within small online groups. Students were studying part time and shared similar professional backgrounds. The research looked at students' overall evaluation of the module, ways in which group work was conducted, and the contribution of the tutor. The approach taken was an interpretive case study using questionnaire survey, text analysis and interviews. The main findings from the study are reported, but the focus is on the strengths of, and difficulties in, using the research methods. Triangulation of methods provides the researcher with a greater degree of confidence in reporting findings, although subjective interpretation is still needed

Assessing context-based learning: Not only rigorous but also relevant

Economic factors are driving significant change in higher education. There is increasing responsiveness to market demand for vocational courses and a growing appreciation of the importance of procedural (tacit) knowledge to service the needs of the Knowledge Economy; the skills in demand are information analysis, collaborative working and 'just-in-time learning'. New pedagogical methods go some way to accommodate these skills, situating learning in context and employing information and communications technology to present realistic simulations and facilitate collaborative exchange. However, what have so far proved resistant to change are the practices of assessment. This paper endorses the case for a scholarship of assessment and proposes the development of technology-supported tools and techniques to assess context-based learning. It also recommends a fundamental rethink of the norm-referenced and summative assessment of propositional knowledge as the principal criterion for student success in universities

Learners’ continuance participation intention of collaborative group project in virtual learning environment: an extended TAM perspective

The aim of this study is to explore learners’ intention to return to the electronic environment through the use of wikipages. The survey is based on students’ participation in a collaborative group project over a one semester course on business information systems. A research model based on the extended Technology Acceptance Model (TAM) has been proposed to investigate what factors will influence learners’ continuance participation in the electronic learning environment. 75% of students returned the questionnaire and the data analysis results based on the extended TAM Shows that the learners’ intention to return to the electronic learning environment was highly associated with their attitude towards the electronic learning tool and the affection associated with the tool

Factors influencing the success of computer mediated communication (CMC) environments in university teaching: a review and case study

Despite its potential benefits, the effectiveness of CMC when used to support learning in higher education is very variable, making it important to identify those factors which best predict successful implementations. A review of the literature from the past few years, presented in the first half of this paper, suggests that, consistent with Activity Theory (Leont'ev, 1978), the critical factors are those which provide a context and rationale for online communication by helping users to establish a shared purpose. However, generating empirical support for this hypothesis presents two kinds of methodological problem: specifying the methods and measures necessary to discern the existence and impact of shared purpose; and dealing with the difficulties of making controlled comparisons in this area. The second half of the paper illustrates, via an implementation case study, something of how these methodological problems might be resolved, and presents evidence in favour of the central importance of shared purpose

Chapter 4: New Assessment Methods

The OTiS (Online Teaching in Scotland) programme, run by the now defunct Scotcit programme, ran an International e-Workshop on Developing Online Tutoring Skills which was held between 8–12 May 2000. It was organised by Heriot–Watt University, Edinburgh and The Robert Gordon University, Aberdeen, UK. Out of this workshop came the seminal Online Tutoring E-Book, a generic primer on e-learning pedagogy and methodology, full of practical implementation guidelines. Although the Scotcit programme ended some years ago, the E-Book has been copied to the SONET site as a series of PDF files, which are now available via the ALT Open Access Repository. The editor, Carol Higgison, is currently working in e-learning at the University of Bradford (see her staff profile) and is the Chair of the Association for Learning Technology (ALT)

The impact of broadband in schools

The report reviews evidence for the impact of broadband in English schools, exploring; Variations in provision in level of broadband connectivity; Links between the level of broadband activity and nationally accessible performance data; Aspects of broadband connectivity and the school environment that contribute to better outcomes for pupils and teachers; Academic and motivational benefits associated with educational uses of this technology