Formative peer assessment in a CSCL environment

In this case study our aim was to gain more insight in the possibilities of qualitative formative peer assessment in a computer supported collaborative learning (CSCL) environment. An approach was chosen in which peer assessment was operationalised in assessment assignments and assessment tools that were embedded in the course material. The course concerned a higher education case-based virtual seminar, in which students were asked to conduct research and write a report in small multidisciplinary teams. The assessment assignments contained the discussion of assessment criteria, the assessment of a group report of a fellow group, and writing an assessment report. A list of feedback rules was one of the assessment tools. A qualitative oriented study was conducted, focussing on the attitude of students towards peer assessment and practical use of peer assessment assignments and tools. Results showed that students’ attitude towards peer assessment was positive and that assessment assignments had added value. However, not all students fulfilled all assessment assignments. Recommendations for implementation of peer assessment in CSCL environments as well as suggestions for future research are discussed

Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Are e-readers suitable tools for scholarly work?

This paper aims to offer insights into the usability, acceptance and limitations of e-readers with regard to the specific requirements of scholarly text work. To fit into the academic workflow non-linear reading, bookmarking, commenting, extracting text or the integration of non-textual elements must be supported. A group of social science students were questioned about their experiences with electronic publications for study purposes. This same group executed several text-related tasks with the digitized material presented to them in two different file formats on four different e-readers. Their performances were subsequently evaluated by means of frequency analyses in detail. Findings - e-Publications have made advances in the academic world; however e-readers do not yet fit seamlessly into the established chain of scholarly text-processing focusing on how readers use material during and after reading. Our tests revealed major deficiencies in these techniques. With a small number of participants (n=26) qualitative insights can be obtained, not representative results. Further testing with participants from various disciplines and of varying academic status is required to arrive at more broadly applicable results. Practical implications - Our test results help to optimize file conversion routines for scholarly texts. We evaluated our data on the basis of descriptive statistics and abstained from any statistical significance test. The usability test of e-readers in a scientific context aligns with both studies on the prevalence of e-books in the sciences and technical test reports of portable reading devices. Still, it takes a distinctive angle in focusing on the characteristics and procedures of textual work in the social sciences and measures the usability of e-readers and file-features against these standards.Comment: 22 pages, 6 figures, accepted for publication in Online Information Revie

Reducing Procrastination while Improving Performance: A Wiki-powered Experiment with Students

© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.Students in higher education are traditionally requested to produce various pieces of written work during the courses they undertake. When students' work is submitted online as a whole, both the ethically questionable act of procrastinating and late submissions afect performance. The objective of this paper is to assess the performance of students from a control group, with that of students from an experimental group. The control group produced work as a unique deliverable to be submitted at the end of the course. On the other hand, the experimental group worked on each part for a week, and their work was managed by a wiki environment and monitored by a speciically developed software. Positive efects were noticed in the experimental group, as both students' time management skills and performance increased. Replications of this experiment can and should be performed, in order to compare results in coursework submission.Final Accepted Versio

Supporting undergraduate students’ acquisition of academic argumentation strategies through computer conferencing

Executive Summary Background This research grows out of work on the importance of argumentation in developingstudents’ critical abilities. It focuses attention on how students argue in computer mediated conferences as opposed to traditionalwritten assignments, investigating the way in which argumentation is realised within the relatively new context of computer conferencing which allows extended written discussions to take place overa period of weeks. Such text-based asynchronous conferencing is typically characterised by features of both spoken and written modes. Aims The main aims of the project were: • to investigate the argumentation strategies used in asynchronous text-based computer conferences; • to compare the argumentation strategies developed through conferencing with those used in the writing of academic assignments; • to examine the strategies used by tutors to encourage and facilitate argumentation in text-based computer conferences. Methods Data was collected over two years for the distance undergraduate course ‘Perspectives on Complementary and Alternative Medicine’ at the Open University.Qualitative data was obtained through interviews with the course chair, tutors and students, and through a student questionnaire. Assignments and computer-mediated tutorials were collected for textual analysis, although the timing of the assignments meant that analysis has only just begun on the essay data. To analyse the argumentation in the computer conferences and assignments a method of categorising, coding and tracking argumentative discourse was developed building on earlier work by the authors. In addition, computational searches were carried out to compare linguistic features across conference and assignment data. Results In tutorial conferences, student discussion tended to take the form of collaborative co-construction of an argument through exchanging information and experience to substantiate a position. However, students were also prepared to challenge other viewpoints. In both cases, they frequently drew on personal and professional experience to support argument claims. The use of these strategies suggests that text-based conferencing lends itself to the collective combining of diverse sources of information, experiences and ideas. Conference discussions were often personalised with fewer explicit logical links marking argument structure. They were also marked by complexity of argument strands, many of which reached no conclusion. Preliminary analysis of argumentation in assignments suggests that this did not, however, adversely affect students’ ability to create a more traditional, linear argument in their essays. Further analysis will be undertaken to compare argumentation strategies across the two sets of data. Tutors expressed concern about levels of participation in the tutorial conferences, which varied quite considerably. They also felt uncertain about their own knowledge of appropriate pedagogic strategies which would encourage students to participate in a collaborative yet critical way, and tended to rely on strategies from face-to-face teaching. Analysis of the conference discussion showed that tutors made fewer claims than students and were also less likely to provide information in support of their claims. There was, therefore, little modelling by tutors of the basic type of argumentation that would be expected in formal written assignments.Despite these concerns, student responses indicated that having a tutor and a group of peers to interact with, or just to observe, was valued as a supportive feature of this form of distance learning. No clear picture arose of how to make conferencing more interactive for more students, and this reinforces the sense gained from the tutor interviews of the difficulty of proposing a model of tutoring in computer conferences that will necessarily engage all students or raise the level of discussion and debate. Conclusions Our study suggests that text-based conferencing has an important role to play in developing students’ argumentation strategies and understanding of academic discourse and conventions. In view of its hybrid nature, somewhere between spontaneous speech and formal academic writing, course designers and tutors should aim to take advantage of both aspects – on the one hand, the informal dialogic exchange of opinions and co-construction of knowledge, and on the other,the opportunity for consolidation, reflection and re-positioning. Our findings reinforce the view that students’ willingness to exchange ideas freely and openly is partly a consequence of how personally engaged, at ease and confident students feel with one another and their tutor. In particular, it seems that there is a role for the interpersonal and, to some extent, the chat and the frivolity, which in some other studies discussed in the literature review have been regarded as negative influences. Recommendations To facilitate students’ development of argumentation and learning more generally,tutors need greater awareness of the ways in which academic argumentation operates in computer conferencing as compared to written assignments. Since pedagogic strategies developed in other contexts may not transfer well to computer conferencing, there is a need for targeted professional development, focussing in particular on: • Choosing topics for discussion and designing effective task prompts; • Supporting weaker students; • Encouraging challenging of ideas; • Finding the right tone to facilitate peer discussions. Some specific suggestions are made within the report, but our recommendations at this stage remain tentative as we still have to complete the analysis of the assignment data and draw conclusions about the impact of the computer conferencing on the quality of written argumentation within this more formal context