Evaluating topic-word review analysis for understanding student peer review performance

© 2013 International Educational Data Mining Society. All rights reserved. Topic modeling is widely used for content analysis of textual documents. While the mined topic terms are considered as a semantic abstraction of the original text, few people evaluate the accuracy of humans’ interpretation of them in the context of an application based on the topic terms. Previously, we proposed RevExplore, an interactive peer-review analytic tool that supports teachers in making sense of large volumes of student peer reviews. To better evaluate the functionality of RevExplore, in this paper we take a closer look at its Natural Language Processing component which automatically compares two groups of reviews at the topic-word level. We employ a user study to evaluate our topic extraction method, as well as the topic-word analysis approach in the context of educational peer-review analysis. Our results show that the proposed method is better than a baseline in terms of capturing student reviewing/writing performance. While users generally identify student writing/reviewing performance correctly, participants who have prior teaching or peer-review experience tend to have better performance on our review exploration tasks, as well as higher satisfaction towards the proposed review analysis approach

Some Temperance on the Doctoral Studies and On-Line Education

Toward the goal of doctoral studies, it is necessary to combine two basic characteristics of independent study. I like to call it an independent study, which would be partial to capture the whole of graduate studies. As for its high honor, the title page of dissertation in vast of universities usually use the phrase “...submitted for the partial fulfillment of doctorate degree...”. That phrase implies that the completion of dissertation would be a major part of doctoral studies, but should be partial depending on some of additional factors. Idealistically, that could be the whole quality as an independent researcher or investigator, and possibly the kind of human paradigm as a prospective teacher. In any case, we would not be incorrect if we see our principal work at the graduate level learning the ways of independent scholar. In this context, I would propose some of elements to be addressed in the end to guide the paradigm of doctoral studies and especially involving the e-age

Wikipedia editing and information literacy: A case study

Purpose: This paper aims to evaluate the success of a Wikipedia editing assessment designed to improve the information literacy skills of a cohort of first-year undergraduate health sciences students. Design/methodology/approach: In this action research case study (known hereafter as “the project” to differentiate this action research from the students’ own research), students researched, wrote and published Wikipedia articles on Australia-centric health topics. Students were given a pre- and post-test to assess levels of self-confidence in finding, evaluating and referencing information. Student work was also analysed in terms of article length and quantity and the type of information sources used. Findings: Tests revealed that students’ self-confidence in their information literacy skills improved overall. Analysis of student work revealed that students wrote longer articles and incorporated more references than expected. References used were of appropriate quality relevant to the article despite minimal instructions. Originality/value: There are few studies that investigate information literacy development through Wikipedia editing in Australian universities. This study shows that Wikipedia editing is an effective way to carry out student assessment prior to essay writing and an innovative platform to improve information literacy skills in undergraduate students

Models for Learning (Mod4L) Final Report: Representing Learning Designs

The Mod4L Models of Practice project is part of the JISC-funded Design for Learning Programme. It ran from 1 May – 31 December 2006. The philosophy underlying the project was that a general split is evident in the e-learning community between development of e-learning tools, services and standards, and research into how teachers can use these most effectively, and is impeding uptake of new tools and methods by teachers. To help overcome this barrier and bridge the gap, a need is felt for practitioner-focused resources which describe a range of learning designs and offer guidance on how these may be chosen and applied, how they can support effective practice in design for learning, and how they can support the development of effective tools, standards and systems with a learning design capability (see, for example, Griffiths and Blat 2005, JISC 2006). Practice models, it was suggested, were such a resource. The aim of the project was to: develop a range of practice models that could be used by practitioners in real life contexts and have a high impact on improving teaching and learning practice. We worked with two definitions of practice models. Practice models are: 1. generic approaches to the structuring and orchestration of learning activities. They express elements of pedagogic principle and allow practitioners to make informed choices (JISC 2006) However, however effective a learning design may be, it can only be shared with others through a representation. The issue of representation of learning designs is, then, central to the concept of sharing and reuse at the heart of JISC’s Design for Learning programme. Thus practice models should be both representations of effective practice, and effective representations of practice. Hence we arrived at the project working definition of practice models as: 2. Common, but decontextualised, learning designs that are represented in a way that is usable by practitioners (teachers, managers, etc).(Mod4L working definition, Falconer & Littlejohn 2006). A learning design is defined as the outcome of the process of designing, planning and orchestrating learning activities as part of a learning session or programme (JISC 2006). Practice models have many potential uses: they describe a range of learning designs that are found to be effective, and offer guidance on their use; they support sharing, reuse and adaptation of learning designs by teachers, and also the development of tools, standards and systems for planning, editing and running the designs. The project took a practitioner-centred approach, working in close collaboration with a focus group of 12 teachers recruited across a range of disciplines and from both FE and HE. Focus group members are listed in Appendix 1. Information was gathered from the focus group through two face to face workshops, and through their contributions to discussions on the project wiki. This was supplemented by an activity at a JISC pedagogy experts meeting in October 2006, and a part workshop at ALT-C in September 2006. The project interim report of August 2006 contained the outcomes of the first workshop (Falconer and Littlejohn, 2006). The current report refines the discussion of issues of representing learning designs for sharing and reuse evidenced in the interim report and highlights problems with the concept of practice models (section 2), characterises the requirements teachers have of effective representations (section 3), evaluates a number of types of representation against these requirements (section 4), explores the more technically focused role of sequencing representations and controlled vocabularies (sections 5 & 6), documents some generic learning designs (section 8.2) and suggests ways forward for bridging the gap between teachers and developers (section 2.6). All quotations are taken from the Mod4L wiki unless otherwise stated

Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Informing Writing: The Benefits of Formative Assessment

Examines whether classroom-based formative writing assessment - designed to provide students with feedback and modified instruction as needed - improves student writing and how teachers can improve such assessment. Suggests best practices

Teaching with infographics: practising new digital competencies and visual literacies

This position paper examines the use of infographics as a teaching assignment in the online college classroom. It argues for the benefits of adopting this type of creative assignment for teaching and learning, and considers the pedagogic and technical challenges that may arise in doing so. Data and insights are drawn from two case studies, both from the communications field, one online class and a blended one, taught at two different institutions. The paper demonstrates how incorporating a research-based graphic design assignment into coursework challenges and encourages students' visual digital literacies. The paper includes practical insights and identifies best practices emerging from the authors' classroom experience with the infographic assignment, and from student feedback. The paper suggests that this kind of creative assignment requires students to practice exactly those digital competencies required to participate in an increasingly visual digital culture

Users' trust in information resources in the Web environment: a status report

This study has three aims; to provide an overview of the ways in which trust is either assessed or asserted in relation to the use and provision of resources in the Web environment for research and learning; to assess what solutions might be worth further investigation and whether establishing ways to assert trust in academic information resources could assist the development of information literacy; to help increase understanding of how perceptions of trust influence the behaviour of information users