Formative e-assessment: Practitioner cases

This paper reports on one aspect of the Joint Information Systems Committee (JISC)-funded project 'Scoping a vision of formative e-assessment', namely on cases of formative e-assessment developed iteratively with the UK education practitioner community. The project, which took place from June 2008 – January 2009, aimed to identify current theories and practices relating to formative assessment of learning where technologies play a key role. The project aimed to scope the 'domain' of formative e-assessment, by developing cases of practice and identifying key formative processes within them, which are affected by the use of technologies. From this analysis, patterns were extracted to inform future software design. A discussion of the key issues emerging from the review of the literature on formative e-assessment, a full account of the project methodology – the design pattern methodology – as well as a critical discussion of the findings – namely the patterns and the role of technology – are the focus of a separate paper (see Daly et al (forthcoming). This paper documents how cases of formative e-assessment were developed during the project by a collaborative methodology involving practitioners from a range of post-16 education contexts. The cases were analysed with reference to key theoretical perspectives on formative assessment, particularly the work of Black and Wiliam (2009). In addition, Laurillard's Conversational Framework (2002, 2007) was used to locate practices of formative assessment within a wider concept of learning and teaching involving technologies, although a detailed discussion of the latter is not within the scope of this paper1

Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Trialing project-based learning in a new EAP ESP course: A collaborative reflective practice of three college English teachers

Currently in many Chinese universities, the traditional College English course is facing the risk of being ‘marginalized’, replaced or even removed, and many hours previously allocated to the course are now being taken by EAP or ESP. At X University in northern China, a curriculum reform as such is taking place, as a result of which a new course has been created called ‘xue ke’ English. Despite the fact that ‘xue ke’ means subject literally, the course designer has made it clear that subject content is not the target, nor is the course the same as EAP or ESP. This curriculum initiative, while possibly having been justified with a rationale of some kind (e.g. to meet with changing social and/or academic needs of students and/or institutions), this is posing a great challenge for, as well as considerable pressure on, a number of College English teachers who have taught this single course for almost their entire teaching career. In such a context, three teachers formed a peer support group in Semester One this year, to work collaboratively co-tackling the challenge, and they chose Project-Based Learning (PBL) for the new course. This presentation will report on the implementation of this project, including the overall designing, operational procedure, and the teachers’ reflections. Based on discussion, pre-agreement was reached on the purpose and manner of collaboration as offering peer support for more effective teaching and learning and fulfilling and pleasant professional development. A WeChat group was set up as the chief platform for messaging, idea-sharing, and resource-exchanging. Physical meetings were supplementary, with sound agenda but flexible time, and venues. Mosoteach cloud class (lan mo yun ban ke) was established as a tool for virtual learning, employed both in and after class. Discussions were held at the beginning of the semester which determined only brief outlines for PBL implementation and allowed space for everyone to autonomously explore in their own way. Constant further discussions followed, which generated a great deal of opportunities for peer learning and lesson plan modifications. A reflective journal, in a greater or lesser detailed manner, was also kept by each teacher to record the journey of the collaboration. At the end of the semester, it was commonly recognized that, although challenges existed, the collaboration was overall a success and they were all willing to continue with it and endeavor to refine it to be a more professional and productive approach

Feedback on Academic Essay Writing through pre-Emptive Hints: Moving Towards "Advice for Action"

This paper adopts an “advice for action” approach to feedback in educational practice: addressing how provision of “hints” to participants before they write academic essays can support their understanding and performance in essay-writing tasks. We explored differences in performance by type of hint, and whether there was a transfer of better performance in subsequent essays. Fifty participants were recruited, consisting of eight men and 42 women aged 18-80. Participants were assigned in rotation to four groups, and asked to write two essays. Groups 1 and 3 received hints before Essay 1, whilst Groups 2 and 4 received hints before Essay 2. Groups 1 and 2 received essential hints; Groups 3 and 4 received helpful hints. Essays were marked against set criteria. The results showed that an “advice for action” approach to essay-writing, in the form of hints, can significantly improve writers’ marks. Specifically higher marks were gained for the introduction, conclusion and use of evidence: critical components of “good” academic essays. As the hints given were content-free, this approach has the potential to instantly benefit tutors and students across subject domains and institutions and is informing the development of a technical system that can offer formative feedback as students draft essays

Collaborative trails in e-learning environments

This deliverable focuses on collaboration within groups of learners, and hence collaborative trails. We begin by reviewing the theoretical background to collaborative learning and looking at the kinds of support that computers can give to groups of learners working collaboratively, and then look more deeply at some of the issues in designing environments to support collaborative learning trails and at tools and techniques, including collaborative filtering, that can be used for analysing collaborative trails. We then review the state-of-the-art in supporting collaborative learning in three different areas – experimental academic systems, systems using mobile technology (which are also generally academic), and commercially available systems. The final part of the deliverable presents three scenarios that show where technology that supports groups working collaboratively and producing collaborative trails may be heading in the near future

Innovating Pedagogy 2015: Open University Innovation Report 4

This series of reports explores new forms of teaching, learning and assessment for an interactive world, to guide teachers and policy makers in productive innovation. This fourth report proposes ten innovations that are already in currency but have not yet had a profound influence on education. To produce it, a group of academics at the Institute of Educational Technology in The Open University collaborated with researchers from the Center for Technology in Learning at SRI International. We proposed a long list of new educational terms, theories, and practices. We then pared these down to ten that have the potential to provoke major shifts in educational practice, particularly in post-school education. Lastly, we drew on published and unpublished writings to compile the ten sketches of new pedagogies that might transform education. These are summarised below in an approximate order of immediacy and timescale to widespread implementation