The Virtual Tutor: Combining Conversational Agents with Learning Analytics to support Formative Assessment in Online Collaborative Learning

The objective of this design science research project is to combine Learning Analytics data with a conversational agent communication interface, the Virtual Tutor, which is able to support formative assessment for educators and learners in online collaborative learning (OCL) environments. The main benefit for educators is providing user-adaptable Learning Analytics data requests to fit the information needs for formative assessment. Learners receive semi-automated feedback on their platform activity in form of reports, which shall trigger self-reflection processes. By extracting requirements from the potential users and deriving design principles, a conversational agent is implemented and evaluated in an online collaborative learning course. The results indicate that the Virtual Tutor reduces the task load of educators, supports formative assessment and gives scaffolded guidance to the learners by reflecting their performance, thus triggering self-reflection processes. This research provides a first step towards data supported (semi-)automated feedback systems for formative assessment in OCL courses

My Science Tutor (MyST) -- A Large Corpus of Children's Conversational Speech

This article describes the MyST corpus developed as part of the My Science Tutor project -- one of the largest collections of children's conversational speech comprising approximately 400 hours, spanning some 230K utterances across about 10.5K virtual tutor sessions by around 1.3K third, fourth and fifth grade students. 100K of all utterances have been transcribed thus far. The corpus is freely available (https://myst.cemantix.org) for non-commercial use using a creative commons license. It is also available for commercial use (https://boulderlearning.com/resources/myst-corpus/). To date, ten organizations have licensed the corpus for commercial use, and approximately 40 university and other not-for-profit research groups have downloaded the corpus. It is our hope that the corpus can be used to improve automatic speech recognition algorithms, build and evaluate conversational AI agents for education, and together help accelerate development of multimodal applications to improve children's excitement and learning about science, and help them learn remotely

Negotiating textual talk : conversation analysis, pedagogy, and the organisation of online asynchronous discourse

This paper uses Conversation Analysis to investigate the ways in which participants in an online asynchronous postgraduate reading group managed and negotiated their contributions within the discussion. Using the conversation analytic concerns with sequential organisation, adjacency pairs and topicality, this article shows the analytic insights that this perspective can bring to the examination of written asynchronous discourse. The paper shows that in the section of the discussion analysed here, the discourse displayed remarkable similarities to the ways in which face-to-face conversation has been seen to operate in terms of the organisation of conversational turns, the application of specific interactional rights, the lineal development of topics of conversation, and the structural use of question-answer turn pairs. The paper concludes by showing how this form of analysis can relate to the formation of reflexive pedagogy in which course design can be created to take account of such findings. It shows how a detailed understanding of how pedagogy is played out in interaction is fundamental for reflecting on the relationship between pedagogic aims and educational practice

A discourse analysis of trainee teacher identity in online discussion forums

Teacher education involves an identity transformation for trainees from being a student to being a teacher. This discourse analysis examined the online discussion board communications of a cohort of trainee teachers to better understand the situated identities of the trainees and how they were presented online. Their discussion board posts were the primary method of communication during placement periods and, as such, provided insight into how the trainees situated their identities in terms of being a student or being a teacher. During the analysis, the community boundaries, language and culture were explored along with the tutor's power and role in the identity transformation process. This involved looking at the lexis used by the students, the use of pronouns to refer to themselves and others such as teachers and pupils, the types of messages allowed in the community and the effect of the tutor's messages on their communication. The research found that the trainees felt comfortable with teaching but did not feel like teachers during the course. Tutors and school teachers need to develop an awareness of the dual nature of trainees' identities and help promote the transition from student to teacher. In the beginning of the course, trainees should be familiarised with teacher vocabulary and practical concepts in addition to pedagogical theory. Towards the end of the course, trainee identity as teachers could be promoted through the use of authentic assessments that mirror real teacher tasks and requirements

Scoping a vision for formative e-assessment: a project report for JISC

Assessment is an integral part of teaching and learning. If the relationship between teaching and learning were causal, i. e. if students always mastered the intended learning outcomes of a particular sequence of instruction, assessment would be superfluous. Experience and research suggest this is not the case: what is learnt can often be quite different from what is taught. Formative assessment is motivated by a concern with the elicitation of relevant information about student understanding and / or achievement, its interpretation and an exploration of how it can lead to actions that result in better learning. In the context of a policy drive towards technology-enhanced approaches to teaching and learning, the question of the role of digital technologies is key and it is the latter on which this project particularly focuses. The project and its deliverables have been informed by recent and relevant literature, in particular recent work by Black andIn this work, they put forward a framework which suggests that assessment for learning their term for formative assessment can be conceptualised as consisting of a number of aspects and five keystrategies. The key aspects revolve around the where the learner is going, where the learner is right now and how she can get there and examines the role played by the teacher, peers and the learner. Language: English Keywords: assessments, case studies, design patterns, e-assessmen

MeIPeAS: An Intelligent Virtual Tutor for Mexican Elementary Schoolchildren

Artículo en revista indizada publicado en Research in Computing ScienceIt is known that virtual tutors have a wide range of functionalities, which have been little exploited and applied in the educational field at the primary level. However, these functionalities allow to offer mechanisms of interaction with students through an interactive dialogue by using text to speech, and even more sophisticated, the recognition and understanding of natural language or speech. In this paper, a personalized virtual tutor for the primary education scenario in Mexico is presented. This virtual tutor is called Mexican Intelligent Pedagogical Agent for Schoolchildren (MeIPeAS) and was created to be used as a pedagogical support mechanism offering a unique attraction for current and future generations of schoolchildren in Mexico. The virtual tutor has been validated in practice in public primary schools of the municipalities of the State of Mexico in Mexico. This validation is to analyze the impact of the user experience from the obtained results having relevant information about the reinforcement of topics taught within the classroom

A window into learning: case studies of online group communication and collaboration

The two case studies presented explore the potential offered by in‐depth qualitative analysis of students’ online discussion to enhance our understanding of how students learn. Both cases are used to illustrate how the monitoring and moderation of online student group communication can open up a ‘window into learning’, providing us with new insights into complex problem‐solving and thinking processes. The cases offer examples of students’ ‘thinking aloud’ while problem‐solving, showing how and why they arrived at particular outcomes and the underlying thought processes involved. It is argued that these insights into students’ learning processes can in turn offer us the opportunity to adapt our own teaching practice in order to achieve a better pedagogical ‘fit’ with the learning needs of our students; for example, through a more precise or more timely intervention. It is also suggested that looking through this ‘window’ enables us to concentrate our assessment more closely on the process of task completion, rather than focusing solely on the end product

Impacts of directed tutorial activities in computer conferencing: a case study

This paper describes a qualitative study of asynchronous electronic conferencing by three tutorial groups on the same postgraduate course (Teaching English to Speakers of Other Languages Worldwide), forming part of an MA in Applied Linguistics (via Distance Education) at the Open University, UK. The groups varied in the degree to which the tutor participated in the discussion and in whether the tutor's input took the form of responding to student posts or the setting of tasks to scaffold the learners' development of academic skills. It is argued that the least interventionist strategy in terms of tutor response and task-setting resulted in the least productive conference discussion in terms of both communicative interaction and academic development, while a more interventionist role by the tutor depended for its success on characteristics of the tutor input and the task set