Discourse-centric learning analytics

Drawing on sociocultural discourse analysis and argumentation theory, we motivate a focus on learners' discourse as a promising site for identifying patterns of activity which correspond to meaningful learning and knowledge construction. However, software platforms must gain access to qualitative information about the rhetorical dimensions to discourse contributions to enable such analytics. This is difficult to extract from naturally occurring text, but the emergence of more-structured annotation and deliberation platforms for learning makes such information available. Using the Cohere web application as a research vehicle, we present examples of analytics at the level of individual learners and groups, showing conceptual and social network patterns, which we propose as indicators of meaningful learning

Discourse-centric learning analytics: mapping the terrain

There is an increasing interest in developing learning analytic techniques for the analysis, and support of, high quality learning discourse. This paper maps the terrain of discourse-centric learning analytics (DCLA), outlining the distinctive contribution of DCLA and outlining a definition for the field moving forwards. It is our claim that DCLA provide the opportunity to explore the ways in which: discourse of various forms both resources and evidences learning; the ways in which small and large groups, and individuals make and share meaning together through their language use; and the particular types of language – from discipline specific, to argumentative and socio-emotional – associated with positive learning outcomes. DCLA is thus not merely a computational aid to help detect or evidence ‘good’ and ‘bad’ performance (the focus of many kinds of analytic), but a tool to help investigate questions of interest to researchers, practitioners, and ultimately learners. The paper ends with three core issues for DCLA researchers – the challenge of context in relation to DCLA; the various systems required for DCLA to be effective; and the means through which DCLA might be delivered for maximum impact at the micro (e.g. learner), meso (e.g. school), and macro (e.g. governmental) levels

Discourse, computation and context – sociocultural DCLA revisited

This paper expands the sociocultural analysis of earlier discourse centric learning analytics (DCLA) to discuss the pedagogic functions of discourse, and the implications of these functions for DCLA. Given the importance of discourse for learning [13], and the potential of computers to (a) scaffold effective discourse and (b) give meaningful feedback on such discourse, it is important that DCLA are well theorised. Sociocultural theory emphasises context, and discourse “in action” in its analysis. If DCLA wishes to ground itself in such theory, work will need to be done to address these aspects of discourse in computational analysis. Given the potential of DCLA to provide support for educational talk – an important aspect of learning – research should be conducted to further develop DCLA approaches to such talk

Critical perspectives on writing analytics

Writing Analytics focuses on the measurement and analysis of written texts for the purpose of understanding writing processes and products, in their educational contexts, and improving the teaching and learning of writing. This workshop adopts a critical, holistic perspective in which the definition of "the system" and "success" is not restricted to IR metrics such as precision and recall, but recognizes the many wider issues that aid or obstruct analytics adoption in educational settings, such as theoretical and pedagogical grounding, usability, user experience, stakeholder design engagement, practitioner development, organizational infrastructure, policy and ethics

Combining automated and peer feedback for effective learning design in writing practices

© 2017 Institute of Electrical and Electronics Engineers Inc.. All rights reserved. The provision of formative feedback has been shown to support self-regulated learning for improving students' writing. Formative peer feedback is a promising approach, but requires scaffolding to be effective for all students. Automated tools making use of writing analytics techniques are another useful means to provide formative feedback on students' writing. However, they should be applied through effective learning designs in pedagogic contexts for better uptake and sense-making by students. Such learning analytics applications open up the possibilities to combine different types of feedback for effective design of interventions in authentic contexts. A framework combining peer feedback and automated feedback is proposed to design effective interventions for improving student writing. Automated feedback is augmented by peer feedback for better contextual feedback and sense making, and peer feedback is enhanced by automated feedback as scaffolding, thus complementing each other

Statistical discourse analysis: A method for modeling online discussion processes

Online forums (synchronous and asynchronous) offer exciting data opportunities to analyze how people influence one another through their interactions. However, researchers must address several analytic difficulties involving the data (missing values, nested structure [messages within topics], non‐sequential messages), outcome variables (discrete outcomes, rare instances, multiple outcome variables, similarities among nearby messages), and explanatory variables (sequences of explanatory variables, indirect mediation effects, false positives, and robustness of results). We explicate a method that addresses these difficulties (Statistical Discourse Analysis or SDA) and illustrate it on 1,330 asynchronous messages written and selfcoded by 17 students during a 13‐week online educational technology course. Both individual characteristics and message attributes were linked to participants’ online messages. Men wrote more messages about their theories than women did. Moreover, some sequences of messages were more likely to precede other messages. For example, opinions were often followed by elaborations, which were often followed by theorizing

Statistical Discourse Analysis: A method for modeling online discussion processes

Online forums (synchronous and asynchronous) offer exciting data opportunities to analyze how people influence one another through their interactions. However, researchers must address several analytic difficulties involving the data (missing values, nested structure [messages within topics], non‐sequential messages), outcome variables (discrete outcomes, rare instances, multiple outcome variables, similarities among nearby messages), and explanatory variables (sequences of explanatory variables, indirect mediation effects, false positives, and robustness of results). We explicate a method that addresses these difficulties (Statistical Discourse Analysis or SDA) and illustrate it on 1,330 asynchronous messages written and selfcoded by 17 students during a 13‐week online educational technology course. Both individual characteristics and message attributes were linked to participants’ online messages. Men wrote more messages about their theories than women did. Moreover, some sequences of messages were more likely to precede other messages. For example, opinions were often followed by elaborations, which were often followed by theorizing

A Visualisation Dashboard for Contested Collective Intelligence. Learning Analytics to Improve Sensemaking of Group Discussion

The skill to take part in and to contribute to debates is important for informal and formal learning. Especially when addressing highly complex issues, it can be difficult to support learners participating in effective group discussion, and to stay abreast of all the information collectively generated during the discussion. Technology can help with the engagement and sensemaking of such large debates, for example, it can monitor how healthy a debate is and provide indicators of participation's distribution. A special framework that aims at harnessing the intelligence of - small to very large – groups with the support of structured discourse and argumentation tools is Contested Collective Intelligence (CCI). CCI tools provide a rich source of semantic data that, if appropriately processed, can generate powerful analytics of the online discourse. This study presents a visualisation dashboard with several visual analytics that show important aspects of online debates that have been facilitated by CCI discussion tools. The dashboard was designed to improve sensemaking and participation in online debates and has been evaluated with two studies, a lab experiment and a field study in the context of two Higher Education institutes. The paper reports findings of a usability evaluation of the visualisation dashboard. The descriptive findings suggest that participants with little experience in using analytics visualisations were able to perform well on given tasks. This constitutes a promising result for the application of such visualisation technologies as discourse-centric learning analytics interfaces can help to support learners' engagement and sensemaking of complex online debates. In Spanish: La habilidad para participar y contribuir en los debates es importante para el aprendizaje informal y formal. Especialmente cuando se abordan temas altamente complejos, puede ser difícil apoyar a los alumnos que participan en una discusión grupal efectiva y mantenerse al tanto de toda la información generada colectivamente durante la discusión. La tecnología puede ayudar con el compromiso y razonamiento en debates tan grandes, por ejemplo, puede monitorear cuán saludable es un debate y proporcionar indicadores sobre la distribución de la participación. Un marco especial que pretende aprovechar la inteligencia de grupos de pequeños a muy grandes con el apoyo de herramientas de discurso y argumentación estructuradas es la Inteligencia Colectiva Controvertida (CCI). Las herramientas de CCI proporcionan una fuente rica de datos semánticos que, si se procesan de manera adecuada, pueden generar un sofisticado análisis del discurso en línea. Este estudio presenta un panel de visualización con varios análisis visuales que muestran aspectos importantes de los debates en línea que han sido facilitados por las herramientas de discusión de CCI. El tablero de instrumentos fue diseñado para mejorar la creación de sentidos y la participación en los debates en línea y se ha evaluado con dos estudios, un experimento de laboratorio y un estudio de campo, en el contexto de dos institutos de educación superior. Este artículo informa sobre los resultados de una evaluación de usabilidad del panel de visualización. Los hallazgos descriptivos sugieren que los participantes con poca experiencia en el uso de visualizaciones analíticas pudieron desempeñarse bien en determinadas tareas. Esto constituye un resultado prometedor para la aplicación de tales tecnologías de visualización, ya que las interfaces analíticas de aprendizaje centradas en el discurso pueden ayudar a apoyar el compromiso de los alumnos y su razonamiento en debates en línea complejos

XIP Dashboard: visual analytics from automated rhetorical parsing of scientific metadiscourse

A key competency that we seek to build in learners is a critical mind, i.e. ability to engage with the ideas in the literature, and to identify when significant claims are being made in articles. The ability to decode such moves in texts is essential, as is the ability to make such moves in one’s own writing. Computational techniques for extracting them are becoming available, using Natural Language Processing (NLP) tuned to recognize the rhetorical signals that authors use when making a significant scholarly move. After reviewing related NLP work, we introduce the Xerox Incremental Parser (XIP), note previous work to render its output, and then motivate the design of the XIP Dashboard, a set of visual analytics modules built on XIP output, using the LAK/EDM open dataset as a test corpus. We report preliminary user reactions to a paper prototype of such a novel dashboard, describe the visualizations implemented to date, and present user scenarios for learners, educators and researchers. We conclude with a summary of ongoing design refinements, potential platform integrations, and questions that need to be investigated through end-user evaluations

Statistical Discourse Analysis of Online Discussion: Informal cognition, social metacognition, and knowledge creation

To statistically model large data sets of knowledge processes during asynchronous, online forums, we must address analytic difficulties involving the whole data set (missing data, nested data and the tree structure of online messages), dependent variables (multiple, infrequent, discrete outcomes and similar adjacent messages), and explanatory variables (sequences, indirect effects, false positives, and robustness). Statistical discourse analysis (SDA) addresses all of these issues, as shown in an analysis of 1,330 asynchronous messages written and self-coded by 17 students during a 13-week online educational technology course. The results showed how attributes at multiple levels (individual and message) affected knowledge creation processes. Men were more likely than women to theorize. Asynchronous messages created a micro-sequence context; opinions and asking about purpose preceded new information; anecdotes, opinions, different opinions, elaborating ideas, and asking about purpose or information preceded theorizing. These results show how informal thinking precedes formal thinking and how social metacognition affects knowledge creation