A Data Mining Toolbox for Collaborative Writing Processes

Collaborative writing (CW) is an essential skill in academia and industry. Providing support during the process of CW can be useful not only for achieving better quality documents, but also for improving the CW skills of the writers. In order to properly support collaborative writing, it is essential to understand how ideas and concepts are developed during the writing process, which consists of a series of steps of writing activities. These steps can be considered as sequence patterns comprising both time events and the semantics of the changes made during those steps. Two techniques can be combined to examine those patterns: process mining, which focuses on extracting process-related knowledge from event logs recorded by an information system; and semantic analysis, which focuses on extracting knowledge about what the student wrote or edited. This thesis contributes (i) techniques to automatically extract process models of collaborative writing processes and (ii) visualisations to describe aspects of collaborative writing. These two techniques form a data mining toolbox for collaborative writing by using process mining, probabilistic graphical models, and text mining. First, I created a framework, WriteProc, for investigating collaborative writing processes, integrated with the existing cloud computing writing tools in Google Docs. Secondly, I created new heuristic to extract the semantic nature of text edits that occur in the document revisions and automatically identify the corresponding writing activities. Thirdly, based on sequences of writing activities, I propose methods to discover the writing process models and transitional state diagrams using a process mining algorithm, Heuristics Miner, and Hidden Markov Models, respectively. Finally, I designed three types of visualisations and made contributions to their underlying techniques for analysing writing processes. All components of the toolbox are validated against annotated writing activities of real documents and a synthetic dataset. I also illustrate how the automatically discovered process models and visualisations are used in the process analysis with real documents written by groups of graduate students. I discuss how the analyses can be used to gain further insight into how students work and create their collaborative documents

The use of text and process mining techniques to study the impact of feedback on students’ writing processes

Understanding the impact of feedback in complex learning activities, such as writing, is challenging. We contribute a combination of writing environments and data and process mining tools that can provide new ways of measuring this impact. We use the tools in a field experiment in an engineering course (N=45). Responses (timing, amount and types of text changes) were examined using log data and process mining techniques. Two experimental conditions were used: reflective followed by directive feedback (A) and vice-versa (B). We found that both forms of feedback were read multiple times. Students required longer times to respond to reflective, compared to directive, feedback. The type of feedback, however, made little difference to the types of revisions that students performed. Overall, our findings point to the difficulty of encouraging students to reconsider and revise what they have already written

Agent-based computer models for learning about climate change and process analysis techniques

This paper describes a design-based research project that investigates the learning of scientific knowledge about climate change through computational models. The design experiment used two NetLogo models and problem-based learning materials developed in partnership with a high school science teacher. In the study, three classes of year nine science students were divided into two groups based upon different levels of structure that was provided during learning activities with the models. The results indicate that there was significant learning of concepts about greenhouse gases and the carbon cycle through engagement with the models. We also describe the process analysis techniques being developed to analyze the log files of the interactions the students had with the computer models

Using process analysis techniques to understand students' learning strategies with computer models

This work is a part of a larger project that investigates how high school students learn scientific knowledge of climate change with computer models. The paper presents our progress developing a methodology for capturing learning process data and preliminary results from the analysis of learning strategies adopted by high achieving and low achieving students. Our approach is based on the analysis of process data using the Hidden Markov Model (HMM) technique. Drawing on the initial results, we illustrate how the HMM can help to depict some important features of students' learning strategies. Overall, our findings indicate that successful learners adopt deeper and more systematic model exploration strategies than less successful learners

Capturing and analysing the processes and patterns of learning in collaborative learning environments

This paper describes our methodological experiences capturing and analyzing student learning processes and patterns in three cases. Agent-based models and a virtual world were used for learning. Several specific features tie these cases together and distinguish our analysis from other studies in the CSCL domain. First, students interacted in real-time for relatively short periods. Second, they interacted with each other and with interactive software tools that dynamically 'shaped,' and were shaped by, their learning process. Our work builds upon and integrates process analytic approaches of dynamically captured video and computer screen activity and automatic e-learning process analysis techniques. The first two cases identify areas in which analysis 'by hand' of small amounts of data has produced findings of initial interest. The third case discusses the use of an automatic pattern discovery technique based on Hidden Markov Models to begin to apply these methods to larger data sets in CSCL environments.</p

ICT as learning media and research instrument: What eResearch can offer for those who research eLearning?

Students' interactions in digital learning environments are distributed over time and space, and many aspects of eLearning phenomenon cannot be investigated using traditional research approaches. At the same time, the possibility to collect digital data about students' online interactions and learning opens a range of new opportunities to use ICT as research tool and apply new research approaches. This symposium brings together some of the recent advancements in the area of ICT-enhanced research and aims to discuss future directions for methodological innovation in this area. The session will include four presentations that will explore different directions of ICT use for eLearning research.</p