A Data Mining Toolbox for Collaborative Writing Processes

Collaborative writing (CW) is an essential skill in academia and industry. Providing support during the process of CW can be useful not only for achieving better quality documents, but also for improving the CW skills of the writers. In order to properly support collaborative writing, it is essential to understand how ideas and concepts are developed during the writing process, which consists of a series of steps of writing activities. These steps can be considered as sequence patterns comprising both time events and the semantics of the changes made during those steps. Two techniques can be combined to examine those patterns: process mining, which focuses on extracting process-related knowledge from event logs recorded by an information system; and semantic analysis, which focuses on extracting knowledge about what the student wrote or edited. This thesis contributes (i) techniques to automatically extract process models of collaborative writing processes and (ii) visualisations to describe aspects of collaborative writing. These two techniques form a data mining toolbox for collaborative writing by using process mining, probabilistic graphical models, and text mining. First, I created a framework, WriteProc, for investigating collaborative writing processes, integrated with the existing cloud computing writing tools in Google Docs. Secondly, I created new heuristic to extract the semantic nature of text edits that occur in the document revisions and automatically identify the corresponding writing activities. Thirdly, based on sequences of writing activities, I propose methods to discover the writing process models and transitional state diagrams using a process mining algorithm, Heuristics Miner, and Hidden Markov Models, respectively. Finally, I designed three types of visualisations and made contributions to their underlying techniques for analysing writing processes. All components of the toolbox are validated against annotated writing activities of real documents and a synthetic dataset. I also illustrate how the automatically discovered process models and visualisations are used in the process analysis with real documents written by groups of graduate students. I discuss how the analyses can be used to gain further insight into how students work and create their collaborative documents

What is behind a summary-evaluation decision?

Research in psychology has reported that, among the variety of possibilities for assessment methodologies, summary evaluation offers a particularly adequate context for inferring text comprehension and topic understanding. However, grades obtained in this methodology are hard to quantify objectively. Therefore, we carried out an empirical study to analyze the decisions underlying human summary-grading behavior. The task consisted of expert evaluation of summaries produced in critically relevant contexts of summarization development, and the resulting data were modeled by means of Bayesian networks using an application called Elvira, which allows for graphically observing the predictive power (if any) of the resultant variables. Thus, in this article, we analyzed summary-evaluation decision making in a computational framewor

Mapping Big Data into Knowledge Space with Cognitive Cyber-Infrastructure

Big data research has attracted great attention in science, technology, industry and society. It is developing with the evolving scientific paradigm, the fourth industrial revolution, and the transformational innovation of technologies. However, its nature and fundamental challenge have not been recognized, and its own methodology has not been formed. This paper explores and answers the following questions: What is big data? What are the basic methods for representing, managing and analyzing big data? What is the relationship between big data and knowledge? Can we find a mapping from big data into knowledge space? What kind of infrastructure is required to support not only big data management and analysis but also knowledge discovery, sharing and management? What is the relationship between big data and science paradigm? What is the nature and fundamental challenge of big data computing? A multi-dimensional perspective is presented toward a methodology of big data computing.Comment: 59 page

Evidence-centered Assessment for Writing with Generative AI

We propose a learning analytics-based methodology for assessing the collaborative writing of humans and generative artificial intelligence. Framed by the evidence-centered design, we used elements of knowledge-telling, knowledge transformation, and cognitive presence to identify assessment claims; we used data collected from the CoAuthor writing tool as potential evidence for these claims; and we used epistemic network analysis to make inferences from the data about the claims. Our findings revealed significant differences in the writing processes of different groups of CoAuthor users, suggesting that our method is a plausible approach to assessing human-AI collaborative writing

Assessment and Active Learning Strategies for Introductory Geology Courses

Educational research findings suggest that instructors can foster the growth of thinking skills and promote science literacy by incorporating active learning strategies into the classroom. This paper describes a variety of such strategies that may be adopted in introductory geology courses to encourage the development of higher-order thinking skills, and provides directions for implementing these techniques in the classroom. It discusses six hierarchical levels of student learning and links them to examples of appropriate assessment tools that were used successfully in several sections of a general education Earth Science course taught by two instructors at the University of Akron. These teaching strategies have been evaluated qualitatively using peer reviews, student written evaluations and semistructured student interviews; and quantitatively by measuring improvements in student retention, exam scores, and scores on a logical thinking assessment instrument. Educational levels: Graduate or professional

A Wikipedia Literature Review

This paper was originally designed as a literature review for a doctoral dissertation focusing on Wikipedia. This exposition gives the structure of Wikipedia and the latest trends in Wikipedia research

Cluster Analysis in Online Learning Communities: A Text Mining Approach

This paper presents a theory-informed blueprint for mining unstructured text data using mixed- and multi-methods to improve understanding of collaboration in asynchronous online discussions (AOD). Grounded in a community of inquiry theoretical framework to systematically combine established research techniques, we investigated how AOD topics and individual reflections on those topics affect formation of clusters or groups in a community. The data for the investigation came from 54 participants and 470 messages. Data analysis combined the analytical efficiency and scalability of topic modeling, social network analysis, and cluster analysis with qualitative content analysis. The cluster analysis found three clusters and that members of the intermediate cluster (i.e., middle of three clusters) played a pivotal role in this community by expressing uncertainty statements, which facilitated a collective sense-making process to resolve misunderstandings. Furthermore, we found that participants’ selected discussion topics and how they discussed those topics influenced cluster formations. Theoretical, practical, and methodological implications are discussed in depth

Latent Dirichlet Allocation for textual student feedback analysis

Ministry of Education, Singapore under its Academic Research Funding Tier 2; National Research Foundation (NRF) Singapore under International Research Centres in Singapore Funding Initiativ