Towards grounding computational linguistic approaches to readability: Modeling reader-text interaction for easy and difficult texts

Computational approaches to readability assessment are generally built and evaluated using gold standard corpora labeled by publishers or teachers rather than being grounded in observations about human performance. Considering that both the reading process and the outcome can be observed, there is an empirical wealth that could be used to ground computational analysis of text readability. This will also support explicit readability models connecting text complexity and the reader’s language proficiency to the reading process and outcomes. This paper takes a step in this direction by reporting on an experiment to study how the relation between text complexity and reader’s language proficiency affects the reading process and performance outcomes of readers after reading We modeled the reading process using three eye tracking variables: fixation count, average fixation count, and second pass reading duration. Our models for these variables explained 78.9%, 74% and 67.4% variance, respectively. Performance outcome was modeled through recall and comprehension questions, and these models explained 58.9% and 27.6% of the variance, respectively. While the online models give us a better understanding of the cognitive correlates of reading with text complexity and language proficiency, modeling of the offline measures can be particularly relevant for incorporating user aspects into readability models

Setting the Right Tone: How Data Science Enables Investor Communication to Choose the Right Language

Facts matter in financial communication, but academic research has revealed that soft information such as the tone and readability also impact the decision-making of investors and ultimately stock returns. Information Systems (IS) research has developed measures to quantify readability and tone of textual content, among which are techniques from statistical learning which are capable of extracting the most relevant words for investors. To make this knowledge accessible, we develop an IS prototype that guides communication practitioners to set the right tone in their investor communication. First, we identify relevant practitioner needs using requirements engineering. Second, we translate these needs into an IS prototype which seamlessly integrates into text editing processes and serves as a decision support system to steer readability and tone. Third, we pilot our IS prototype with 37 companies and successfully validate our prototype’s capabilities in a survey with financial professionals

Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

The accuracy of Automated Speech Recognition (ASR) technology has improved, but it is still imperfect in many settings. Researchers who evaluate ASR performance often focus on improving the Word Error Rate (WER) metric, but WER has been found to have little correlation with human-subject performance on many applications. We propose a new captioning-focused evaluation metric that better predicts the impact of ASR recognition errors on the usability of automatically generated captions for people who are Deaf or Hard of Hearing (DHH). Through a user study with 30 DHH users, we compared our new metric with the traditional WER metric on a caption usability evaluation task. In a side-by-side comparison of pairs of ASR text output (with identical WER), the texts preferred by our new metric were preferred by DHH participants. Further, our metric had significantly higher correlation with DHH participants' subjective scores on the usability of a caption, as compared to the correlation between WER metric and participant subjective scores. This new metric could be used to select ASR systems for captioning applications, and it may be a better metric for ASR researchers to consider when optimizing ASR systems.Comment: 10 pages, 8 figures, published in ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17

Unraveling the Relationship between Content Design and Kinesthetic Learning on Communities of Practice Platforms

As a variant of the sharing economy, Communities of Practice (CoP) platforms have allowed kinesthetic learners to acquire skillsets corresponding to their interests for immediate or future use in practice. However, the impact of digital learning content design on kinesthetic learning remains underexplored in the field of information systems. We hence extend prior research by advancing content richness and structure clarity as antecedents affecting kinesthetic learners’ digestibility of contents, culminating in differential kinesthetic learning effects. To substantiate our arguments, we collected data from a leading Chinese recipe sharing platform. Whereas content richness was measured in terms of readability, verb richness, and prototypicality, structure clarity was operationalized as block structure, block quantity, and block regularity. Employing a machine learning model, we simulated and tested learners’ digestibility of image content embodied within recipes. Plans for future research beyond the current study are also discussed