STARC: Structured Annotations for Reading Comprehension

We present STARC (Structured Annotations for Reading Comprehension), a new annotation framework for assessing reading comprehension with multiple choice questions. Our framework introduces a principled structure for the answer choices and ties them to textual span annotations. The framework is implemented in OneStopQA, a new high-quality dataset for evaluation and analysis of reading comprehension in English. We use this dataset to demonstrate that STARC can be leveraged for a key new application for the development of SAT-like reading comprehension materials: automatic annotation quality probing via span ablation experiments. We further show that it enables in-depth analyses and comparisons between machine and human reading comprehension behavior, including error distributions and guessing ability. Our experiments also reveal that the standard multiple choice dataset in NLP, RACE, is limited in its ability to measure reading comprehension. 47% of its questions can be guessed by machines without accessing the passage, and 18% are unanimously judged by humans as not having a unique correct answer. OneStopQA provides an alternative test set for reading comprehension which alleviates these shortcomings and has a substantially higher human ceiling performance.Comment: ACL 2020. OneStopQA dataset, STARC guidelines and human experiments data are available at https://github.com/berzak/onestop-q

The differential relations between verbal, numerical and spatial working memory abilities and children’s reading comprehension

Working memory predicts children's reading comprehension but it is not clear whether this relation is due to a modality-specific or general working memory. This study, which investigated the relations between children's reading skills and working memory (WM) abilities in 3 modalities, extends previous work by including measures of both reading comprehension and reading accuracy. Tests of word reading accuracy and reading comprehension, and working memory tests in three different modalities (verbal, numerical and spatial), were given to 197 6- to 11-year old children. The results support the view that working memory tasks that require the processing and recall of symbolic information (words and numbers) are better predictors of reading comprehension than tasks that require visuo-spatial storage and processing. The different measures of verbal and numerical working memory were not equally good predictors of reading comprehension, but their predictive power depended on neither the word vs. numerical contrast nor the complexity of the processing component. In general, performance on the verbal and numerical working memory tasks predicted reading comprehension, but not reading accuracy, and spatial WM did not predict either. The patterns of relations between the measures of working memory and reading comprehension ability were relatively constant across the age group tested

The Effectiveness of Paraphrasing Strategy in Increasing University Students' Reading Comprehension and Writing Achievement

Reading comprehension and writing as the crucial skills must be instructed effectively in order to engage the students in the meaningful teaching and learning process. One of the ways to increase students' reading comprehension and writing achievement is by the use of paraphrasing strategy in the classroom instruction. Through the application of the paraphrasing strategy, it is easy for the students to internalize the information of the original source comprehensively; thus, students' reading comprehension achievement is increased. In relation to the improvement of students' reading comprehension achievement, students' writing achievement is also increased by the use of paraphrasing strategy since the students can rewrite the text in to their own writing style. Therefore, the use of paraphrasing strategy is considered as one of the beneficial ways used to enhance students' reading comprehension and writing achievement

A Span-Extraction Dataset for Chinese Machine Reading Comprehension

Machine Reading Comprehension (MRC) has become enormously popular recently and has attracted a lot of attention. However, the existing reading comprehension datasets are mostly in English. In this paper, we introduce a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities in this area. The dataset is composed by near 20,000 real questions annotated on Wikipedia paragraphs by human experts. We also annotated a challenge set which contains the questions that need comprehensive understanding and multi-sentence inference throughout the context. We present several baseline systems as well as anonymous submissions for demonstrating the difficulties in this dataset. With the release of the dataset, we hosted the Second Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC 2018). We hope the release of the dataset could further accelerate the Chinese machine reading comprehension research. Resources are available: https://github.com/ymcui/cmrc2018Comment: 6 pages, accepted as a conference paper at EMNLP-IJCNLP 2019 (short paper

Assessing direct contributions of morphological awareness and prosodic sensitivity to children’s word reading and reading comprehension

We examined the independent contributions of prosodic sensitivity and morphological awareness to word reading, text reading accuracy, and reading comprehension. We did so in a longitudinal study of English-speaking children (N = 70). At 5 to 7 years of age, children completed the metalinguistic measures along with control measures of phonological awareness and vocabulary. Children completed the reading measures two years later. Morphological awareness, but not prosodic sensitivity made a significant independent contribution to word reading, text reading accuracy and reading comprehension. The effects of morphological awareness on reading comprehension remained after controls for word reading. These results suggest that morphological awareness needs to be considered seriously in models of reading development and that prosodic sensitivity might have primarily indirect relations to reading outcomes. Keywords: Morphological Awareness; Prosody; Word Reading; Reading Comprehension