Reading and Rereading Shakespeare’s Sonnets: Combining Quantitative Narrative Analysis and Predictive Modeling

Natural reading is rather like a juggling feat, as our eyes and minds are kept on several things at the same time. Instead, reading texts developed by researchers (so-called “textoids”; Graesser, Millis, & Zwaan, 1997) may be fairly simple, since this facilitates an experimental investigation. It thus provides the chance for clear statements regarding the effect of predefined variables. Likewise, most empirical studies focused only a few selected features while ignoring the great diversity of possibly important others (e.g., Rayner et al., 2001; Reichle, Rayner, & Pollatsek, 2003; Rayner & Pollatsek, 2006; Engbert et al., 2005; Rayner, 2009). However, it is not possible to directly transfer the results generated from textoids to natural reading due to the identification of more than 100 features on different hierarchical levels, which may influence processing a natural text (Graf, Nagler, & Jacobs, 2005; Jacobs, 2015a, b; Jacobs et al., 2017). The present dissertation differed from past research in that it used a literary text, i.e., Shakespeare’s sonnets, instead of texts constructed by the experimenter. The goal of the present dissertation was to investigate how psycholinguistic features may influence the reading behavior during poem perception. To this end, two problems need to be handled: Firstly, complex natural texts need to be broken up into measurable and testable features by “turning words into numbers” (Franzosi, 2010) for the sake of statistical analysis. Secondly, statistical ways were sought to deal with the non-linear webs of correlations among different features, which has long been a concern of Jacob’s working group (e.g., Willems, 2015; Willems & Jacobs, 2016; Jacobs & Willems, 2018). A quantitative narrative analysis (QNA) based predictive modeling approach was suggested to solve the above problems (e.g., Jacobs et al., 2017; Jacobs, 2017, 2018a, b). Since it is impossible to identify all relevant features of a natural text [e.g., over 50 features mentioned for single word recognition (Graf et al., 2005) or over 100 features computed for the corpus of Shakespeare sonnets (Jacobs et al., 2017)] and including more inter/supra-lexical features also requires extending sample sizes (i.e., more/longer texts and more participants), my dissertation focuses on lexical features. Seven of these are surface features (word length, word frequency, orthographic neighborhood density, higher frequency neighbors, orthographic dissimilarity index, consonant vowel quotient, and the sonority score) and two are affective-semantic features (valence and arousal). By applying the QNA-based predictive modeling approach, I conducted three eye tracking studies: study 1 (Chapter 5) asked English native speakers to read three of Shakespeare’s sonnets (sonnet 27, 60, and 66), aiming to investigate the role of seven surface psycholinguistic features in sonnets reading. Study 2 (Chapter 6) used a rereading paradigm and let another group of English natives read two of the three sonnets (sonnet 27 and 66), to find out whether the roles of the surface psycholinguistic features may be changed in rereading. In study 3 (Chapter 7), I reanalyzed the data of study 2, in which beyond the surface features I started to pay attention to the affective-semantic features, hoping to examine whether the roles of surface and affective-semantic features may be different throughout reading sessions. The three studies show highly reliable data for high feature importance of surface variables, and in rereading an increasing impact of affective-semantic features in reading Shakespeare’s sonnets. From a methodological viewpoint, all three studies show a much better sufficiency of neural net approach than the classical general linear model approach in psycholinguistic eye tracking research. For the rereading studies, in general, compared to the first reading, rereading improved the fluency of reading on poem level (shorter total reading times, shorter regression times, and lower fixation probability) and the depth of comprehension (e.g., Hakemulder, 2004; Kuijpers & Hakemulder, 2018). Contrary to the other rereading studies using literary texts (e.g., Dixon et al., 1993; Millis, 1995; Kuijpers & Hakemulder, 2018), no increase in appreciation was apparent. In summary, this dissertation can show that the application of predictive modeling to investigate poetry might be far more suitable to capture the highly interactive, non-linear composition of linguistic features in natural texts that guide reading behavior and reception. Besides, surface features seem to influence reading during all reading sessions, while affective-semantic features seem to increase their importance in line with processing depth as indicated by higher influence during rereading. The results seem to be stable and valid as I could replicate these novel findings using machine learning algorithms within my dissertation project. My dissertation project is a first step towards a more differentiated picture of the guiding factors of poetry reception and a poetry specific reading model

Performance Assessment of ChatGPT vs Bard in Detecting Alzheimer's Dementia

Large language models (LLMs) find increasing applications in many fields. Here, three LLM chatbots (ChatGPT-3.5, ChatGPT-4 and Bard) are assessed - in their current form, as publicly available - for their ability to recognize Alzheimer's Dementia (AD) and Cognitively Normal (CN) individuals using textual input derived from spontaneous speech recordings. Zero-shot learning approach is used at two levels of independent queries, with the second query (chain-of-thought prompting) eliciting more detailed than the first. Each LLM chatbot's performance is evaluated on the prediction generated in terms of accuracy, sensitivity, specificity, precision and F1 score. LLM chatbots generated three-class outcome ("AD", "CN", or "Unsure"). When positively identifying AD, Bard produced highest true-positives (89% recall) and highest F1 score (71%), but tended to misidentify CN as AD, with high confidence (low "Unsure" rates); for positively identifying CN, GPT-4 resulted in the highest true-negatives at 56% and highest F1 score (62%), adopting a diplomatic stance (moderate "Unsure" rates). Overall, three LLM chatbots identify AD vs CN surpassing chance-levels but do not currently satisfy clinical application.Comment: 22 page

AN ANALYSIS OF THE EFL SECONDARY READING CURRICULUM IN MALAYSIA: APPROACHES TO READING AND PREPARATION FOR HIGHER EDUCATION

This case study examined the overarching approaches to second language (L2) reading instruction reflected in the Malaysian EFL secondary curriculum and how well this curriculum prepares students for tertiary reading in EFL. The Malaysian context was chosen because it highly values EFL instruction and has many similarities with other English as Foreign Language (EFL) countries, in terms of EFL reading issues at the tertiary level. The research questions for this study included: What types of reading tasks are reflected in the Malaysian EFL secondary reading curriculum? What types and length of reading passages are used in the Malaysian Form Five English language textbook? What levels of cognitive demand of the reading tasks are reflected in the Malaysian EFL secondary reading curriculum? What types of learner roles are reflected in the Malaysian EFL secondary reading curriculum? This explorative study used document reviews as the primary data collection and analysis method. The Malaysian EFL Secondary Curriculum and the EFL secondary textbook were analyzed using a revision of Richards and Rodgers's (2001) framework for analyzing EFL teaching. The findings indicate that the Malaysian EFL secondary reading curriculum frequently uses reading as an explicit skill to achieve the listed learning outcomes in the EFL Secondary Curriculum. Nonetheless, the curriculum is developed based on the cognitive information processing theory of SLA, Top-Down theory of L2 reading reflecting Non-Interactive Whole Language instruction as well as learner roles that are primarily in the form of individual tasks. The findings on passage analysis show that the EFL textbook primarily uses narrative passages with the majority of passages below grade-level length. The curriculum, however, emphasizes reading tasks that require high cognitive demand as well as important types of reading tasks

Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Does Discussion Make a Difference in Vocabulary Learning From Expository Text Read Alouds?

Title of dissertation: DOES DISCUSSION MAKE A DIFFERENCE IN VOCABULARY LEARNING FROM EXPOSITORY TEXT READ ALOUDS? Sarah Beall Zelinke, Doctor of Philosophy, 2011 Dissertation directed by: Dr. Mariam Jean Dreher Department of Curriculum & Instruction University of Maryland, College Park This study investigated the effects of discussion on vocabulary learning from expository text read alouds. This study used a pre-/post within-subjects design to investigate whether discussion contributed to improved vocabulary knowledge from expository text read alouds and whether the placement of discussion makes a difference in vocabulary learning. Fifty-five second-grade students participated in a total of six read aloud sessions. There were two sessions for each of three expository texts. Intact classrooms were randomly assigned to condition by book. All participants experienced each of three discussion conditions, which varied by book. For the Discussion During (DD) condition, students experienced discussion of target vocabulary words during the read aloud sessions. For the Discussion After (DA) condition, students experienced discussion of target vocabulary words after the read aloud sessions. For the No Discussion (ND) condition, students listened to an expository text read aloud without discussing the text at all. An expressive vocabulary measure was used to examine growth in vocabulary knowledge. For each book, no difference was found for the ND condition. However, statistically significant treatment effects, with large effect sizes, were found for both the DA and DD conditions, indicating that discussion contributed to greater growth in vocabulary knowledge than no discussion. Post-hoc tests revealed that for one book, the DA condition led to significantly greater vocabulary growth than the DD condition. However, for the other two books there was no difference between the DD and DA conditions. These findings indicate that discussion of vocabulary words is an important factor in vocabulary learning

The Effects of Teaching a Specific Top-Level Structure on the Organization of Written Texts

The purpose of this study was to investigate the effectiveness of teaching a specific top-level structure on students\u27 recall and organization of expository text. The hypothesis to be investigated was that students explicitly taught the scientific report text structure schema would show improved recall and organization of written report text protocols. The report text structure utilized in this study was derived from Sloan and Latham\u27s top-level structure of text organization devised from schema theory and semantic memory models

Using visual representations to improve instructional materials for distance education computing students

Understanding how to develop instructional materials for distance education students is a challenging problem, but it is exacerbated when a domain is complex to teach, such as computer science. Visual representations have a history of use in computing as a means to alleviate the difficulties of learning abstract concepts. However, it is not clear whether improvements observed are as a result of improvements in the visual representations used in instructional materials or due to individual differences in students. This research examines the two themes of individual differences and visual representation in order to investigate how they collectively impact on improving instructional materials for distance education students studying computer science. It investigates the impact of different representations on learning while additionally investigating the relationship between individual differences and student learning.The research in this thesis shows that visual representations are important in designing instructional materials. In particular, texts with visual representations have the power to cue students to perceive instructional materials as easier to process and more engaging.Investigation into the impact of concrete high-imagery versus abstract low-imagery visual representations illustrated that concrete visual representations incurred fewer cognitive overheads for computer science students and were able to ameliorate the challenges of learning computing.The research in this thesis into individual differences demonstrated that Imagers did benefit more from studying instructional materials containing text with visual components. However the research indicates that appropriate selection of individual difference tests is dependent upon the application, i.e., whether the results are to be used to assess generalised tendencies or episodes in learning and whether the tests examine underlying approaches to cognition or practices in education.An underlying question was whether students studying instructional materials containing low-imagery visual representations would cope as well as those studying high-imagery ones. Accomplished learners demonstrated that they could perform as well as with those receiving high-imagery visual representations. However, studying and recalling these materials did incur more cognitive processing.This thesis argues that improving instructional materials by including appropriate visual representations is a useful basis for improving learning for distance education computer science students