Fine-tuning Large Language Model (LLM) Artificial Intelligence Chatbots in Ophthalmology and LLM-based evaluation using GPT-4

Purpose: To assess the alignment of GPT-4-based evaluation to human clinician experts, for the evaluation of responses to ophthalmology-related patient queries generated by fine-tuned LLM chatbots. Methods: 400 ophthalmology questions and paired answers were created by ophthalmologists to represent commonly asked patient questions, divided into fine-tuning (368; 92%), and testing (40; 8%). We find-tuned 5 different LLMs, including LLAMA2-7b, LLAMA2-7b-Chat, LLAMA2-13b, and LLAMA2-13b-Chat. For the testing dataset, additional 8 glaucoma QnA pairs were included. 200 responses to the testing dataset were generated by 5 fine-tuned LLMs for evaluation. A customized clinical evaluation rubric was used to guide GPT-4 evaluation, grounded on clinical accuracy, relevance, patient safety, and ease of understanding. GPT-4 evaluation was then compared against ranking by 5 clinicians for clinical alignment. Results: Among all fine-tuned LLMs, GPT-3.5 scored the highest (87.1%), followed by LLAMA2-13b (80.9%), LLAMA2-13b-chat (75.5%), LLAMA2-7b-Chat (70%) and LLAMA2-7b (68.8%) based on the GPT-4 evaluation. GPT-4 evaluation demonstrated significant agreement with human clinician rankings, with Spearman and Kendall Tau correlation coefficients of 0.90 and 0.80 respectively; while correlation based on Cohen Kappa was more modest at 0.50. Notably, qualitative analysis and the glaucoma sub-analysis revealed clinical inaccuracies in the LLM-generated responses, which were appropriately identified by the GPT-4 evaluation. Conclusion: The notable clinical alignment of GPT-4 evaluation highlighted its potential to streamline the clinical evaluation of LLM chatbot responses to healthcare-related queries. By complementing the existing clinician-dependent manual grading, this efficient and automated evaluation could assist the validation of future developments in LLM applications for healthcare.Comment: 13 Pages, 1 Figure, 8 Table

Attitudes towards Finnish-accented English

The thesis opens with a discussion of what attitudes are, and develops with a review of studies of attitudes towards pronunciation error, attitudes towards foreign accents and perception of foreign-accented speakers. The empirical part of the thesis attempts to identify how native (British) and Finnish listeners of English react to and evaluate typical segmental features of mispronunciation in the English speech of Finnish men and women of various ages. Two experiments using modifications of the matched-guise technique were conducted, one to consider error evaluation and to establish a hierarchy of segmental mispronunciation, the other to examine speaker evaluation, the image of the speaker created by the mispronunciation. Recordings of Finnish-accented English were presented to male and female listeners of various ages, and reactions collected. Statistical analyses of the results were carried out and the following general conclusions were drawn: the English labiodental lenis fricative /v/ when mispronounced in the typical Finnish manner as a labiodental frictionless continuant [u] is not tolerated by native English listeners at all, though it is highly tolerated by Finnish-speaking listeners (and Swedish-speaking Finns) themselves; the degree of mispronunciation in Finnish-accented English seriously affects listeners' estimations of the speaker's age, bad mispronunciation prompting under-estimation of age and good pronunciation over-estimation; both Finnish-speaking listeners and English-speaking listeners have almost identical clear pre-set standards about what constitutes `good' and `bad' pronunciation; a Finnish speaker's phonemically `better' and `worse' pronunciation affects the image listeners have of the speaker, status/competence traits in particular being up-graded for better pronunciation, solidarity/benevolence traits remaining broadly unaffected, and Englishspeaking listeners generally being more positive towards the Finnish-accented speakers than compatriot Finns

Far From the Dream : Exploring the Gap in Educational Opportunities for Black Americans

Substantial evidence demonstrates the inequity in educational opportunity that currently exists for Black students in U.S. schools. Combatting this inequity requires an understanding of what educational opportunity looks like in practice and insight into the various mechanisms that maintain and reproduce inequity. This dissertation explores the topic of educational opportunity through the lens of race with the aim of identifying the ways in which schools and policies can ensure all students receive the necessary support to be successful both in and out of school. In essence, this dissertation aims to capture what makes an education freeing and equitable. To address the research aims, this dissertation employed a qualitatively driven, multi-method research design that consisted of four interrelated studies. The research approach and methods used for each of the four studies differed, but in combination, the studies sought to fulfill the overarching project aims. The multi-method design enabled a complex examination of racial inequity and educational opportunity from multiple perspectives and at four different levels: conceptual, policy, school, and individual. The first study was a conversion mixed methods integrative review that focused on educational opportunity at the conceptual level. The study resulted in the development of an educational opportunity framework that has implications for both practice and theory. The framework can be adapted in different contexts to guide schools and researchers in evaluating or improving equity in educational opportunities. Study II adopted an embedded mixed methods case study approach to examine the racial status quo within a majority-White high school (school level) and Black students’ experiences within that school (individual level). The analysis resulted in the development of a figure depicting the cycle of inequity within the racial status quo and how this impacted students’ experiences. The figure can be used as a model for schools to examine their own status quo, and improve practices and curriculum to ensure all students are provided equitable opportunities. The third study used interpretative phenomenological analysis to more closely examine the same set of data as Study II by focusing on the qualitative data collected from the five Black girl participants (individual level). The study provides insight into school practices that result in feelings of entrapment, as well as more supportive structures and practices that give students a sense of freedom. Based on these results, suggestions are made for how schools can be more autonomy-supportive and inclusive. The suggestions do not just apply to Black girls, but could lead to a more freeing education for the broader student population. The final study was a convergent mixed methods critical policy analysis that explored how racial/ethnic equity was promoted or inhibited in 61 educational policies that were introduced and enacted between 2020 and 2022. Though the majority of policies were found to promote equity, a critical evaluation of the policies revealed that those promoting equity presented more symbolic rather than meaningful action. In addition, they failed to address many of the structural issues that reproduce racial inequity. These findings necessitate education policies that move beyond race neutrality and explicitly target systemic racism. The results of these studies taken together demonstrate the ways that racial inequity occurs through school practices and policies, and what this inequity means for Black students’ school experiences. By centering Black students’ experiences in a majority-White school, this dissertation points to the more subtle mechanisms that exist within schools that create inequity in accessing opportunities and having positive school experiences. Moreover, this dissertation examines educational opportunity at multiple levels enabling a more complex understanding of the interconnected factors that contribute to inequity. In sum, this dissertation contributes to both theory and practice by offering practical solutions to improving racial equity and presenting new frameworks for evaluating and conceptualizing educational opportunity.Forskning visar att svarta elever i USA:s skolor för närvarande inte har lika stora möjligheter till utbildning. För att motarbeta denna ojämlikhet krävs en förståelse för hur utbildningsmöjligheterna ser ut i praktiken och en insikt i de olika mekanismer som upprätthåller och reproducerar orättvisa. Denna avhandling utforskar temat utbildningsmöjligheter via ras som lins med målet att identifiera hur skolor och politik kan säkerställa att alla elever får det stöd som krävs för att lyckas både i och utanför skolan. I huvudsak syftar avhandlingen till att fånga upp vad som gör en utbildning emancipatorisk och rättvis. I avhandlingen tillämpas en kvalitativt driven forskningsdesign med flera metoder i fyra sammanhängande studier. Forskningsansatsen och metoderna för de fyra studierna är olika men besvarar sammantaget de övergripande forskningsfrågorna. Designen med flera metoder möjliggör en komplex undersökning av ojämlikhet som baserar sig på ras och utbildningsmöjligheter ur flera olika perspektiv och på fyra olika nivåer: konceptuell, politisk, institutionell (skola) och individuell. Den första studien är en studie där kvantitativa och kvalitativa metoder med ett mixed methods upplägg fokuserar på utbildningsmöjligheter på konceptuell nivå. Som resultat av studien har utvecklats ett ramverk för utbildningsmöjligheter med konsekvenser för både teori och praktik. Ramverket kan anpassas till olika sammanhang för att vägleda skolor och forskare när det gäller att utvärdera eller förbättra jämlika utbildningsmöjligheter. I den andra studien används en fallstudie med mixed methods för att undersöka status quo gällande ras inom en gymnasieskola med vit majoritet (skolnivå) och fem svarta elevers erfarenheter av skolan (individnivå). Analysen resulterade i en cirkelmodell som synliggör ojämlikhet i fråga om status quo enligt ras och hur detta påverkar elevernas erfarenheter. Cirkelmodellen kan användas av skolor för att undersöka egen status quo och förbättra praxis och undervisningsinnehåll för att se till att alla elever får jämlika möjligheter. I den tredje studien används tolkande fenomenologisk analys för att närmare undersöka samma uppsättning data som i den andra studien genom att fokusera på de kvalitativa data som samlades in från fem svarta flickor som deltog (individuell nivå). Studien ger en inblick i skolpraktiker som leder till en känsla av att vara fångad (entrapment), men också mer stödjande strukturer och praktiker som ger eleverna en känsla av frihet. Utifrån dessa resultat ges förslag på hur skolor kan vara mer autonomistödjande och inkluderande. Förslagen gäller inte bara svarta flickor, utan kan leda till en mer emancipatorisk utbildning för en bredare elevpopulation. Den sista studien är en kritisk policyanalys där konvergent mixed methods används i syfte att undersöka hur rasmässig/etnisk jämlikhet främjades eller förhindrades i 61 utbildningspolitiska policyer som infördes och antogs mellan 2020 och 2022. Även om majoriteten av policyområdena visade sig främja jämlikhet, visade en kritisk utvärdering av områdena att de som främjade jämlikhet snarare var symboliska än meningsfulla. Dessutom misslyckades policyerna med att ta itu med många av de strukturella frågor som reproducerar rasrelaterad ojämlikhet. Dessa resultat kräver en utbildningspolitik som går bortom rasneutralitet och uttryckligen riktar sig mot systemisk rasism. Resultaten av dessa studier visar sammantaget hur rasmässig ojämlikhet uppstår genom skolpraktik och skolpolicyer och vad denna ojämlikhet innebär för svarta elevers skolgång. Genom att fokusera på svarta elevers erfarenheter i en skola med vit majoritet pekar den här avhandlingen på de mer subtila mekanismer som finns i skolorna och som skapar ojämlikhet när det gäller att få tillgång till möjligheter och positiva skolupplevelser. Dessutom undersöker avhandlingen utbildningsmöjligheter på flera olika nivåer, vilket möjliggör en mer komplex förståelse av de sammankopplade faktorer som bidrar till ojämlikhet. Sammanfattningsvis bidrar denna avhandling till både teori och praktik genom att erbjuda praktiska lösningar för att förbättra jämlikheten mellan raser samtidigt som den presenterar nya ramar för att utvärdera och konceptualisera utbildningsmöjligheter

Blinding eye disease in Western Australia: perspectives on data integration

This thesis explores the utility of linked hospital administrative data for evaluating blinding eye diseases and the eye care provided for them in Western Australia. Alternative data sources and methodologies that complement linked data methods were explored. Areas of study were complications of cataract surgery, the epidemiology of blindness, diabetic retinopathy screening and management, causes of vision loss in remote Aboriginals, and the safety of intravitreal anti-vascular endothelial growth factors in age-related macular degeneration

An embodied approach to disability sport: the lived experience of visually impaired cricket players

This thesis investigates the England Visually Impaired Cricket Team, whose squad members comprise sixteen men aged 18-54, and their lived experiences' of playing visually impaired cricket. This is the first piece of research to examine elite visually impaired cricket and the first to explicitly analyse the social dynamics of any visually impaired sports team. Through an embodied theoretical approach, that accounts for the corporeal experience of impairment alongside the role of social institutions and discourse in the high performance culture of modern disability sport, this thesis establishes the significant aspects of this previously unexamined research 'site', both on and off the pitch. This study consisted of ten months of ethnographic fieldwork using participant observation and semi-structured interviews shaped by a new method of recording and eliciting data. To capture the participants' sensorial experiences of playing visually impaired cricket, 'soundscape elicitation', the process of composing auditory 'tracks' of the players' participation and then using these recordings during semi-structured interviews to prompt sensorial discussions, was utilised. This original and innovative method was central to the production of previously unexamined knowledge and is a significant methodological advancement in the wider field of sensory studies. The findings present a number of original contributions to knowledge regarding 'sporting bodies', the sensorial experiences of sport, and the construction of identity through disability sport. The participants' embodied experiences of playing visually impaired cricket reveal an alternative way of 'being' in sport and physical activity. However, it is the inescapable ocularcentric value of 'sight' that inhibits the resistive potential of the game. Instead of the presumed empowering experience, elite visually impaired cricket is disempowering for many participants due to the irreversible relationship of blind cricket institutions with mainstream cricketing bodies. Furthermore, a 'hierarchy of sight' based upon the official sight classification process emerges that highly values those players with the highest sight classifications and marginalises the blind players. All of these factors inform visually impaired cricket players’ construction of their own identities. Although many players view visually impaired cricket as a way of demonstrating their 'normality', it actually accentuates the impairment that they are attempting to dissociate from and is one of the few social situations where they are 'outed' as disabled or blind

Reuniting word and deed : negotiation for real peacemaking and authentic classroom writing

The issue at stake in this dissertation is the relationship between word and deed. The problem it addresses is the way in which categories of discourse undermine that relationship. It argues that discourse taxonomies divide word from deed because they categorize persuasion and deliberation as characteristics of some uses of language but not others. This splitting of word and deed Informs and 1s informed by other divisions—between writer and reader, meaning and consequence, form and content, text and context. As a result, it silences the second part of each of these hierarchies—reader, consequence, content, and context. These divisions, this dissertation illustrates, represent an inaccurate and destructive theory and practice of language. The first chapter discusses differences in a theory of rhetoric as all language use and of rhetoric as one use of language. It argues in favor of Kenneth Burke's dialogical philosophy of language as symbolic action and against dlchotomous theories of rhetoric as a singular category of discourse. The second chapter analyzes the contemporary theory of one category in particular—epideictic—as evidence of the erroneous and debilitating effects of the dichotomy of deliberative/epideictic created by discursive categories

Race(ing) around in rhetoric and composition circles: racial literacy as the way out

Rhetoric and composition studies has no recognizable, critical theory or body of knowledge about race that attends to the field's foundation: language, rhetoric, persuasion, literacy, or writing. The field is in a crisis, a theoretical wasteland, when it comes to critical race praxis. Presently, it is stuck in racially liberal paradigms, such as multiculturalism, diversity, and color-blindness that are anti-progressive and linguistically and rhetorically lacking. These paradigms sustain what Charles W. Mills terms an "epistemology of ignorance" that regards knowledge of race and racism as hazardous to white dominance (17). To respond, I propose a theory of racial literacy that answers Keith Gilyard's call for a "narrative about racial formation that would be useful in composition classrooms, one accessible yet sufficient in scope" (49). I argue that racial literacy is a topic of investigation for the field of rhetoric and composition studies because it examines race as a discursive system with implications for knowledge construction, interpretation, and literacy performance. To move from racial liberalism to racial literacy, the field needs to make three shifts: 1) shift the rhetorical subject from the individual to language; 2) shift the theoretical paradigm from racial liberalism to racial literacy, and 3) shift from a pedagogy stuck between racial liberalism and critical pedagogy to one grounded in racial literacy strategies. The three shifts I propose should spark a new discussion in the field about students' right to study language, similar to the 1974 College Composition and Communication Resolution Students' Right to Their Own Language. In chapter 2, I survey the field of rhetoric and composition to reveal its lack of critical engagement with race. In chapter 3, I outline the theoretical foundation of racial literacy, analyzing race as a discursive system. Chapter 4 describes my experiences implementing racial literacy in a rhetoric and composition classroom, while being attentive to university parameters and students' rights to study language. I also provide a preliminary examination of the implications of teaching in the age of Barack Obama since his election as President of the United States. Looking to the future, chapter 5 lays out strategies for grounding racial literacy in the field

Addressing diversity: A case study of teacher educators\u27 views and instructional practices at a Midwestern university

It is well documented that while the student population is becoming more and more diverse, the teacher population remains mostly white, female, monolingual, and monocultural (Banks, 2001a; Darling-Hammond, 2002; Nieto, 2000a). Consequently, teacher education programs have the important role of addressing diversity throughout curriculum, field experiences and faculty recruitment. The purpose of this qualitative research was to gain more understanding, from the teacher educators\u27 perspectives, about the way diversity is being addressed at a Midwestern university by examining teacher educators\u27 various methods of instruction, and their course materials, which focus on diversity issues and the needs of diverse learners. The case study methodology concentrated on three full-time, tenured professors from Special Education Department, who are teaching the same required course entitled Meeting the Needs of Diverse Learners for the preservice teachers. The primary strategies of data collection were individual interviews, class observations and the study of written documentation such as vitas, syllabi and extra materials used during the classes. The cross-case analysis of the data was related, through the emergent themes, with the postmodern perspective. The results emphasized the benefits of using postmodernist aspects (such as knowledge construction, and local narratives and the proliferation of difference/the Other ) when teaching diversity issues with a strength-based approach within the curriculum and specific instructional practices for a transformative, meaning-making learning experience. Even with different approaches to teaching, all three professors linked their knowledge, beliefs, and values with the practicality of it, challenging students to use constant reflection in their learning experiences. The findings raised the issues of presumed neutrality in the classroom, and the possibility of renaming the course as Recognizing the Strengths and Meeting the Needs of Diverse Learners. Among the further recommendations, the following were mentioned: more inter- and intra-departmental information, collaboration and dialogue regarding diversity; cross-curricular teaching and curriculum changes with the inclusion of field experiences in diverse classrooms; and faculty recruitment from diverse backgrounds. More research regarding how to teach a course on diversity issues was also recommended

Is empathy an emotion?

The main aim of my thesis is to ascertain whether empathy has the required qualities of an emotion. Disagreement is rife regarding the process leading to the arousal of an emotion, which creates uncertainty as to what exactly an emotion is, and how it appears. This is the first issue I tackle in my work, as I concentrate on examining some of the significant cognitive and feeling theories of emotions. My study of these theories outlines their downsides, and I instead propose to retain a hybrid definition that combines the advantages of both families of theories to provide a balanced approach that recognises the importance of both physical changes and cognitions. The focus of my work then moves specifically onto empathy, with the intention of precisely defining this term too, its functioning, as well as the meaning of the expression ‘feeling empathy for someone’. The existing literature on empathy fails to provide a clear understanding of empathy’s classification as an emotion or a skill. My work is original in that I avoid assertions and clearly establish that empathy constitutes an emotion based on the definition of emotion I advance in the first part of my work