Fine-tuning Large Language Model (LLM) Artificial Intelligence Chatbots in Ophthalmology and LLM-based evaluation using GPT-4

Purpose: To assess the alignment of GPT-4-based evaluation to human clinician experts, for the evaluation of responses to ophthalmology-related patient queries generated by fine-tuned LLM chatbots. Methods: 400 ophthalmology questions and paired answers were created by ophthalmologists to represent commonly asked patient questions, divided into fine-tuning (368; 92%), and testing (40; 8%). We find-tuned 5 different LLMs, including LLAMA2-7b, LLAMA2-7b-Chat, LLAMA2-13b, and LLAMA2-13b-Chat. For the testing dataset, additional 8 glaucoma QnA pairs were included. 200 responses to the testing dataset were generated by 5 fine-tuned LLMs for evaluation. A customized clinical evaluation rubric was used to guide GPT-4 evaluation, grounded on clinical accuracy, relevance, patient safety, and ease of understanding. GPT-4 evaluation was then compared against ranking by 5 clinicians for clinical alignment. Results: Among all fine-tuned LLMs, GPT-3.5 scored the highest (87.1%), followed by LLAMA2-13b (80.9%), LLAMA2-13b-chat (75.5%), LLAMA2-7b-Chat (70%) and LLAMA2-7b (68.8%) based on the GPT-4 evaluation. GPT-4 evaluation demonstrated significant agreement with human clinician rankings, with Spearman and Kendall Tau correlation coefficients of 0.90 and 0.80 respectively; while correlation based on Cohen Kappa was more modest at 0.50. Notably, qualitative analysis and the glaucoma sub-analysis revealed clinical inaccuracies in the LLM-generated responses, which were appropriately identified by the GPT-4 evaluation. Conclusion: The notable clinical alignment of GPT-4 evaluation highlighted its potential to streamline the clinical evaluation of LLM chatbot responses to healthcare-related queries. By complementing the existing clinician-dependent manual grading, this efficient and automated evaluation could assist the validation of future developments in LLM applications for healthcare.Comment: 13 Pages, 1 Figure, 8 Table

The Effect of cue attributes on the willingness to engage car-sharing service providers

This paper examined the predictive power of various variables, which include intermediary trust, perceived convenience, perceived utility, and summary evaluations, in influencing consumers’ willingness to use a car-sharing service provider. In the study, participants (N=259) were exposed to six mockups that depicted a range of summary evaluations (star ratings) of a sharing service provider before indicating their willingness to engage the depicted service provider. Aligning with previous studies, the results revealed that summary evaluations played a pivotal role in affecting perceptions about trust in the service provider and service quality standards, which in turn predicted willingness to engage the sharing service provider. Results also showed that willingness to use the sharing service acts as a mediator on willingness to use highly-rated service providers. Future research can focus on exploring attitudes and beliefs attached to the varying levels of summary evaluations that resulted in differences in willingness to engage the service provider.Bachelor of Communication Studie

Reputation cues as signals in the sharing economy

Reputation cues, like star ratings, signal qualities of service providers in the sharing economy and may affect user behavior. Guided by concepts from signaling theory and using a repeated measures experiment (N = 221), this study manipulated the level of star ratings of ride sharing drivers. Intuitive findings are perceived service quality and willingness to use the service provider are higher when the star rating is high versus low. Extending prior work, perceived service quality mediates the effect of reputation on willingness, explaining 83% of the total effect. Also, the direct effect of reputation cues on perceived service quality depends, albeit weakly (η2p = 0.02), on how much users say they pay attention to them. These novel findings clarify the kinds of mental processing that occur when users of shared services evaluate reputation cues. We discuss findings in terms of costly signaling and consider practical implications for users and providers.Published versio

Traumatic Globe Luxation With Chiasmal Avulsion

Background: To describe an unusual case of traumatic globe luxation with optic chiasmal avulsion and review the existing literature on this rare condition for further discussion of mechanisms, diagnosis, and management. Methods: Case report and review of existing case reports and case series identified through literature search. Results: A 28-year-old woman, with no previous medical history, had left globe luxation and optic chiasm avulsion after being stabbed directly into the left orbit with the use of the stiletto high heel of a shoe. Automated visual field testing detected a temporal hemianopia in the unaffected eye despite normal central visual acuity. Chiasmal avulsion was demonstrated by MRI. Conclusions: This case suggests that perimetry and MRI should always be considered in traumatic globe luxation to localize the site of injury. Temporal hemianopia in the fellow eye indicates a concomitant chiasmal injury

Exclusive enteral nutrition with concomitant early thiopurine use was effective in maintaining steroid-free remission in a Southeast Asian cohort of children with Crohn’s disease

Abstract Background Exclusive enteral nutrition (EEN) is as effective as corticosteroids in inducing remission in children with Crohn’s disease (CD). However, over 50% of these children relapse by 12 months of diagnosis. Thiopurines are commonly prescribed as maintenance therapy for CD, but evidence for its efficacy is controversial. Data on the effectiveness of EEN in Southeast Asian (SEA) children with CD is scarce. This study aims to evaluate the efficacy of EEN induction therapy in a cohort of SEA children with newly diagnosed CD. The secondary aim was to evaluate concomitant early azathioprine (EAZ) use in determining remission rate at 6 and 12 months. Methods Case records of all children with newly diagnosed CD from 2011 to 2014 were reviewed and relevant demographic as well as clinical data were extracted. The primary outcome measure was the number of patients who completed EEN induction therapy and achieved remission (Paediatric Crohn’s Disease Activity Index; PCDAI≤10). Factors influencing duration of remission were evaluated in particular early azathioprine (EAZ) defined as starting azathioprine within one month of diagnosis versus late azathioprine (LAZ) use. Results Forty children with newly diagnosed CD were identified. Thirty-three children: 67% boys, median age 13y (range 3–17) completed 8 weeks of EEN induction therapy and 91% achieved remission. Significant improvements were seen in PCDAI scores (32.7 ± 9.2 to 4.2 ± 5.1; p < 0.001), mean BMI z-score (− 1.38 ± 1.57 to − 0.82 ± 1.27; p = 0.004) and baseline inflammatory markers: Erythrocyte Sedimentation Rate (51.6 ± 30.1 mm/h to 13.3 ± 7.1 mm/h; p < 0.0001) C-Reactive Protein (44.6 ± 51.0 mg/L to 5.2 ± 7.6 mg/L; p = 0.001), Albumin (30.7 ± 7.5 g/L to 38.7 ± 3.9 g/L; p < 0.0001), Platelets (464 ± 161 × 109 to 370 ± 111 × 109; p < 0.0001),. Early azathioprine initiation was associated with a remission rate of 80 and 73% at 6 and 12 months respectively. Remission was also maintained for longer duration in EAZ vs LAZ groups (p = 0.048). Conclusion EEN effectively induces remission in this cohort of SEA children with newly diagnosed CD. Early initiation of thiopurine with EEN induction therapy is effective in maintaining steroid-free remission for at least one year

Ag/AgFeO₂: An Outstanding Magnetically Responsive Photocatalyst for HeLa Cell Eradication

A superfast, room-temperature, one-step carrier-solvent-assisted interfacial reaction process was developed to prepare Ag/AgFeO₂ composite nanocrystals (NCs) of less than 10 nm in size within a 1 min reaction time. These composite NCs were with a direct energy band gap of 2.0 eV and were paramagnetic, making them suitable for optical activation and magnetic manipulation. These composite NCs, applied as a photocatalyst for the treatment of HeLa cells, achieved a significant reduction of 74% in cell viability within 30 min. These Ag/AgFeO₂ composite NCs proved to be a promising magnetically guidable photocatalyst for cancer cell treatment