Large language models approach expert-level clinical knowledge and reasoning in ophthalmology:A head-to-head cross-sectional study

Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64–90%), ophthalmology trainees (median 59%, range 57–63%), and unspecialised junior doctors (median 43%, range 41–44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p > 0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p < 0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted

Is Sensory Loss an Understudied Risk Factor for Frailty? A Systematic Review and Meta-analysis

[Abstract] Background. Age-related sensory loss and frailty are common conditions among older adults, but epidemiologic research on their possible links has been inconclusive. Clarifying this relationship is important because sensory loss may be a clinically relevant risk factor for frailty. Methods. In this systematic review and meta-analysis, we searched 3 databases for observational studies investigating 4 sensory impairments—vision (VI), hearing (HI), smell (SI), and taste (TI)—and their relationships with frailty. We meta-analyzed the cross-sectional associations of VI/HI each with pre-frailty and frailty, investigated sources of heterogeneity using meta-regression and subgroup analyses, and assessed publication bias using Egger’s test. Results. We included 17 cross-sectional and 7 longitudinal studies in our review (N = 34,085) from 766 records. Our cross-sectional meta-analyses found that HI and VI were, respectively, associated with 1.5- to 2-fold greater odds of pre-frailty and 2.5- to 3-fold greater odds of frailty. Our results remained largely unchanged after subgroup analyses and meta-regression, though the association between HI and pre-frailty was no longer significant in 2 subgroups which lacked sufficient studies. We did not detect publication bias. Longitudinal studies largely found positive associations between VI/HI and frailty progression from baseline robustness, though they were inconclusive about frailty progression from baseline pre-frailty. Sparse literature and heterogenous methods precluded meta-analyses and conclusions on the SI/TI–frailty relationships. Conclusions. Our meta-analyses demonstrate significant cross-sectional associations between VI/HI with pre-frailty and frailty. Our review also highlights knowledge gaps on the directionality and modifiability of these relationships and the impact of SI/TI and multiple sensory impairments on frailty

Childhood acid suppressants may increase allergy risk - a systematic review and meta-analysis

10.1016/j.jaip.2022.09.042Journal of Allergy and Clinical Immunology: In Practic

Bidirectional association between glaucoma and chronic kidney disease: A systematic review and meta-analysis

10.1016/j.eclinm.2022.101498ECLINICALMEDICINE4

Efficacy of covid-19 vaccines in immunocompromised patients: systematic review and meta-analysis

10.1136/bmj-2021-068632BMJ-BRITISH MEDICAL JOURNAL37

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study.

Acknowledgements: The authors extend their thanks to Mr Arunachalam Thirunavukarasu (Betsi Cadwaladr University Health Board) for his advice and assistance with recruitment.Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64-90%), ophthalmology trainees (median 59%, range 57-63%), and unspecialised junior doctors (median 43%, range 41-44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p>0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p<0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted

Factors associated with self-reported burnout level in allied healthcare professionals in a tertiary hospital in Singapore.

BackgroundBurnout has adverse implications in healthcare settings, compromising patient care. Allied health professionals (AHPs) are defined as individuals who work collaboratively to deliver routine and essential healthcare services, excluding physicians and nurses. There is a lack of studies on burnout among AHPs in Singapore. This study explored factors associated with a self-reported burnout level and barriers to seeking psychological help among AHPs in Singapore.MethodsWe conducted a cross-sectional study in a sample of AHPs in a tertiary hospital from October to December 2019. We emailed a four-component survey to 1127 eligible participants. The survey comprised four components: (1) sociodemographic characteristics, (2) Maslach Burnout Inventory (MBI-HSS), (3) Areas of Worklife Survey, and (4) Perceived Barriers to Psychological Treatment (PBPT). We performed a multiple logistic regression analysis to identify factors associated with burnout. Adjusted odds ratios (AORs) and associated 95% confidence intervals (CIs) were computed.ResultsIn total, 328 participants completed the questionnaire. The self-reported burnout level (emotional exhaustion>27 and/or depersonalization>10) was 67.4%. The majority of the respondents were female (83.9%), Singaporean (73.5%), aged 40 years and below (84.2%), and Chinese ethnicity (79.9%). In the multiple logistic regression model, high burnout level was negatively associated with being in the age groups of 31 to 40 (AOR 0.39, 95% CI 0.16-0.93) and 40 years and older (AOR 0.30, 95% CI 0.10-0.87) and a low self-reported workload (AOR 0.35, 95% CI 0.23-0.52). High burnout level was positively associated with a work experience of three to five years (AOR 5.27, 95% CI 1.44-20.93) and more than five years (AOR 4.24; 95% CI 1.16-16.79. One hundred and ninety participants completed the PBPT component. The most frequently cited barriers to seeking psychological help by participants with burnout (n = 130) were 'negative evaluation of therapy' and 'time constraints.'ConclusionsThis study shows a high self-reported burnout level and identifies its associated factors among AHPs in a tertiary hospital. The findings revealed the urgency of addressing burnout in AHPs and the need for effective interventions to reduce burnout. Concurrently, proper consideration of the barriers to seeking help is warranted to improve AHPs' mental well-being

Results of statistical tests of variation in performance between question subjects and types, for each trialled LLM, expert ophthalmologist, and trainee doctor.

Statistically significant results are highlighted in green. (XLSX)</p

Heat map of Kappa statistics quantifying agreement between answers given by LLMs, expert ophthalmologists, and trainee doctors.

Agreement correlates strongly with overall performance and stratification analysis found no particular question type or subject was associated with better performance of LLMs or doctors, indicating that LLM knowledge and reasoning ability is general across ophthalmology rather than restricted to particular subspecialties or question types.</p

Experience of expert ophthalmologists (E1-E5), ophthalmology trainees (T1-T3), and unspecialised junior doctors (J1-J2) involved in experiments.

Experience of expert ophthalmologists (E1-E5), ophthalmology trainees (T1-T3), and unspecialised junior doctors (J1-J2) involved in experiments.</p