Named Entity Recognition in Electronic Health Records Using Transfer Learning Bootstrapped Neural Networks

Neural networks (NNs) have become the state of the art in many machine learning applications, especially in image and sound processing [1]. The same, although to a lesser extent [2,3], could be said in natural language processing (NLP) tasks, such as named entity recognition. However, the success of NNs remains dependent on the availability of large labelled datasets, which is a significant hurdle in many important applications. One such case are electronic health records (EHRs), which are arguably the largest source of medical data, most of which lies hidden in natural text [4,5]. Data access is difficult due to data privacy concerns, and therefore annotated datasets are scarce. With scarce data, NNs will likely not be able to extract this hidden information with practical accuracy. In our study, we develop an approach that solves these problems for named entity recognition, obtaining 94.6 F1 score in I2B2 2009 Medical Extraction Challenge [6], 4.3 above the architecture that won the competition. Beyond the official I2B2 challenge, we further achieve 82.4 F1 on extracting relationships between medical terms. To reach this state-of-the-art accuracy, our approach applies transfer learning to leverage on datasets annotated for other I2B2 tasks, and designs and trains embeddings that specially benefit from such transfer.Comment: 11 pages, 4 figures, 8 table

Headache and type 2 diabetes association: a US national ambulatory case-control study

Objective We investigate the joint observation between type 2 diabetes and headache using a case-control study of a US ambulatory dataset. Background Recent whole-population cohort studies propose that type 2 diabetes may have a protective effect against headache prevalence. With headaches ranked as a leading cause of disability, headache-associated comorbidities could help identify shared molecular mechanisms. Methods We performed a case-control study using the US National Ambulatory Medical Care Survey, 2009, on the joint observation between headache and specific comorbidities, namely type 2 diabetes, hypertension and anxiety, for all patients between 18 and 65 years of age. The odds ratio of having a headache and a comorbidity were calculated using conditional logistic regression, controlling for gender and age over a study population of 3,327,947 electronic health records in the absence of prescription medication data. Results We observed estimated odds ratio of 0.89 (95% CI: 0.83-0.95) of having a headache and a record of type 2 diabetes over the population, and 0.83 (95% CI: 2.02-2.57) and 0.89 (95% CI: 3.00-3.49) for male and female, respectively. Conclusions We find that patients with type 2 diabetes are less likely to present a recorded headache indication. Patients with hypertension are almost twice as likely of having a headache indication and patients with an anxiety disorder are almost three times as likely. Given the possibility of confounding indications and prescribed medications, additional studies are recommended

Clinical prompt learning with frozen language models

When the first transformer-based language models were published in the late 2010s, pretraining with general text and then fine-tuning the model on a task-specific dataset often achieved the state-of-the-art performance. However, more recent work suggests that for some tasks, directly prompting the pretrained model matches or surpasses fine-tuning in performance with few or no model parameter updates required. The use of prompts with language models for natural language processing (NLP) tasks is known as prompt learning. We investigated the viability of prompt learning on clinically meaningful decision tasks and directly compared this with more traditional fine-tuning methods. Results show that prompt learning methods were able to match or surpass the performance of traditional fine-tuning with up to 1000 times fewer trainable parameters, less training time, less training data, and lower computation resource requirements. We argue that these characteristics make prompt learning a very desirable alternative to traditional fine-tuning for clinical tasks, where the computational resources of public health providers are limited, and where data can often not be made available or not be used for fine-tuning due to patient privacy concerns. The complementary code to reproduce the experiments presented in this work can be found at https://github.com/NtaylorOX/Public_Clinical_Prompt

Learning to Detect Bipolar Disorder and Borderline Personality Disorder with Language and Speech in Non-Clinical Interviews

Bipolar disorder (BD) and borderline personality disorder (BPD) are both chronic psychiatric disorders. However, their overlapping symptoms and common comorbidity make it challenging for the clinicians to distinguish the two conditions on the basis of a clinical interview. In this work, we first present a new multi-modal dataset containing interviews involving individuals with BD or BPD being interviewed about a non-clinical topic . We investigate the automatic detection of the two conditions, and demonstrate a good linear classifier that can be learnt using a down-selected set of features from the different aspects of the interviews and a novel approach of summarising these features. Finally, we find that different sets of features characterise BD and BPD, thus providing insights into the difference between the automatic screening of the two conditions

Validation of UK Biobank data for mental health outcomes : a pilot study using secondary care electronic health records

The study was funded by the MRC Pathfinder Grant (MC_PC_17215); the National Institute for Health Research’s (NIHR) Oxford Health Biomedical Research Centre (BRC-1215-20005) and the Virtual Brain Cloud from European Commission (grant no. H2020SC1-DTH-2018-1). This work was supported by the UK Clinical Record Interactive Search (UK-CRIS) system funded by the National Institute for Health Research (NIHR) and the Medical Research Council, with the University of Oxford, using data and systems of the NIHR Oxford Health Biomedical Research Centre (BRC-1215-20005).UK Biobank (UKB) is widely employed to investigate mental health disorders and related exposures; however, its applicability and relevance in a clinical setting and the assumptions required have not been sufficiently and systematically investigated. Here, we present the first validation study using secondary care mental health data with linkage to UKB from Oxford - Clinical Record Interactive Search (CRIS) focusing on comparison of demographic information, diagnostic outcome, medication record and cognitive test results, with missing data and the implied bias from both resources depicted. We applied a natural language processing model to extract information embedded in unstructured text from clinical notes and attachments. Using a contingency table we compared the demographic information recorded in UKB and CRIS. We calculated the positive predictive value (PPV, proportion of true positives cases detected) for mental health diagnosis and relevant medication. Amongst the cohort of 854 subjects, PPVs for any mental health diagnosis for dementia, depression, bipolar disorder and schizophrenia were 41.6%, and were 59.5%, 12.5%, 50.0% and 52.6%, respectively. Self-reported medication records in UKB had general PPV of 47.0%, with the prevalence of frequently prescribed medicines to each typical mental health disorder considerably different from the information provided by CRIS. UKB is highly multimodal, but with limited follow-up records, whereas CRIS offers a longitudinal high-resolution clinical picture with more than ten years of observations. The linkage of both datasets will reduce the self-report bias and synergistically augment diverse modalities into a unified resource to facilitate more robust research in mental health.Peer reviewe

High blood pressure and risk of dementia : a two-sample Mendelian randomization study in the UK biobank

This work was supported by Janssen Research and Development , LLC (of Johnson & Johnson).Background: Findings from randomized controlled trials have yielded conflicting results on the association between blood pressure (BP) and dementia traits. We tested the hypothesis that a causal relationship exists between systolic BP (SBP) and/or diastolic BP (DBP) and risk of Alzheimer's disease (AD). Methods: We performed a generalized summary Mendelian randomization (GSMR) analysis using summary statistics of a genome-wide association study meta-analysis of 299,024 individuals of SBP or DBP as exposure variables against three different outcomes: 1) AD diagnosis (International Genomics of Alzheimer's Project), 2) maternal family history of AD (UK Biobank), and 3) paternal family history of AD (UK Biobank). Finally, a combined meta-analysis of 368,440 individuals that included these three summary statistics was used as final outcome. Results: GSMR applied to the International Genomics of Alzheimer's Project dataset revealed a significant effect of high SBP lowering the risk of AD (βGSMR = −0.19, p =.04). GSMR applied to the maternal family history of AD UK Biobank dataset (SBP [βGSMR = −0.12, p =.02], DBP [βGSMR = −0.10, p =.05]) and to the paternal family history of AD UK Biobank dataset (SBP [βGSMR = −0.16, p =.02], DBP [βGSMR = −0.24, p = 7.4 × 10−4]) showed the same effect. A subsequent combined meta-analysis confirmed the overall significant effect for the other SBP analyses (βGSMR = −0.14, p =.03). The DBP analysis in the combined meta-analysis also confirmed a DBP effect on AD (βGSMR = −0.14, p =.03). Conclusions: A causal effect exists between high BP and a reduced late-life risk of AD. The results were obtained through careful consideration of confounding factors and the application of complementary MR methods on independent cohorts.Peer reviewe

Comparative effect of metformin versus sulfonylureas with dementia and Parkinson's disease risk in US patients over 50 with type 2 diabetes mellitus

This work was supported by Janssen Pharmaceuticals. LJL is supported by the National Institute on Aging Intramural Research Program, USA. Additional funds were provided by Rosetrees Trust (M937) and John Black Charitable Fund (ID A2926). AJN-H has received funding from Janssen Pharmaceuticals, GlaxoSmithKline and Ono Pharma. QSL is an employee of Janssen Research & Development, Johnson & Johnson, and may hold equity in Johnson & Johnson.Introduction Type 2 diabetes is a risk factor for dementia and Parkinson's disease (PD). Drug treatments for diabetes, such as metformin, could be used as novel treatments for these neurological conditions. Using electronic health records from the USA (OPTUM EHR) we aimed to assess the association of metformin with all-cause dementia, dementia subtypes and PD compared with sulfonylureas. Research design and methods A new user comparator study design was conducted in patients ≥50 years old with diabetes who were new users of metformin or sulfonylureas between 2006 and 2018. Primary outcomes were all-cause dementia and PD. Secondary outcomes were Alzheimer's disease (AD), vascular dementia (VD) and mild cognitive impairment (MCI). Cox proportional hazards models with inverse probability of treatment weighting (IPTW) were used to estimate the HRs. Subanalyses included stratification by age, race, renal function, and glycemic control. Results We identified 96 140 and 16 451 new users of metformin and sulfonylureas, respectively. Mean age was 66.4±8.2 years (48% male, 83% Caucasian). Over the 5-year follow-up, 3207 patients developed all-cause dementia (2256 (2.3%) metformin, 951 (5.8%) sulfonylurea users) and 760 patients developed PD (625 (0.7%) metformin, 135 (0.8%) sulfonylurea users). After IPTW, HRs for all-cause dementia and PD were 0.80 (95% CI 0.73 to 0.88) and 1.00 (95% CI 0.79 to 1.28). HRs for AD, VD and MCI were 0.81 (0.70-0.94), 0.79 (0.63-1.00) and 0.91 (0.79-1.04). Stronger associations were observed in patients who were younger (<75 years old), Caucasian, and with moderate renal function. Conclusions Metformin users compared with sulfonylurea users were associated with a lower risk of all-cause dementia, AD and VD but not with PD or MCI. Age and renal function modified risk reduction. Our findings support the hypothesis that metformin provides more neuroprotection for dementia than sulfonylureas but not for PD, but further work is required to assess causality.Peer reviewe