AutoNet-generated deep layer-wise convex networks for ECG classification

The design of neural networks typically involves trial-and-error, a time-consuming process for obtaining an optimal architecture, even for experienced researchers. Additionally, it is widely accepted that loss functions of deep neural networks are generally non-convex with respect to the parameters to be optimised. We propose the Layer-wise Convex Theorem to ensure that the loss is convex with respect to the parameters of a given layer, achieved by constraining each layer to be an overdetermined system of non-linear equations. Based on this theorem, we developed an end-to-end algorithm (the AutoNet) to automatically generate layer-wise convex networks (LCNs) for any given training set. We then demonstrate the performance of the AutoNet-generated LCNs (AutoNet-LCNs) compared to state-of-the-art models on three electrocardiogram (ECG) classification benchmark datasets, with further validation on two non-ECG benchmark datasets for more general tasks. The AutoNet-LCN was able to find networks customised for each dataset without manual fine-tuning under 2 GPU-hours, and the resulting networks outperformed the state-of-the-art models with fewer than 5% parameters on all the above five benchmark datasets. The efficiency and robustness of the AutoNet-LCN markedly reduce model discovery costs and enable efficient training of deep learning models in resource-constrained settings

Modelling physiological deterioration in post-operative patient vital-sign data

Patients who undergo upper-gastrointestinal surgery have a high incidence of post-operative complications, often requiring admission to the intensive care unit several days after surgery. A dataset comprising observational vital-sign data from 171 post-operative patients taking part in a two-phase clinical trial at the Oxford Cancer Centre, was used to explore the trajectory of patients’ vital-sign changes during their stay in the post-operative ward using both univariate and multivariate analyses. A model of normality based vital-sign data from patients who had a “normal” recovery was constructed using a kernel density estimate, and tested with “abnormal” data from patients who deteriorated sufficiently to be re-admitted to the intensive care unit. The vital-sign distributions from “normal” patients were found to vary over time from admission to the post-operative ward to their discharge home, but no significant changes in their distributions were observed from halfway through their stay on the ward to the time of discharge. The model of normality identified patient deterioration when tested with unseen “abnormal” data, suggesting that such techniques may be used to provide early warning of adverse physiological events

Decoding 2.3 million ECGs: interpretable deep learning for advancing cardiovascular diagnosis and mortality risk stratification

Electrocardiogram (ECG) is widely considered the primary test for evaluating cardiovascular diseases. However, the use of artificial intelligence to advance these medical practices and learn new clinical insights from ECGs remains largely unexplored. Utilising a dataset of 2,322,513 ECGs collected from 1,558,772 patients with 7 years of follow-up, we developed a deep learning model with state-of-the-art granularity for the interpretable diagnosis of cardiac abnormalities, gender identification, and hyper- tension screening solely from ECGs, which are then used to stratify the risk of mortality. The model achieved the area under the receiver operating characteristic curve (AUC) scores of 0.998 (95% confidence interval (CI), 0.995-0.999), 0.964 (0.963-0.965), and 0.839 (0.837-0.841) for the three diagnostic tasks separately. Using ECG-predicted results, we find high risks of mortality for subjects with sinus tachycardia (adjusted hazard ratio (HR) of 2.24, 1.96-2.57), and atrial fibrillation (adjusted HR of 2.22, 1.99-2.48). We further use salient morphologies produced by the deep learning model to identify key ECG leads that achieved similar performance for the three diagnoses, and we find that the V1 ECG lead is important for hypertension screening and mortality risk stratification of hypertensive cohorts, with an AUC of 0.816 (0.814-0.818) and a univariate HR of 1.70 (1.61-1.79) for the two tasks separately. Using ECGs alone, our developed model showed cardiologist-level accuracy in interpretable cardiac diagnosis, and the advancement in mortality risk stratification; In addition, the potential to facilitate clinical knowledge discovery for gender and hypertension detection which are not readily available

Lightweight transformers for clinical natural language processing

Specialised pre-trained language models are becoming more frequent in Natural language Processing (NLP) since they can potentially outperform models trained on generic texts. BioBERT (Sanh et al., Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 1910.01108, 2019) and BioClinicalBERT (Alsentzer et al., Publicly available clinical bert embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 72–78, 2019) are two examples of such models that have shown promise in medical NLP tasks. Many of these models are overparametrised and resource-intensive, but thanks to techniques like knowledge distillation, it is possible to create smaller versions that perform almost as well as their larger counterparts. In this work, we specifically focus on development of compact language models for processing clinical texts (i.e. progress notes, discharge summaries, etc). We developed a number of efficient lightweight clinical transformers using knowledge distillation and continual learning, with the number of parameters ranging from million to million. These models performed comparably to larger models such as BioBERT and ClinicalBioBERT and significantly outperformed other compact models trained on general or biomedical data. Our extensive evaluation was done across several standard datasets and covered a wide range of clinical text-mining tasks, including natural language inference, relation extraction, named entity recognition and sequence classification. To our knowledge, this is the first comprehensive study specifically focused on creating efficient and compact transformers for clinical NLP tasks. The models and code used in this study can be found on our Huggingface profile at https://huggingface.co/nlpie and Github page at https://github.com/nlpie-research/Lightweight-Clinical-Transformers, respectively, promoting reproducibility of our results

Sample selection bias in machine learning for healthcare

While machine learning algorithms hold promise for personalised medicine, their clinical adoption remains limited. One critical factor contributing to this restraint is sample selection bias (SSB) which refers to the study population being less representative of the target population, leading to biased and potentially harmful decisions. Despite being well-known in the literature, SSB remains scarcely studied in machine learning for healthcare. Moreover, the existing techniques try to correct the bias by balancing distributions between the study and the target populations, which may result in a loss of predictive performance. To address these problems, our study illustrates the potential risks associated with SSB by examining SSB's impact on the performance of machine learning algorithms. Most importantly, we propose a new research direction for addressing SSB, based on the target population identification rather than the bias correction. Specifically, we propose two independent networks (T-Net) and a multitasking network (MT-Net) for addressing SSB, where one network/task identifies the target subpopulation which is representative of the study population and the second makes predictions for the identified subpopulation. Our empirical results with synthetic and semi-synthetic datasets highlight that SSB can lead to a large drop in the performance of an algorithm for the target population as compared with the study population, as well as a substantial difference in the performance for the target subpopulations that are representative of the selected and the non-selected patients from the study population. Furthermore, our proposed techniques demonstrate robustness across various settings, including different dataset sizes, event rates, and selection rates, outperforming the existing bias correction techniques

A medical multimodal large language model for future pandemics

Deep neural networks have been integrated into the whole clinical decision procedure which can improve the efficiency of diagnosis and alleviate the heavy workload of physicians. Since most neural networks are supervised, their performance heavily depends on the volume and quality of available labels. However, few such labels exist for rare diseases (e.g., new pandemics). Here we report a medical multimodal large language model (Med-MLLM) for radiograph representation learning, which can learn broad medical knowledge (e.g., image understanding, text semantics, and clinical phenotypes) from unlabelled data. As a result, when encountering a rare disease, our Med-MLLM can be rapidly deployed and easily adapted to them with limited labels. Furthermore, our model supports medical data across visual modality (e.g., chest X-ray and CT) and textual modality (e.g., medical report and free-text clinical note); therefore, it can be used for clinical tasks that involve both visual and textual data. We demonstrate the effectiveness of our Med-MLLM by showing how it would perform using the COVID-19 pandemic “in replay”. In the retrospective setting, we test the model on the early COVID-19 datasets; and in the prospective setting, we test the model on the new variant COVID-19-Omicron. The experiments are conducted on 1) three kinds of input data; 2) three kinds of downstream tasks, including disease reporting, diagnosis, and prognosis; 3) five COVID-19 datasets; and 4) three different languages, including English, Chinese, and Spanish. All experiments show that our model can make accurate and robust COVID-19 decision-support with little labelled data

Intelligent electrocardiogram acquisition via ubiquitous photoplethysmography monitoring

Recent advances in machine learning, particularly deep neural network architectures, have shown substantial promise in classifying and predicting cardiac abnormalities from electrocardiogram (ECG) data. Such data are rich in information content, typically in morphology and timing, due to the close correlation between cardiac function and the ECG. However, the ECG is usually not measured ubiquitously in a passive manner from consumer devices, and generally requires ‘active’ sampling whereby the user prompts a device to take an ECG measurement. Conversely, photoplethysmography (PPG) data are typically measured passively by consumer devices, and therefore available for long-period monitoring and suitable in duration for identifying transient cardiac events. However, classifying or predicting cardiac abnormalities from the PPG is very difficult, because it is a peripherally-measured signal. Hence, the use of the PPG for predictive inference is often limited to deriving physiological parameters (heart rate, breathing rate, etc.) or for obvious abnormalities in cardiac timing, such as atrial fibrillation/flutter (“palpitations”). This work aims to combine the best of both worlds: using continuously-monitored, near-ubiquitous PPG to identify periods of sufficient abnormality in the PPG such that prompting the user to take an ECG would be informative of cardiac risk. We propose a dual-convolutional-attention network (DCA-Net) to achieve this ECG-based PPG classification. With DCA-Net, we prove the plausibility of this concept on MIMIC Waveform Database with high performance level (AUROC > 0.9 and AUPRC > 0.7) and receive satisfactory result when testing the model on an independent dataset (AUROC > 0.7 and AUPRC > 0.6) which it is not perfectly-matched to the MIMIC dataset

Polygenic risk of prediabetes, undiagnosed diabetes, and incident type 2 diabetes stratified by diabetes risk factors

Context: Early diagnosis of type 2 diabetes is crucial to reduce severe comorbidities and complications. Current screening recommendations for type 2 diabetes include traditional risk factors, primarily body mass index (BMI) and family history, however genetics also plays a key role in type 2 diabetes risk. It is important to understand whether genetic predisposition to type 2 diabetes modifies the effect of these traditional factors on type 2 diabetes risk. Objective: This work aimed to investigate whether genetic risk of type 2 diabetes modifies associations between BMI and first-degree family history of diabetes with 1) prevalent prediabetes or undiagnosed diabetes; and 2) incident confirmed type 2 diabetes. Methods: We included 431 658 individuals aged 40 to 69 years at baseline of multiethnic ancestry from the UK Biobank. We used a multiethnic polygenic risk score for type 2 diabetes (PRST2D) developed by Genomics PLC. Prediabetes or undiagnosed diabetes was defined as baseline glycated hemoglobin greater than or equal to 42 mmol/mol (6.0%), and incident type 2 diabetes was derived from medical records. Results: At baseline, 43 472 participants had prediabetes or undiagnosed diabetes, and 17 259 developed type 2 diabetes over 15 years follow-up. Dose-response associations were observed for PRST2D with each outcome in each category of BMI or first-degree family history of diabetes. Those in the highest quintile of PRST2D with a normal BMI were at a similar risk as those in the middle quintile who were overweight. Participants who were in the highest quintile of PRST2D and did not have a first-degree family history of diabetes were at a similar risk as those with a family history who were in the middle category of PRST2D. Conclusion: Genetic risk of type 2 diabetes remains strongly associated with risk of prediabetes, undiagnosed diabetes, and future type 2 diabetes within categories of nongenetic risk factors. This could have important implications for identifying individuals at risk of type 2 diabetes for prevention and early diagnosis programs

Assessing the importance of primary care diagnoses in the UK Biobank.

The UK Biobank has made general practitioner (GP) data (censoring date 2016-2017) available for approximately 45% of the cohort, whilst hospital inpatient and death registry (referred to as "HES/Death") data are available cohort-wide through 2018-2022 depending on whether the data comes from England, Wales or Scotland. We assessed the importance of case ascertainment via different data sources in UKB for three diseases that are usually first diagnosed in primary care: Parkinson's disease (PD), type 2 diabetes (T2D), and all-cause dementia. Including GP data at least doubled the number of incident cases in the subset of the cohort with primary care data (e.g. from 619 to 1390 for dementia). Among the 786 dementia cases that were only captured in the GP data before the GP censoring date, only 421 (54%) were subsequently recorded in HES. Therefore, estimates of the absolute incidence or risk-stratified incidence are misleadingly low when based only on the HES/Death data. For incident cases present in both HES/Death and GP data during the full follow-up period (i.e. until the HES censoring date), the median time difference between an incident diagnosis of dementia being recorded in GP and HES/Death was 2.25 years (i.e. recorded 2.25 years earlier in the GP records). Similar lag periods were also observed for PD (median 2.31 years earlier) and T2D (median 2.82 years earlier). For participants with an incident GP diagnosis, only 65.6% of dementia cases, 69.0% of PD cases, and 58.5% of T2D cases had their diagnosis recorded in HES/Death within 7 years since GP diagnosis. The effect estimates (hazard ratios, HR) of established risk factors for the three health outcomes mostly remain in the same direction and with a similar strength of association when cases are ascertained either using HES only or further adding GP data. The confidence intervals of the HR became narrower when adding GP data, due to the increased statistical power from the additional cases. In conclusion, it is desirable to extend both the coverage and follow-up period of GP data to allow researchers to maximise case ascertainment of chronic health conditions in the UK

Kidney function, albuminuria, and their modification by genetic factors and risk of incident dementia in UK Biobank.

BACKGROUND: Associations between kidney function and dementia risk are inconclusive. Chronic kidney disease (CKD) severity is determined by levels of both estimated glomerular filtration rate (eGFR) and the urine albumin to creatinine ratio (ACR). However, whether there is a graded increase in dementia risk for worse eGFR in each ACR category is unclear. Also, whether genetic risk for dementia impacts the associations is unknown. The current study aims to investigate the associations between eGFR and albuminuria with dementia risk both individually and jointly, whether the associations vary by different follow-up periods, and whether genetic factors modified the associations. METHODS: In 202,702 participants aged ≥ 60 years from the UK Biobank, Cox proportional-hazards models were used to examine the associations between eGFR and urine albumin creatinine ratio (ACR) with risk of incident dementia. GFR was estimated based on serum creatinine, cystatin C, or both. The models were restricted to different follow-up periods (< 5 years, 5-10 years, and ≥ 10 years) to investigate potential reverse causation. RESULTS: Over 15 years of follow-up, 6,042 participants developed dementia. Decreased kidney function (eGFR < 60 ml/min/1.73m2) was associated with an increased risk of dementia (Hazard Ratio [HR] = 1.42, 95% Confidence Interval [CI] 1.28-1.58), compared to normal kidney function (≥ 90 ml/min/1.73m2). The strength of the association remained consistent when the models were restricted to different periods of follow-up. The HRs for incident dementia were 1.16 (95% CI 1.07-1.26) and 2.24 (95% CI 1.79-2.80) for moderate (3-30 mg/mmol) and severely increased ACR (≥ 30 mg/mmol) compared to normal ACR (< 3 mg/mmol). Dose-response associations were observed when combining eGFR and ACR, with those in the severest eGFR and ACR group having the greatest risk of dementia (HR = 4.70, 95% CI 2.34-9.43). APOE status significantly modified the association (p = 0.04), with stronger associations observed among participants with a lower genetic risk of dementia. There was no evidence of an interaction between kidney function and non-APOE polygenic risk of dementia with dementia risk (p = 0.42). CONCLUSIONS: Kidney dysfunction and albuminuria were individually and jointly associated with higher dementia risk. The associations were greater amongst participants with a lower genetic risk of dementia based on APOE, but not non-APOE polygenic risk