Artificial Intelligence-Based Methods for Fusion of Electronic Health Records and Imaging Data

Healthcare data are inherently multimodal, including electronic health records (EHR), medical images, and multi-omics data. Combining these multimodal data sources contributes to a better understanding of human health and provides optimal personalized healthcare. Advances in artificial intelligence (AI) technologies, particularly machine learning (ML), enable the fusion of these different data modalities to provide multimodal insights. To this end, in this scoping review, we focus on synthesizing and analyzing the literature that uses AI techniques to fuse multimodal medical data for different clinical applications. More specifically, we focus on studies that only fused EHR with medical imaging data to develop various AI methods for clinical applications. We present a comprehensive analysis of the various fusion strategies, the diseases and clinical outcomes for which multimodal fusion was used, the ML algorithms used to perform multimodal fusion for each clinical application, and the available multimodal medical datasets. We followed the PRISMA-ScR guidelines. We searched Embase, PubMed, Scopus, and Google Scholar to retrieve relevant studies. We extracted data from 34 studies that fulfilled the inclusion criteria. In our analysis, a typical workflow was observed: feeding raw data, fusing different data modalities by applying conventional machine learning (ML) or deep learning (DL) algorithms, and finally, evaluating the multimodal fusion through clinical outcome predictions. Specifically, early fusion was the most used technique in most applications for multimodal learning (22 out of 34 studies). We found that multimodality fusion models outperformed traditional single-modality models for the same task. Disease diagnosis and prediction were the most common clinical outcomes (reported in 20 and 10 studies, respectively) from a clinical outcome perspective.Comment: Accepted in Nature Scientific Reports. 20 page

A review on a deep learning perspective in brain cancer classification

AWorld Health Organization (WHO) Feb 2018 report has recently shown that mortality rate due to brain or central nervous system (CNS) cancer is the highest in the Asian continent. It is of critical importance that cancer be detected earlier so that many of these lives can be saved. Cancer grading is an important aspect for targeted therapy. As cancer diagnosis is highly invasive, time consuming and expensive, there is an immediate requirement to develop a non-invasive, cost-effective and efficient tools for brain cancer characterization and grade estimation. Brain scans using magnetic resonance imaging (MRI), computed tomography (CT), as well as other imaging modalities, are fast and safer methods for tumor detection. In this paper, we tried to summarize the pathophysiology of brain cancer, imaging modalities of brain cancer and automatic computer assisted methods for brain cancer characterization in a machine and deep learning paradigm. Another objective of this paper is to find the current issues in existing engineering methods and also project a future paradigm. Further, we have highlighted the relationship between brain cancer and other brain disorders like stroke, Alzheimer’s, Parkinson’s, andWilson’s disease, leukoriaosis, and other neurological disorders in the context of machine learning and the deep learning paradigm

A deep phenotyping approach to understand major depressive disorder and responses to antidepressant pharmacotherapy

Major depressive disorder (MDD) is a debilitating psychiatric disorder characterised by a complex underlying biology and poor response to pharmacological antidepressant strategies. Given the heterogeneity of MDD and the diverse range of available treatment options, there is an increasing desire to develop and implement precision medicine approaches to tailor existing treatment strategies to the biological system of the individual. In this thesis, high-resolution omics data (connectomics [fMRI], metabolomics [1H NMR] and immunomics [inflammatory cytokines]) collected from the Canadian Biomarker Integration Network in Depression (CAN-BIND) study has been integrated to facilitate the deep phenotyping of MDD. In addition, this approach has been used to predict the treatment response to two common antidepressant drugs, monotherapy with the selective serotonin reuptake inhibitor (SSRI) escitalopram (10-20 mg) or combination therapy with escitalopram and the dopaminergic antipsychotic aripiprazole (2-10 mg). This approach identified a multi-modal panel of sex-specific biomarkers of MDD and treatment response, highlighting a strong immunometabolic component in depressed males, but not females. Unsupervised clustering methods indicated the superiority of biological (neuroimaging) over symptom-based (clinical questionnaires) data for the stratification of patients into MDD subtypes with differential response to treatment. More importantly, a set of multi-modal, sex-specific biomarkers were identified that predicted treatment response with escitalopram monotherapy (84.7% accuracy) or aripiprazole augmentation (88.5% accuracy). In addition to highlighting potential new aspects of the biology of MDD (e.g. relevance of lipoprotein size and density for their relation to depression), this work is one of the first attempts to apply systems biology approaches to high-resolution biological data from a large clinical trial to predict later treatment outcome. With the validation of the findings presented in this thesis in independent cohorts, and with further development of omics technologies, leading to cheaper and high-throughput screening of the patient population, pre-dose biomarkers have the potential to achieve personalised treatment. Each year, escitalopram and aripiprazole are prescribed to an estimated 26 million and 7 million individuals respectively, and over one third of them do not respond. Thus, being able to predict response to antidepressant medication from baseline biomarkers has enormous clinical and socioeconomic benefits.Open Acces

Optical Diagnostics in Human Diseases

Optical technologies provide unique opportunities for the diagnosis of various pathological disorders. The range of biophotonics applications in clinical practice is considerably wide given that the optical properties of biological tissues are subject to significant changes during disease progression. Due to the small size of studied objects (from μm to mm) and despite some minimum restrictions (low-intensity light is used), these technologies have great diagnostic potential both as an additional tool and in cases of separate use, for example, to assess conditions affecting microcirculatory bed and tissue viability. This Special Issue presents topical articles by researchers engaged in the development of new methods and devices for optical non-invasive diagnostics in various fields of medicine. Several studies in this Special Issue demonstrate new information relevant to surgical procedures, especially in oncology and gynecology. Two articles are dedicated to the topical problem of breast cancer early detection, including during surgery. One of the articles is devoted to urology, namely to the problem of chronic or recurrent episodic urethral pain. Several works describe the studies in otolaryngology and dentistry. One of the studies is devoted to diagnosing liver diseases. A number of articles contribute to the studying of the alterations caused by diabetes mellitus and cardiovascular diseases. The results of all the presented articles reflect novel innovative research and emerging ideas in optical non-invasive diagnostics aimed at their wider translation into clinical practice

Enabling cardiovascular multimodal, high dimensional, integrative analytics

While traditionally the understanding of cardiovascular morbidity relied on the acquisition and interpretation of health data, the advances in health technologies has enabled us to collect far larger amount of health data. This thesis explores the application of advanced analytics that utilise powerful mechanisms for integrating health data across different modalities and dimensions into a single and holistic environment to better understand different diseases, with a focus on cardiovascular conditions. Different statistical methodologies are applied across a number of case studies supported by a novel methodology to integrate and simplify data collection. The work culminates in the different dataset modalities explaining different effects on morbidity: blood biomarkers, electrocardiogram recordings, RNA-Seq measurements, and different population effects piece together the understanding of a person morbidity. More specifically, explainable artificial intelligence methods were employed on structured datasets from patients with atrial fibrillation to improve the screening for the disease. Omics datasets, including RNA-sequencing and genotype datasets, were examined and new biomarkers were discovered allowing a better understanding of atrial fibrillation. Electrocardiogram signal data were used to assess the early risk prediction of heart failure, enabling clinicians to use this novel approach to estimate future incidences. Population-level data were applied to the identification of associations and temporal trajectory of diseases to better understand disease dependencies in different clinical cohorts

Dynamic And Quantitative Radiomics Analysis In Interventional Radiology

Interventional Radiology (IR) is a subspecialty of radiology that performs invasive procedures driven by diagnostic imaging for predictive and therapeutic purpose. The development of artificial intelligence (AI) has revolutionized the industry of IR. Researchers have created sophisticated models backed by machine learning algorithms and optimization methodologies for image registration, cellular structure detection and computer-aided disease diagnosis and prognosis predictions. However, due to the incapacity of the human eye to detect tiny structural characteristics and inter-radiologist heterogeneity, conventional experience-based IR visual evaluations may have drawbacks. Radiomics, a technique that utilizes machine learning, offers a practical and quantifiable solution to this issue. This technology has been used to evaluate the heterogeneity of malignancies that are difficult to detect by the human eye by creating an automated pipeline for the extraction and analysis of high throughput computational imaging characteristics from radiological medical pictures. However, it is a demanding task to directly put radiomics into applications in IR because of the heterogeneity and complexity of medical imaging data. Furthermore, recent radiomics studies are based on static images, while many clinical applications (such as detecting the occurrence and development of tumors and assessing patient response to chemotherapy and immunotherapy) is a dynamic process. Merely incorporating static features cannot comprehensively reflect the metabolic characteristics and dynamic processes of tumors or soft tissues. To address these issues, we proposed a robust feature selection framework to manage the high-dimensional small-size data. Apart from that, we explore and propose a descriptor in the view of computer vision and physiology by integrating static radiomics features with time-varying information in tumor dynamics. The major contributions to this study include: Firstly, we construct a result-driven feature selection framework, which could efficiently reduce the dimension of the original feature set. The framework integrates different feature selection techniques to ensure the distinctiveness, uniqueness, and generalization ability of the output feature set. In the task of classification hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC) in primary liver cancer, only three radiomics features (chosen from more than 1, 800 features of the proposed framework) can obtain an AUC of 0.83 in the independent dataset. Besides, we also analyze features’ pattern and contributions to the results, enhancing clinical interpretability of radiomics biomarkers. Secondly, we explore and build a pulmonary perfusion descriptor based on 18F-FDG whole-body dynamic PET images. Our major novelties include: 1) propose a physiology-and-computer-vision-interpretable descriptor construction framework by the decomposition of spatiotemporal information into three dimensions: shades of grey levels, textures, and dynamics. 2) The spatio-temporal comparison of pulmonary descriptor intra and inter patients is feasible, making it possible to be an auxiliary diagnostic tool in pulmonary function assessment. 3) Compared with traditional PET metabolic biomarker analysis, the proposed descriptor incorporates image’s temporal information, which enables a better understanding of the time-various mechanisms and detection of visual perfusion abnormalities among different patients. 4) The proposed descriptor eliminates the impact of vascular branching structure and gravity effect by utilizing time warping algorithms. Our experimental results showed that our proposed framework and descriptor are promising tools to medical imaging analysis

Aetiology of an Unknown Liver Disease in Northern Ethiopia

Cases of a novel form of liver disease, localised to a cluster of villages in the Tigray region of Northern Ethiopia, were first reported in 2001. Up to January 2010, 591 cases were recorded, including 228 deaths. Symptoms include epigastric and abdominal swelling. Children are particularly susceptible, with some children having died within three months of the onset of symptoms, whereas some adults report living with the disease for many years. The pattern of spread suggested an environmental toxin was the causative agent, with investigations implicating the plant toxins, pyrrolizidine alkaloids. The ultimate aim of the research presented in this thesis was to determine the aetiology of the disease and a multi-disciplinary approach, involving clinical research, ethnography, biochemical and metabonomic analysis and toxicology, has been employed to achieve this. The disease was first characterised clinically: The disease presents with epigastric pain, initial bloody diarrhoea and progressive ascites with either hepatomegaly or splenomegaly or both. Histology of liver biopsy specimens revealed centrilobular necrosis in acute cases and bile ductular proliferation and fibrosis in chronic cases. Biochemical analysis identified γ-glutamyl transferase as a sensitive indicator of the disease. Ethnographic investigation of the lifestyle of the residents of the affected villages revealed a monotonous diet, the presence of a similar disease among livestock and suggested millet and animal products as potential sources of exposure to the aetiological agent(s). The pesticide DDT was identified as a further potential risk factor for the disease. A global metabonomic analysis of patient urine samples using 1H NMR spectroscopy was conducted and several metabolites associated with the disease were identified. Targeted LC-MS assays were then developed to detect pyrrolizidine alkaloid exposure among patients, either through detection of the parent alkaloid, acetyllycopsamine (AL), in urine samples or extraction and detection of pyrrolic metabolite adducts from whole blood. Relative AL urinary concentrations were found to be significantly higher among cases. AL was shown to be hepatotoxic in the mouse model, and at high doses induced centrilobular hepatic necrosis and metabolic changes similar to that observed in the human disease. Finally, DDT was detected at high levels in patient urine and serum samples. DDT was shown to substantially enhance the toxic effects of AL in the mouse model, through induction of the CYP3A11 enzyme, and in combination with AL induced liver pathology closely resembling the human disease. In conclusion, the available evidence suggests that the disease arises from co-exposure to DDT and pyrrolizidine alkaloids, including AL