Adapting State-of-the-Art Deep Language Models to Clinical Information Extraction Systems: Potentials, Challenges, and Solutions

Background: Deep learning (DL) has been widely used to solve problems with success in speech recognition, visual object recognition, and object detection for drug discovery and genomics. Natural language processing has achieved noticeable progress in artificial intelligence. This gives an opportunity to improve on the accuracy and human-computer interaction of clinical informatics. However, due to difference of vocabularies and context between a clinical environment and generic English, transplanting language models directly from up-to-date methods to real-world health care settings is not always satisfactory. Moreover, the legal restriction on using privacy-sensitive patient records hinders the progress in applying machine learning (ML) to clinical language processing.Objective: The aim of this study was to investigate 2 ways to adapt state-of-the-art language models to extracting patient information from free-form clinical narratives to populate a handover form at a nursing shift change automatically for proofing and revising by hand: first, by using domain-specific word representations and second, by using transfer learning models to adapt knowledge from general to clinical English. We have described the practical problem, composed it as an ML task known as information extraction, proposed methods for solving the task, and evaluated their performance.Methods: First, word representations trained from different domains served as the input of a DL system for information extraction. Second, the transfer learning model was applied as a way to adapt the knowledge learned from general text sources to the task domain. The goal was to gain improvements in the extraction performance, especially for the classes that were topically related but did not have a sufficient amount of model solutions available for ML directly from the target domain. A total of 3 independent datasets were generated for this task, and they were used as the training (101 patient reports), validation (100 patient reports), and test (100 patient reports) sets in our experiments.Results: Our system is now the state-of-the-art in this task. Domain-specific word representations improved the macroaveraged F1 by 3.4%. Transferring the knowledge from general English corpora to the task-specific domain contributed a further 7.1% improvement. The best performance in populating the handover form with 37 headings was the macroaveraged F1 of 41.6% and F1 of 81.1% for filtering out irrelevant information. Performance differences between this system and its baseline were statistically significant (PConclusions: To our knowledge, our study is the first attempt to transfer models from general deep models to specific tasks in health care and gain a significant improvement. As transfer learning shows its advantage over other methods, especially on classes with a limited amount of training data, less experts' time is needed to annotate data for ML, which may enable good results even in resource-poor domains.</p

Enhancing traditional Chinese medicine diagnostics: Integrating ontological knowledge for multi-label symptom entity classification

In traditional Chinese medicine (TCM), artificial intelligence (AI)-assisted syndrome differentiation and disease diagnoses primarily confront the challenges of accurate symptom identification and classification. This study introduces a multi-label entity extraction model grounded in TCM symptom ontology, specifically designed to address the limitations of existing entity recognition models characterized by limited label spaces and an insufficient integration of domain knowledge. This model synergizes a knowledge graph with the TCM symptom ontology framework to facilitate a standardized symptom classification system and enrich it with domain-specific knowledge. It innovatively merges the conventional bidirectional encoder representations from transformers (BERT) + bidirectional long short-term memory (Bi-LSTM) + conditional random fields (CRF) entity recognition methodology with a multi-label classification strategy, thereby adeptly navigating the intricate label interdependencies in the textual data. Introducing a multi-associative feature fusion module is a significant advancement, thereby enabling the extraction of pivotal entity features while discerning the interrelations among diverse categorical labels. The experimental outcomes affirm the model's superior performance in multi-label symptom extraction and substantially elevates the efficiency and accuracy. This advancement robustly underpins research in TCM syndrome differentiation and disease diagnoses

Scalable Feature Selection Applications for Genome-Wide Association Studies of Complex Diseases

Personalized medicine will revolutionize our capabilities to combat disease. Working toward this goal, a fundamental task is the deciphering of geneticvariants that are predictive of complex diseases. Modern studies, in the formof genome-wide association studies (GWAS) have afforded researchers with the opportunity to reveal new genotype-phenotype relationships through the extensive scanning of genetic variants. These studies typically contain over half a million genetic features for thousands of individuals. Examining this with methods other than univariate statistics is a challenging task requiring advanced algorithms that are scalable to the genome-wide level. In the future, next-generation sequencing studies (NGS) will contain an even larger number of common and rare variants. Machine learning-based feature selection algorithms have been shown to have the ability to effectively create predictive models for various genotype-phenotype relationships. This work explores the problem of selecting genetic variant subsets that are the most predictive of complex disease phenotypes through various feature selection methodologies, including filter, wrapper and embedded algorithms. The examined machine learning algorithms were demonstrated to not only be effective at predicting the disease phenotypes, but also doing so efficiently through the use of computational shortcuts. While much of the work was able to be run on high-end desktops, some work was further extended so that it could be implemented on parallel computers helping to assure that they will also scale to the NGS data sets. Further, these studies analyzed the relationships between various feature selection methods and demonstrated the need for careful testing when selecting an algorithm. It was shown that there is no universally optimal algorithm for variant selection in GWAS, but rather methodologies need to be selected based on the desired outcome, such as the number of features to be included in the prediction model. It was also demonstrated that without proper model validation, for example using nested cross-validation, the models can result in overly-optimistic prediction accuracies and decreased generalization ability. It is through the implementation and application of machine learning methods that one can extract predictive genotype–phenotype relationships and biological insights from genetic data sets.Siirretty Doriast

Health systems data interoperability and implementation

Objective The objective of this study was to use machine learning and health standards to address the problem of clinical data interoperability across healthcare institutions. Addressing this problem has the potential to make clinical data comparable, searchable and exchangeable between healthcare providers. Data sources Structured and unstructured data has been used to conduct the experiments in this study. The data was collected from two disparate data sources namely MIMIC-III and NHanes. The MIMIC-III database stored data from two electronic health record systems which are CareVue and MetaVision. The data stored in these systems was not recorded with the same standards; therefore, it was not comparable because some values were conflicting, while one system would store an abbreviation of a clinical concept, the other would store the full concept name and some of the attributes contained missing information. These few issues that have been identified make this form of data a good candidate for this study. From the identified data sources, laboratory, physical examination, vital signs, and behavioural data were used for this study. Methods This research employed a CRISP-DM framework as a guideline for all the stages of data mining. Two sets of classification experiments were conducted, one for the classification of structured data, and the other for unstructured data. For the first experiment, Edit distance, TFIDF and JaroWinkler were used to calculate the similarity weights between two datasets, one coded with the LOINC terminology standard and another not coded. Similar sets of data were classified as matches while dissimilar sets were classified as non-matching. Then soundex indexing method was used to reduce the number of potential comparisons. Thereafter, three classification algorithms were trained and tested, and the performance of each was evaluated through the ROC curve. Alternatively the second experiment was aimed at extracting patient’s smoking status information from a clinical corpus. A sequence-oriented classification algorithm called CRF was used for learning related concepts from the given clinical corpus. Hence, word embedding, random indexing, and word shape features were used for understanding the meaning in the corpus. Results Having optimized all the model’s parameters through the v-fold cross validation on a sampled training set of structured data ( ), out of 24 features, only ( 8) were selected for a classification task. RapidMiner was used to train and test all the classification algorithms. On the final run of classification process, the last contenders were SVM and the decision tree classifier. SVM yielded an accuracy of 92.5% when the and parameters were set to and . These results were obtained after more relevant features were identified, having observed that the classifiers were biased on the initial data. On the other side, unstructured data was annotated via the UIMA Ruta scripting language, then trained through the CRFSuite which comes with the CLAMP toolkit. The CRF classifier obtained an F-measure of 94.8% for “nonsmoker” class, 83.0% for “currentsmoker”, and 65.7% for “pastsmoker”. It was observed that as more relevant data was added, the performance of the classifier improved. The results show that there is a need for the use of FHIR resources for exchanging clinical data between healthcare institutions. FHIR is free, it uses: profiles to extend coding standards; RESTFul API to exchange messages; and JSON, XML and turtle for representing messages. Data could be stored as JSON format on a NoSQL database such as CouchDB, which makes it available for further post extraction exploration. Conclusion This study has provided a method for learning a clinical coding standard by a computer algorithm, then applying that learned standard to unstandardized data so that unstandardized data could be easily exchangeable, comparable and searchable and ultimately achieve data interoperability. Even though this study was applied on a limited scale, in future, the study would explore the standardization of patient’s long-lived data from multiple sources using the SHARPn open-sourced tools and data scaling platformsInformation ScienceM. Sc. (Computing

Detecting deception using interview assistive technology

This thesis presents the design, implementation and evaluation of an application designed to support interviewers in detecting deception. This application is evaluated in a job interviewing study using novice interviewers, which shows it to be a highly effective method of de- ception detection, correctly identifying 68.8% of deceivers overall, an increase of 107% and 97% over two baselines without application sup- port, while reducing false positives. We follow work that suggests effective test questioning is the key to detecting deception in interviewing. The rationale behind this ap- proach is that a good breadth and depth of questioning increases cog- nitive load in deceivers, which greatly increases the chance of eliciting detectable behaviour change indicative of deception. Our application is based on Controlled Cognitive Engagement (CCE). Our motivation for supporting interviewers is the difficulty of the interviewing task. Interviewers must simultaneously manage the in- terview process, observe and control the interviewee while generat- ing probing test questions for subjects they potentially know little or nothing about. The application developed in this thesis, called Intek, for Interview Technology, is designed to assist interviewers in generating test ques- tions and providing checkable answers, while also providing a basis to keep track of interview progress. The information supplied by In- tek aims to provide unexpected tests of expected knowledge relevant to the specific personal information provided in a CV or elicited dur- ing an interview. Intek uses multiple information extraction pipelines, from multiple data sources, driven by state-of-the-art Natural Language Processing (NLP) techniques, such as BART for abstractive summarisation, spaCy for fast and accurate Named Entity Recognition (NER) and BERT fine-tuned on the CoNLL-2003 NER dataset for slower but best accur- acy NER. These pipelines integrate into a single simple user interface which may be used by an interviewer for real-time questioning. While most of the underlying NLP technology we used was "off the shelf", we discovered an opportunity to investigate a novel approach to web named entity recognition using HTML tags. Our Text+Tags ap- proach resulted in F1 improvements of between 0.9% and 13.2% over a collection of five datasets and two NER models. Our approach is suitable for extracting named entities from websites containing vary- ing amounts of HTML structure, as well as applicable to other NLP tasks

Harnessing the Power of Machine Learning in Dementia Informatics Research: Issues, Opportunities and Challenges

Dementia is a chronic and degenerative condition affecting millions globally. The care of patients with dementia presents an ever continuing challenge to healthcare systems in the 21st century. Medical and health sciences have generated unprecedented volumes of data related to health and wellbeing for patients with dementia due to advances in information technology, such as genetics, neuroimaging, cognitive assessment, free texts, routine electronic health records etc. Making the best use of these diverse and strategic resources will lead to high quality care of patients with dementia. As such, machine learning becomes a crucial factor in achieving this objective. The aim of this paper is to provide a state-of-the-art review of machine learning methods applied to health informatics for dementia care. We collate and review the existing scientific methodologies and identify the relevant issues and challenges when faced with big health data. Machine learning has demonstrated promising applications to neuroimaging data analysis for dementia care, while relatively less efforts have been made to make use of integrated heterogeneous data via advanced machine learning approaches. We further indicate the future potentials and research directions of applying advanced machine learning, such as deep learning, to dementia informatics

Monitoring outpatients in palliative care through wearable devices

Patients in palliative care suffer from a life-threatening disease. Holistic treatment includes control of symptoms (e. g., pain, nausea, sleeplessness) as well as psychosocial and spiritual help which is also extended to the relatives of a patient. For advanced cancer patients in palliative care, a crucial phase is the transition from palliative care in the hospital to the home setting, where care around the clock is not guaranteed any more, leads to an increased number of unplanned hospital re-admissions and emergency visits. Physicians aim to fill this care gap by monitoring physical and social activities as well as vital signs. Daily monitoring data, provided to caregivers, could enable caregivers to timely intervene when symptoms of a patient deteriorate. Besides patients in palliative care, also cancer survivors suffering from cancer-related fatigue could benefit from activity monitoring. Up to now, the remedies and effective treatments for cancer-related fatigue are limited. Research still has to unveil the underlying mechanisms that lead to a state of chronic exhaustedness. Measures that help healthy people like regenerative sleep show no or little effect in fatigued patients. Besides psycho-stimulants that come with the risk of addiction, cognitive behavioural therapy and moderate physical exercise have been shown to be effective. However, research still has to investigate timing, frequency and intensity of physical activity and researchers need a better understanding how the fatigue evolves during the day and in long-term. This thesis investigates the possibilities and limitations of activity monitoring using wearable devices such as smartphones and an armworn devices that is capable of measuring vital signs such as heart rate. Three studies involving cancer patients are conducted: - An interview study including 12 cancer patients enabled a patient-centric design for an Android activity monitoring app for smartphones. - Only using the smartphone as monitoring device, a study with 7 cancer survivors suffering from cancer-related fatigue was conducted as a pre-study in order to gain first experiences and to explore the possible knowledge gain about cancer-related fatigue through activity monitoring. - During a planned study period of 12 weeks per patient, 30 patients in ambulatory palliative care were wearing a smartphone and the arm-worn sensor as monitoring devices. The age range of the study participants was 39 to 85 years. In weekly interviews, patients were asked about their experiences with the devices and their quality of life. The aim of the study was to evaluate feasibility and acceptance of activity monitoring in this patient group. Furthermore, exploratory data analysis investigated the possibilities and limitations of unsupervised methods on this real-world data set. The two data sets, collected during the fatigue study and during the palliative care study, were pre-processed including cleaning steps, classification and clustering methods to add higher level information such as visited locations (anonymized). From these prepared data sets, features were extracted such as number of places visited per day. On the resulting datasets of features, statistical methods were applied to explore relations between sensor data, self-reports and, in case of the palliative care study, emergency visits to the hospital. For the latter analysis, patients who experienced an emergency room visit and those who did not were compared by means of hypothesis testing. For each feature, the underlying alternative hypothesis was that the change of a feature between the first week of study participation at home and the week before an emergency visit (or the last week of study participation for the patients without an emergency visit), differs in the two patient groups. The rate of change was defined by the ratio of the medians of the two weeks. Changes of three features, namely resting heart rate, resting heart rate variability and step speed were identified to have significant group differences: - The resting heart rate had an increasing trend in the group with emergency visits (median=1.01, interquartile range [0.96, 1.12]) and a decreasing trend in the group without an emergency visit (median=0.9, interquartile range [0.89, 0.99]) with a nominal significance of p=.021 and a medium effect size r=.46. - The resting heart rate variability had a decreasing trend in the group with emergency visits (mean=0.81, standard deviation=0.14) and an increasing trend in the group without an emergency visit (mean=1.17, standard deviation=0.46) with a nominal significance of p=.011 and a large effect size r=.53. - The step speed had an increasing trend in the group with emergency visits (median=1.1, interquartile range [1.08, 1.13]) and a decreasing trend in the group without an emergency visit (median=0.99, interquartile range [0.96, 1.04]) with a nominal significance of p=.003 and a large effect size r=.61. In contrast, hypothesis testing for features based on patients’ subjective self-reports for pain, distres and global quality of life did not reveil any significant differences. Hence, activity monitoring of vital signs and physical activity outperformed patients’ self-reports. However, a power analysis based on the three nominally significant results would recommend an independent study with 84 patients to confirm the results of this study. Furthermore, a set of recommendations for future research was concluded from the experiences gained through conducting these studies