Temporal self-attention network for medical concept embedding

© 2019 IEEE. In longitudinal electronic health records (EHRs), the event records of a patient are distributed over a long period of time and the temporal relations between the events reflect sufficient domain knowledge to benefit prediction tasks such as the rate of inpatient mortality. Medical concept embedding as a feature extraction method that transforms a set of medical concepts with a specific time stamp into a vector, which will be fed into a supervised learning algorithm. The quality of the embedding significantly determines the learning performance over the medical data. In this paper, we propose a medical concept embedding method based on applying a self-attention mechanism to represent each medical concept. We propose a novel attention mechanism which captures the contextual information and temporal relationships between medical concepts. A light-weight neural net, 'Temporal Self-Attention Network (TeSAN)', is then proposed to learn medical concept embedding based solely on the proposed attention mechanism. To test the effectiveness of our proposed methods, we have conducted clustering and prediction tasks on two public EHRs datasets comparing TeSAN against five state-of-the-art embedding methods. The experimental results demonstrate that the proposed TeSAN model is superior to all the compared methods. To the best of our knowledge, this work is the first to exploit temporal self-attentive relations between medical events

Gauge against the machine: improving representations within sociotechnical instruments to enrich context and identify biases

The proliferation of digital data across all areas of society has transformed our ability to hypothesize, study, and understand social systems.From this richness of data we have seen the development of innovative instruments to study---and make decisions with---the digital artifacts of the modern day. These developments build on advancements in computation, connectivity, analytical methodologies, and sociological theories. The sociotechnical instruments we have developed have been revolutionary to how we understand society and how we conduct business, but with these broad leaps comes ample room (and need) for more nuanced advancements. As with the development of any field, as the digital humanities evolve there is opportunity for targeted progress and the need for more tectonic shifts in practices. Iterative improvements include building more full-featured instruments that include a broader set of variables when analyzing and presenting results. More profound topics such as fairness, accountability, transparency, and ethics need increased attention as well---especially to create equitable, pro-social tools. Both in academia and in industry, there is room to improve how we curate, study, and operationalize data sets and the AI pipelines that sit atop them. Here we use natural language processing, machine learning, tools from data ethics, and other methods to explore how we can contextualize results and improve representations within instruments used to understand sociotechnical systems. In the first study we examine the dynamics of responses to posts by US presidents on Twitter. These results offer a piece of culturally significant data in themselves---the ratio of response types is an unofficial measurement on the platform. Moreover, the results improve our understanding of the temporal dynamics that lead to the final counts that users may ultimately see. Deeply analyzing response activity dynamics provides insights on how the public responds to posts, the tenacity of supporters, and abnormalities that may be indicative of inauthentic behavior. The second study examines the interaction between gender biases in health records and language models and how to mitigate these biases. We present specific language that is more commonly associated with female and male patients. We go on to demonstrate how the deliberate augmentation of text can minimize the gender signal present in data while retaining performance on medically relevant tasks. We conclude by showing how much of this bias is domain specific, and the non-trivial interaction with general-purpose language models. Our final study investigates gender bias in resume text and relates this bias to the gender wage-gap. We show that language differences within occupations are associated with the gender pay gap. Our results highlight the value of utilizing high dimensional representations of individuals and the potential for previously undocumented biases to influence hiring pipelines

Deep learning for Alzheimer’s disease: towards the development of an assistive diagnostic tool

The past decade has witnessed rapid advances at the intersection of machine learning and medicine. Owing to the tremendous amount of digitized hospital data, machine learning is poised to bring innovation to the traditional healthcare workflow. Though machine learning models have strong predictive power, it is challenging to translate a research project into a clinical tool partly due to the lack of a rigorous validation framework. In this dissertation, I presented a range of machine learning models that were trained to classify Alzheimer’s disease - a condition with an insidious onset - using routinely collected clinical data. In addition to reporting the model performance, I discussed several considerations, including feature selection, data harmonization, effect of confounding variables, diagnostic scope, model interpretability and validation, which are critical to the design, development, and validation of machine learning models. From the methodological standpoint, I presented a multidisciplinary collaboration in which medical domain knowledge which was obtained from experts and tissue examinations was tightly integrated with the interpretable outcomes derived from our machine learning frameworks. I demonstrated that the model, which generalized well on multiple independent cohorts, achieved diagnostic performance on par with a group of medical professionals. The interpretable analysis of our model showed that its underlying decision logic corresponds with expert ratings and neuropathological findings. Taken together, this work presented a machine learning system for classification of Alzheimer’s disease, marking an important milestone towards a translatable clinical application in the future