Synergizing human-machine intelligence: Visualizing, labeling, and mining the electronic health record

We live in a world where data surround us in every aspect of our lives. The key challenge for humans and machines is how we can make better use of such data. Imagine what would happen if you were to have intelligent machines that could give you insight into the data. Insight that will enable you to better 1) reason about, 2) learn, and 3) understand the underlying phenomena that produced the data. The possibilities of combined human-machine intelligence are endless and will impact our lives in ways we can not even imagine today. Synergistic human-machine intelligence aims to facilitate the analytical reasoning and inference process of humans by creating machines that maximize a human's ability to 1) reason about, 2) learn, and 3) understand large, complex, and heterogeneous data. Combined human-machine intelligence is a powerful symbiosis of mutual benefit, in which we depend on the computational capabilities of the machine for the tasks we are not good at, and the machine requires human intervention for the tasks it performs poorly on. This relationship provides a compelling alternative to either approach in isolation for solving today's and tomorrow's arising data challenges. In his regard, this dissertation proposes a diverse analytical framework that leverages synergistic human-machine intelligence to maximize a human's ability to better 1) reason about, 2) learn, and 3) understand different biomedical imaging and healthcare data present in the patient's electronic health record (EHR). Correspondingly, we approach the data analyses problem from the 1) visualization, 2) labeling, and 3) mining perspective and demonstrate the efficacy of our analytics on specific application scenarios and various data domains. In the first part of this dissertation we explore the question how we can build intelligent imaging analytics that are commensurate with human capabilities and constraints, specifically for optimizing data visualization and automated labeling workflows. Our journey starts with heuristic rule-based analytical models that are derived from task-specific human knowledge. From this experience, we move on to data-driven analytics, where we adapt and combine the intelligence of the model based on prior information provided by the human and synthetic knowledge learned from partial data observations. Within this realm, we propose a novel Bayesian transductive Markov random field model that requires minimal human intervention and is able to cope with scarce label information to learn and infer object shapes in complex spatial, multimodal, spatio-temporal, and longitudinal data. We then study the question how machines can learn discriminative object representations from dense human provided label information by investigating learning and inference mechanisms that make use of deep learning architectures. The developed analytics can aid visualization and labeling tasks, which enables the interpretation and quantification of clinically relevant image information. The second part explores the question how we can build data-driven analytics for exploratory analysis in longitudinal event data that are commensurate with human capabilities and constraints. We propose human-intuitive analytics that enable the representation and discovery of interpretable event patterns to ease knowledge absorption and comprehension of the employed analytics model and the underlying data. We propose a novel doubly-constrained convolutional sparse-coding framework that learns interpretable and shift-invariant latent temporal event patterns. We apply the model to mine complex event data in EHRs. By mapping the event space to heterogeneous patient encounters in the EHR we explore the linkage between healthcare resource utilization (HRU) in relation to disease severity. This linkage may help to better understand how disease specific co-morbidities and their clinical attributes incur different HRU patterns. Such insight helps to characterize the patient's care history, which then enables the comparison against clinical practice guidelines, the discovery of prevailing practices based on common HRU group patterns, and the identification of outliers that might indicate poor patient management

Apport de l’IRM structurelle multimodale dans la chirurgie d’épilepsie : le cas de l’épilepsie insulaire

L’épilepsie insulaire (ÉI) est une forme rare d’épilepsie focale qui, en raison des défis liés à son diagnostic, est difficilement cernable. De plus, la prise en charge des patients avec ÉI s’avère complexifiée par le fait que cette pathologie est fréquemment résistante aux médicaments anti-crises. Pour ces cas médico-réfractaires, la chirurgie insulaire est une option viable. Cela dit, les patients subissant une telle intervention développent fréquemment des déficits neurologiques postopératoires; heureusement, la grande majorité de ceux-ci récupèrent complètement et rapidement. Or, le mécanisme sous-tendant ce singulier rétablissement fonctionnel demeure à ce jour mal compris. Deux modalités modernes d’IRM structurelle, soit l’analyse d’épaisseur corticale et la tractographie, ont permis, dans les dernières années, de décrire les altérations architecturales caractéristiques et potentiellement diagnostiques de divers types d’épilepsie ainsi que de caractériser les remodelages plastiques qui suivent la chirurgie de l’épilepsie extra-insulaire. Cependant, à ce jour, aucune étude ne s’est encore penchée sur le cas de l’ÉI. De ce fait, les études qui constituent cette thèse exploitent l’IRM structurelle afin, d’une part, de dépeindre les altérations d’épaisseur du cortex et de connectivité de matière blanche associées à l’ÉI et, d’autre part, de définir les réarrangements de connectivité subséquents à la chirurgie insulaire pour contrôle épileptique. Les deux premières études de cette thèse ont révélé que l’ÉI était associée à un pattern majoritairement ipsilatéral d’atrophie corticale et d’hyperconnectivité impliquant principalement des sous-régions insulaires et des régions connectées à l’insula. De manière intéressante, la topologie de ces changements correspondait, au moins en partie, à celle du réseau épileptique de l’ÉI. Ensuite, la troisième étude visait à décrire, par le biais d’une méta-analyse, l’histoire naturelle postopératoire des patients subissant une chirurgie pour ÉI. Cette analyse a, entre autres, confirmé que cette chirurgie était efficace (66.7% de disparition des crises) et qu’elle était fréquemment accompagnée de complications neurologiques (42.5%) qui, dans la plupart des cas, étaient transitoires (78.7% des complications) et récupéraient entièrement dans les trois mois postopératoires (91.6% des complications transitoires). Finalement, la quatrième étude a révélé que la chirurgie pour ÉI était suivie d’altérations de connectivité diffuses et bilatérales. Notamment, les connexions présentant une augmentation de connectivité concernaient particulièrement des régions localisées soit près de la cavité chirurgicale ou dans l’hémisphère controlatéral à l’intervention. De plus, la majorité de ces renforcements structurels se sont produits dans les six premiers mois suivant la chirurgie, un délai comparable à celui durant lequel la majeure partie de la récupération fonctionnelle postopératoire a été observée dans notre méta-analyse. En somme, nos résultats suggèrent que les altérations morphologiques en lien avec l’ÉI peuvent correspondre à son réseau épileptique sous-jacent. La topologie de ces changements pourrait constituer un biomarqueur structurel diagnostique qui aiderait à la reconnaissance de l’ÉI et, concomitamment, favoriserait possiblement un traitement chirurgical plus adapté et plus efficace. De plus, les augmentations de connectivité postopératoires pourraient correspondre à des réponses neuroplastiques permettant de prendre en charge les fonctions altérées par la chirurgie. Nos constats ont ainsi contribué à la caractérisation des mécanismes étayant la singulière récupération fonctionnelle accompagnant la chirurgie pour ÉI. À plus grande échelle, nos travaux offrent un aperçu du potentiel de l’IRM structurelle à assister au diagnostic de l’épilepsie focale ainsi qu’à participer à la description des changements plastiques subséquents à une résection neurochirurgicale.Insular epilepsy (IE) is a rare type of focal epilepsy that is difficult to diagnose. In addition to the challenging nature of IE detection, management of patients with this condition is complicated by the tendency of insular seizures to be resistant to anti-seizure medications. For such medically refractory cases, insular surgery constitutes a viable and long-lasting therapeutic option. That said, patients who undergo an insular resection for seizure control frequently develop postoperative neurological deficits; fortunately, most of these impairments recover fully and rapidly. While this favorable postoperative course contributes to improving the outcome of IE surgery, the mechanism underlying the functional recovery remains unknown. Two contemporary structural MRI modalities, namely cortical thickness analysis and tractography, have recently been used to describe characteristic structural alterations of focal epilepsies and to elucidate the postoperative plastic remodeling associated with surgery for extra-insular epilepsy. While these analyses added to our understanding of several localization-related epilepsies, none specifically studied IE. In this thesis, we exploit structural MRI techniques to, first, depict the alterations of cortical thickness and white matter connectivity in IE and, second, define the progressive rearrangements that follow insular surgery for epilepsy. The first two studies of the current thesis showed that IE is associated with a primarily ipsilateral pattern of cortical thinning and hyperconnectivity that mainly involves insular subregions and insula-connected regions. Interestingly, the topology of these changes corresponded, at least in part, to the epileptic network of IE. Furthermore, the third study aimed to describe, via a meta-analysis, the postoperative outcome of patients undergoing surgery for IE. Among other findings, the analysis revealed that insular surgery was effective (66.7% seizure freedom rate) but was associated with a significant risk of neurological complications (42.5%) which, in most cases, were transient (78.7% of all complications) and recovered fully within three months (91.6% of transient complications). Finally, the fourth study showed that surgery for IE was followed by a diffuse pattern of bilateral structural connectivity changes. Notably, connections exhibiting an increase in connectivity were specifically located near the surgical cavity and in the contralateral healthy hemisphere. In addition, the majority of the structural strengthening occurred in the first six months following surgery, a time course that is consistent with the short delay during which most of the postoperative functional recovery was observed in our meta-analysis. Our results suggest that the morphological alterations in IE may reflect its underlying epileptic network. The topology of these changes may constitute a structural biomarker that could help diagnose IE more readily and, concomitantly, potentially enable a more targeted and more effective surgical treatment. Moreover, the postoperative increases in connectivity may be compatible with compensatory neuroplastic responses, a process that arose to recoup the functions of the injured insular cortex. Our findings have therefore contributed to the characterization of the driving process that supports the striking functional recovery seen following surgery for IE. On a larger scale, our work provides insights into the potential of structural MRI to assist in the diagnosis of focal epilepsy and to describe plastic changes following neurosurgical resections