Contextual Understanding of Sequential Data Across Multiple Modalities

In recent years, progress in computing and networking has made it possible to collect large volumes of data for various different applications in data mining and data analytics using machine learning methods. Data may come from different sources and in different shapes and forms depending on their inherent nature and the acquisition process. In this dissertation, we focus specifically on sequential data, which have been exponentially growing in recent years on platforms such as YouTube, social media, news agency sites, and other platforms. An important characteristic of sequential data is the inherent causal structure with latent patterns that can be discovered and learned from samples of the dataset. With this in mind, we target problems in two different domains of Computer Vision and Natural Language Processing that deal with sequential data and share the common characteristics of such data. The first one is action recognition based on video data, which is a fundamental problem in computer vision. This problem aims to find generalized patterns from videos to recognize or predict human actions. A video contains two important sets of information, i.e. appearance and motion. These information are complementary, and therefore an accurate recognition or prediction of activities or actions in video data depend significantly on our ability to extract them both. However, effective extraction of these information is a non-trivial task due to several challenges, such as viewpoint changes, camera motions, and scale variations, to name a few. It is thus crucial to design effective and generalized representations of video data that learn these variations and/or are invariant to such variations. We propose different models that learn and extract spatio-temporal correlations from video frames by using deep networks that overcome these challenges. The second problem that we study in this dissertation in the context of sequential data analysis is text summarization in multi-document processing. Sentences consist of sequence of words that imply context. The summarization task requires learning and understanding the contextual information from each sentence in order to determine which subset of sentences forms the best representative of a given article. With the progress made by deep learning, better representations of words have been achieved, leading in turn to better contextual representations of sentences. We propose summarization methods that combine mathematical optimization, Determinantal Point Processes (DPPs), and deep learning models that outperform the state of the art in multi-document text summarization

Lithium and GSK3-β promoter gene variants influence white matter microstructure in bipolar disorder

Lithium is the mainstay for the treatment of bipolar disorder (BD) and inhibits glycogen synthase kinase 3-β (GSK3-β). The less active GSK3-β promoter gene variants have been associated with less detrimental clinical features of BD. GSK3-β gene variants and lithium can influence brain gray matter structure in psychiatric conditions. Diffusion tensor imaging (DTI) measures of white matter (WM) integrity showed widespred disruption of WM structure in BD. In a sample of 70 patients affected by a major depressive episode in course of BD, we investigated the effect of ongoing long-term lithium treatment and GSK3-β promoter rs334558 polymorphism on WM microstructure, using DTI and tract-based spatial statistics with threshold-free cluster enhancement. We report that the less active GSK3-β rs334558*C gene-promoter variants, and the long-term administration of the GSK3-β inhibitor lithium, were associated with increases of DTI measures of axial diffusivity (AD) in several WM fiber tracts, including corpus callosum, forceps major, anterior and posterior cingulum bundle (bilaterally including its hippocampal part), left superior and inferior longitudinal fasciculus, left inferior fronto-occipital fasciculus, left posterior thalamic radiation, bilateral superior and posterior corona radiata, and bilateral corticospinal tract. AD reflects the integrity of axons and myelin sheaths. We suggest that GSK3-β inhibition and lithium could counteract the detrimental influences of BD on WM structure, with specific benefits resulting from effects on specific WM tracts contributing to the functional integrity of the brain and involving interhemispheric, limbic, and large frontal, parietal, and fronto-occipital connections

Relationship between white matter changes and aggression in methamphetamine dependence

Background: Methamphetamine (MA) abuse is a growing problem in the world and especially in South Africa’s Western Cape. Amphetamine-type stimulants have become the second most widely abused illicit drugs worldwide. Admission data from substance abuse treatment centres in the Western Cape show the fastest increase for any drug ever noted in the country in MA related admissions. MA has neurotoxic effects on the brain leading, amongst other effects, to white matter (WM) changes. Moreover, increased levels of aggression are commonly found in individuals with MA abuse. Although behavioural deficits are well described, the underlying mechanisms are still poorly understood. While previous studies have examined WM abnormalities relating to cognitive impairment, none have investigated associations between WM integrity in individuals with MA dependence and aggression. Methods: Diffusion Tensor Imaging (DTI) was used to investigate WM changes in 40 individuals with MA dependence and 40 matched healthy control subjects. Aggression was measured with the Buss & Perry Questionnaire in 40 MA users and 36 controls. Two approaches to assess WM integrity in the brain were employed. First, whole brain voxel wise comparison across subjects using tract based spatial statistics (TBSS) in FSL was used. Fractional anisotropy (FA), mean diffusivity (MD), parallel diffusivity (λ║) and perpendicular diffusivity (λ┴) were compared between the two groups. Second, a region of interest (ROI) approach was used, which focused on three WM tracts in the frontal brain, commonly found to play a role in aggressive behaviour: (1) the genu of the corpus callosum (CC), (2) the cingulum and (3) the uncinate fasciculus

Knowledge Modelling and Learning through Cognitive Networks

One of the most promising developments in modelling knowledge is cognitive network science, which aims to investigate cognitive phenomena driven by the networked, associative organization of knowledge. For example, investigating the structure of semantic memory via semantic networks has illuminated how memory recall patterns influence phenomena such as creativity, memory search, learning, and more generally, knowledge acquisition, exploration, and exploitation. In parallel, neural network models for artificial intelligence (AI) are also becoming more widespread as inferential models for understanding which features drive language-related phenomena such as meaning reconstruction, stance detection, and emotional profiling. Whereas cognitive networks map explicitly which entities engage in associative relationships, neural networks perform an implicit mapping of correlations in cognitive data as weights, obtained after training over labelled data and whose interpretation is not immediately evident to the experimenter. This book aims to bring together quantitative, innovative research that focuses on modelling knowledge through cognitive and neural networks to gain insight into mechanisms driving cognitive processes related to knowledge structuring, exploration, and learning. The book comprises a variety of publication types, including reviews and theoretical papers, empirical research, computational modelling, and big data analysis. All papers here share a commonality: they demonstrate how the application of network science and AI can extend and broaden cognitive science in ways that traditional approaches cannot

Deep Learning -- A first Meta-Survey of selected Reviews across Scientific Disciplines, their Commonalities, Challenges and Research Impact

Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Similar to the basic structure of a brain, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed with (sensory) data, like texts, images, videos or sounds. These networks outperform the state-of-the-art methods in different tasks and, because of this, the whole field saw an exponential growth during the last years. This growth resulted in way over 10,000 publications per year in the last years. For example, the search engine PubMed alone, which covers only a sub-set of all publications in the medical field, provides already over 11,000 results in Q3 2020 for the search term 'deep learning', and around 90% of these results are from the last three years. Consequently, a complete overview over the field of deep learning is already impossible to obtain and, in the near future, it will potentially become difficult to obtain an overview over a subfield. However, there are several review articles about deep learning, which are focused on specific scientific fields or applications, for example deep learning advances in computer vision or in specific tasks like object detection. With these surveys as a foundation, the aim of this contribution is to provide a first high-level, categorized meta-survey of selected reviews on deep learning across different scientific disciplines. The categories (computer vision, language processing, medical informatics and additional works) have been chosen according to the underlying data sources (image, language, medical, mixed). In addition, we review the common architectures, methods, pros, cons, evaluations, challenges and future directions for every sub-category.Comment: 83 pages, 22 figures, 9 tables, 100 reference

An Unsupervised Framework for Online Spatiotemporal Detection of Activities of Daily Living by Hierarchical Activity Models

International audienceAutomatic detection and analysis of human activities captured by various sensors (e.g. 1 sequence of images captured by RGB camera) play an essential role in various research fields in order 2 to understand the semantic content of a captured scene. The main focus of the earlier studies has 3 been widely on supervised classification problem, where a label is assigned for a given short clip. 4 Nevertheless, in real-world scenarios, such as in Activities of Daily Living (ADL), the challenge is 5 to automatically browse long-term (days and weeks) stream of videos to identify segments with 6 semantics corresponding to the model activities and their temporal boundaries. This paper proposes 7 an unsupervised solution to address this problem by generating hierarchical models that combine 8 global trajectory information with local dynamics of the human body. Global information helps in 9 modeling the spatiotemporal evolution of long-term activities and hence, their spatial and temporal 10 localization. Moreover, the local dynamic information incorporates complex local motion patterns of 11 daily activities into the models. Our proposed method is evaluated using realistic datasets captured 12 from observation rooms in hospitals and nursing homes. The experimental data on a variety of 13 monitoring scenarios in hospital settings reveals how this framework can be exploited to provide 14 timely diagnose and medical interventions for cognitive disorders such as Alzheimer's disease. The 15 obtained results show that our framework is a promising attempt capable of generating activity 16 models without any supervision. 1