MULTIVARIATE MODELING OF COGNITIVE PERFORMANCE AND CATEGORICAL PERCEPTION FROM NEUROIMAGING DATA

State-of-the-art cognitive-neuroscience mainly uses hypothesis-driven statistical testing to characterize and model neural disorders and diseases. While such techniques have proven to be powerful in understanding diseases and disorders, they are inadequate in explaining causal relationships as well as individuality and variations. In this study, we proposed multivariate data-driven approaches for predictive modeling of cognitive events and disorders. We developed network descriptions of both structural and functional connectivities that are critical in multivariate modeling of cognitive performance (i.e., fluency, attention, and working memory) and categorical perceptions (i.e., emotion, speech perception). We also performed dynamic network analysis on brain connectivity measures to determine the role of different functional areas in relation to categorical perceptions and cognitive events. Our empirical studies of structural connectivity were performed using Diffusion Tensor Imaging (DTI). The main objective was to discover the role of structural connectivity in selecting clinically interpretable features that are consistent over a large range of model parameters in classifying cognitive performances in relation to Acute Lymphoblastic Leukemia (ALL). The proposed approach substantially improved accuracy (13% - 26%) over existing models and also selected a relevant, small subset of features that were verified by domain experts. In summary, the proposed approach produced interpretable models with better generalization.Functional connectivity is related to similar patterns of activation in different brain regions regardless of the apparent physical connectedness of the regions. The proposed data-driven approach to the source localized electroencephalogram (EEG) data includes an array of tools such as graph mining, feature selection, and multivariate analysis to determine the functional connectivity in categorical perceptions. We used the network description to correctly classify listeners behavioral responses with an accuracy over 92% on 35 participants. State-of-the-art network description of human brain assumes static connectivities. However, brain networks in relation to perception and cognition are complex and dynamic. Analysis of transient functional networks with spatiotemporal variations to understand cognitive functions remains challenging. One of the critical missing links is the lack of sophisticated methodologies in understanding dynamics neural activity patterns. We proposed a clustering-based complex dynamic network analysis on source localized EEG data to understand the commonality and differences in gender-specific emotion processing. Besides, we also adopted Bayesian nonparametric framework for segmentation neural activity with a finite number of microstates. This approach enabled us to find the default network and transient pattern of the underlying neural mechanism in relation to categorical perception. In summary, multivariate and dynamic network analysis methods developed in this dissertation to analyze structural and functional connectivities will have a far-reaching impact on computational neuroscience to identify meaningful changes in spatiotemporal brain activities

Feasibility of Hidden Markov Models for the Description of Time-Varying Physiologic State After Severe Traumatic Brain Injury.

OBJECTIVES: Continuous assessment of physiology after traumatic brain injury is essential to prevent secondary brain insults. The present work aims at the development of a method for detecting physiologic states associated with the outcome from time-series physiologic measurements using a hidden Markov model. DESIGN: Unsupervised clustering of hourly values of intracranial pressure/cerebral perfusion pressure, the compensatory reserve index, and autoregulation status was attempted using a hidden Markov model. A ternary state variable was learned to classify the patient's physiologic state at any point in time into three categories ("good," "intermediate," or "poor") and determined the physiologic parameters associated with each state. SETTING: The proposed hidden Markov model was trained and applied on a large dataset (28,939 hr of data) using a stratified 20-fold cross-validation. PATIENTS: The data were collected from 379 traumatic brain injury patients admitted to Addenbrooke's Hospital, Cambridge between 2002 and 2016. INTERVENTIONS: Retrospective observational analysis. MEASUREMENTS AND MAIN RESULTS: Unsupervised training of the hidden Markov model yielded states characterized by intracranial pressure, cerebral perfusion pressure, compensatory reserve index, and autoregulation status that were physiologically plausible. The resulting classifier retained a dose-dependent prognostic ability. Dynamic analysis suggested that the hidden Markov model was stable over short periods of time consistent with typical timescales for traumatic brain injury pathogenesis. CONCLUSIONS: To our knowledge, this is the first application of unsupervised learning to multidimensional time-series traumatic brain injury physiology. We demonstrated that clustering using a hidden Markov model can reduce a complex set of physiologic variables to a simple sequence of clinically plausible time-sensitive physiologic states while retaining prognostic information in a dose-dependent manner. Such states may provide a more natural and parsimonious basis for triggering intervention decisions

Passive acoustic monitoring for assessment of natural and anthropogenic sound sources in the marine environment using automatic recognition

In the marine environment, sound can be an efficient source of information. Indeed, several marine species, including fish, use sound to navigate, select habitats, detect predators and prey, and to attract mates. Therefore, all the abiotic, biotic and manmade sounds that comprise the soundscape, have the potential to be used to assess and monitor species and marine environments. Passive acoustic monitoring (PAM) involves the use of acoustic sensors to record sound in the environment, from which relevant ecological information can be inferred. This thesis studied marine soundscapes, with special attention on fish communities, anthropogenic noise, and applied several methods to analyse acoustic recordings. Most of the focus was on the Tagus estuary, where the presence of two highly vocal species is known: the Lusitanian toadfish (Halobatrachus didactylus) and the meagre (Argyrosomus regius). Azorean and Mozambique soundscapes were also analysed. Several methods were applied to extract information and to visualize soundscape characteristics, including sound recognition systems based on hidden Markov models to recognize fish sounds and boat passages. Analysis of several types of marine environments and time scales showed several advantages and disadvantages of different methods. The use of sound pressure level on different frequency bands allowed the quantification of daily and seasonal patterns. Ecoacoustic indices appear to be cost-effective tools to monitor biodiversity in some marine environments. Using automatic recognition, vocal rhythms (diel and seasonal patterns) and vocal interactions among individuals were also characterized. Furthermore, boat noise effects on fish were studied: we encountered impacts on the audition, vocal behaviour and reproduction. Overall, we used PAM as a tool to remotely assess and monitor soundscapes, biodiversity, fish communities’ seasonal patterns, fish behaviour, species presence, and the effect of anthropogenic noise aiming to contribute for the management and conservation of marine ecosystems

An Investigation and Application of Biology and Bioinformatics for Activity Recognition

Activity recognition in a smart home context is inherently difficult due to the variable nature of human activities and tracking artifacts introduced by video-based tracking systems. This thesis addresses the activity recognition problem via introducing a biologically-inspired chemotactic approach and bioinformatics-inspired sequence alignment techniques to recognise spatial activities. The approaches are demonstrated in real world conditions to improve robustness and recognise activities in the presence of innate activity variability and tracking noise

The listening talker: A review of human and algorithmic context-induced modifications of speech

International audienceSpeech output technology is finding widespread application, including in scenarios where intelligibility might be compromised - at least for some listeners - by adverse conditions. Unlike most current algorithms, talkers continually adapt their speech patterns as a response to the immediate context of spoken communication, where the type of interlocutor and the environment are the dominant situational factors influencing speech production. Observations of talker behaviour can motivate the design of more robust speech output algorithms. Starting with a listener-oriented categorisation of possible goals for speech modification, this review article summarises the extensive set of behavioural findings related to human speech modification, identifies which factors appear to be beneficial, and goes on to examine previous computational attempts to improve intelligibility in noise. The review concludes by tabulating 46 speech modifications, many of which have yet to be perceptually or algorithmically evaluated. Consequently, the review provides a roadmap for future work in improving the robustness of speech output

Vocal rhythms in nesting Lusitanian toadfish, Halobatrachus didactylus

Males of several fish species aggregate and vocalize together, increasing the detection range of the sounds and their chances of mating. In the Lusitanian toadfish (Halobatrachus didactylus), breeding males build nests under rocks in close proximity and produce hundreds of boatwhistles (BW) an hour to attract females to lay their demersal eggs on their nests. Chorusing behaviour includes fine-scale interactions between individuals, a behavioural dynamic worth investigating in this highly vocal fish. Here we present a study to further investigate this species’ vocal temporal patterns on a fine (individual rhythms and male-male interactions) and large (chorus daily patterns) scales. Several datasets recorded in the Tagus estuary were labelled with the support of an automatic recognition system based on hidden Markov models. Fine-scale vocal temporal patterns exhibit high variability between and within individuals, varying from an almost isochronous to an apparent aperiodic pattern. When in a chorus, males exhibited alternation or synchrony calling patterns, possibly depending on motivation and social context (mating or male-male competition). When engaged in sustained calling, males usually alternated vocalizations with their close neighbours thus avoiding superposition of calls. Synchrony was observed mostly in fish with lower mean calling rate. Interaction patterns were less obvious in more distanced males. Daily choruses showed periods with several active calling males and periods of low activity with no significant diel patterns in shallower intertidal waters. Here, chorusing activity was mainly affected by tide level. In contrast, at a deeper location, although tidal currents causes a decrease in calling rate, tide level did not significantly influence calling, and there was a higher calling rate at night. These data show that photoperiod and tide levels can influence broad patterns of Lusitanian toadfish calling activity as in other shallow-water fishes, but fine temporal patterns in acoustic interactions among nesting males is more complex than previously known for fishes.Fundação para a Ciência e Tecnologia - FCTinfo:eu-repo/semantics/publishedVersio

SYNTHESIZING DYSARTHRIC SPEECH USING MULTI-SPEAKER TTS FOR DSYARTHRIC SPEECH RECOGNITION

Dysarthria is a motor speech disorder often characterized by reduced speech intelligibility through slow, uncoordinated control of speech production muscles. Automatic Speech recognition (ASR) systems may help dysarthric talkers communicate more effectively. However, robust dysarthria-specific ASR requires a significant amount of training speech is required, which is not readily available for dysarthric talkers. In this dissertation, we investigate dysarthric speech augmentation and synthesis methods. To better understand differences in prosodic and acoustic characteristics of dysarthric spontaneous speech at varying severity levels, a comparative study between typical and dysarthric speech was conducted. These characteristics are important components for dysarthric speech modeling, synthesis, and augmentation. For augmentation, prosodic transformation and time-feature masking have been proposed. For dysarthric speech synthesis, this dissertation has introduced a modified neural multi-talker TTS by adding a dysarthria severity level coefficient and a pause insertion model to synthesize dysarthric speech for varying severity levels. In addition, we have extended this work by using a label propagation technique to create more meaningful control variables such as a continuous Respiration, Laryngeal and Tongue (RLT) parameter, even for datasets that only provide discrete dysarthria severity level information. This approach increases the controllability of the system, so we are able to generate more dysarthric speech with a broader range. To evaluate their effectiveness for synthesis of training data, dysarthria-specific speech recognition was used. Results show that a DNN-HMM model trained on additional synthetic dysarthric speech achieves WER improvement of 12.2% compared to the baseline, and that the addition of the severity level and pause insertion controls decrease WER by 6.5%, showing the effectiveness of adding these parameters. Overall results on the TORGO database demonstrate that using dysarthric synthetic speech to increase the amount of dysarthric-patterned speech for training has a significant impact on the dysarthric ASR systems