Complete-linkage clustering for voice activity detection in audio and visual speech

We propose a novel technique for conducting robust voice activity detection (VAD) in high-noise recordings. We use Gaussian mixture modeling (GMM) to train two generic models; speech and non-speech. We then score smaller segments of a given (unseen) recording against each of these GMMs to obtain two respective likelihood scores for each segment. These scores are used to compute a dissimilarity measure between pairs of segments and to carry out complete-linkage clustering of the segments into speech and non-speech clusters. We compare the accuracy of our method against state-of-the-art and standardised VAD techniques to demonstrate an absolute improvement of 15% in half-total error rate (HTER) over the best performing baseline system and across the QUT-NOISE-TIMIT database. We then apply our approach to the Audio-Visual Database of American English (AVDBAE) to demonstrate the performance of our algorithm in using visual, audio-visual or a proposed fusion of these features

A Novel Nonparametric Test for Heterogeneity Detection and Assessment of Fluid Removal Among CRRT Patients in ICU

Over the past decade acute kidney injury (AKI) has been occurring among 20%-50% of patients admitted to the intensive care unit (ICU) in United States. Continuous renal replacement therapy (CRRT) has become a popular treatment method among these critically ill patients. But there are multiple complications in implementing this treatment, including discrepancies in practiced and prescribed fluid removal, possibly related to the heterogeneity among these patients. With mixture modeling there have been several techniques in detecting heterogeneity with their specific limitations. In this dissertation a novel nonparametric ‘d test’ will be used to detect heterogeneity among CRRT patients in ICU. Along with heterogeneity detection, this dissertation will also seek to understand ongoing issues with fluid removal and discrepancy in treatment implementations

The role of sleep and shift work in dementia and cognitive aging : an epidemiological approach

Dementia is a syndrome that afflicts older persons and is characterized by progressive deterioration in memory and other cognitive functions, behavior, and the ability to perform day-to-day activities. Older adults, and even more so those suffering from dementia, often experience disturbances in the sleep-wake cycle. This thesis sought to investigate the longterm effect of sleep and shift work (SW), which can affect sleep patterns and sleep quality, in dementia and cognitive aging using prospective and longitudinal studies of the Swedish Twin Registry (STR). Study I prospectively explored whether sleep characteristics such as time in bed (TIB), rising time, bedtime, sleep quality, non-restorative sleep, and snoring in late life were associated with dementia incidence while accounting for baseline cognitive functioning. The study was based on a sample of 11,247 participants aged 65 and older. Short (≤6 hours) and extended (>9 hours) TIB as well as late rising time (rising 8:00 AM or later) were associated with higher dementia incidence in the following 17 years. After stratifying by baseline cognitive status, only the association between short TIB and dementia remained in those cognitively intact at baseline. Altogether, the findings suggest that extended TIB and late rising represent prodromal features whereas short TIB appeared to be a risk factor for dementia. Study II employed quantitative genetic methods to investigate the relative importance of genetic influences for late-life sleep characteristics, dementia and Alzheimer’s disease (AD), and whether dementia-related sleep characteristics modified genetic influences on dementia and AD in a sample of 10,894 twins. Genetic influences accounted for about half of the variation in liability to dementia and AD, and for late rise time and bedtime. For the other sleep phenotypes assessed, non-shared environment contributed a larger part of the phenotypic variation. TIB and late rising were associated with dementia incidence, but these sleep traits did not moderate the genetic influences on dementia and AD. Study III examined the association between SW and dementia incidence. The study was based on two cohorts: one cohort comprised 13,283 individuals with information on anytype SW and another cohort comprised 41,199 individuals with information on night work (NW), with follow-up time spanning 41 and 14 years, respectively. History of SW, including NW, was associated with higher dementia incidence. Further, longer duration of SW and NW appeared to be associated with greater risk of dementia. Study IV estimated the impact of SW on change in cognitive performance before and after retirement. The study included 595 individuals at least 50 years of age at baseline who had been employed and who had undergone up to 9 assessments of cognitive performance over a period of 27 years. Latent growth curve modeling showed that SW and NW during midlife were not associated with greater rate of change in any of the cognitive domains (verbal, spatial, memory, and processing speed) later in life. In summary, the work in this thesis has combined unique population-based data sources and modern epidemiological methods to evaluate the role of sleep and SW in dementia and cognitive aging. While SW did not appear to explain differences in rate of normative cognitive change in later life, findings indicate that SW and sleep characteristics are associated with increased risk of dementia

Prediction and Monitoring of Progression of Alzheimer’s Disease : Multivariable approaches for decision support

Alzheimerin tauti, yksi yleisimmistä muistisairauksista, on hitaasti etenevä aivoja rappeuttava tauti, jolle ei ole vielä parantavaa hoitoa. Tietyt lääkkeet ja elämäntapainterventiot voivat kuitenkin hidastaa taudin etenemistä ja lievittää sen oireita, mikä parantaa potilaiden elämänlaatua ja terveydenhuollon kustannusvaikuttavuutta. Alzheimerin taudin varhainen diagnostiikka on erittäin tärkeää, koska erilaiset interventiot pitäisi aloittaa jo taudin varhaisessa vaiheessa, jotta niillä saataisiin aikaan paras mahdollinen vaikutus. Taudin varhainen diagnostiikka on kuitenkin haastavaa, koska muutokset aivoissa alkavat vuosia tai vuosikymmeniä ennen ensimmäisten oireiden ilmaantumista. Lisäksi viime vuosien tutkimus on tuottanut tietoa suuresta määrästä erilaisia testejä ja biomarkkereita, jotka voivat vaikuttaa taudin diagnoosiin ja prognoosiin. Tiedon suuri määrä saattaa aiheuttaa informaatioähkyä kliinikoille vaikeuttaen heidän päätöksentekoaan. Datalähtöiset analytiikka- ja visualisointimenetelmät voivat auttaa suuren ja heterogeenisen tietomäärän tulkinnassa ja hyödyntämisessä. Ne voivat siten tukea kliinikkoa hänen päätöksenteossaan. Lisäksi nämä menetelmät voivat auttaa tunnistamaan sopivia potilaita kliinisiin lääketutkimuksiin, joiden tavoitteena on kehittää Alzheimerin taudin etenemistä hidastavia lääkkeitä. Tämän väitöskirjan tavoitteena oli kehittää datalähtöisiä menetelmiä Alzheimerin taudin etenemisen ennustamiseen ja seurantaan taudin eri vaiheisiin alkaen normaalista kognitiosta ja edeten kuolemaan. Mallien kehittämisessä hyödynnettiin kognitiivisten ja neuropsykologisten testien tuloksia, magneettikuvantamista (MRI), selkäydinnestenäytteitä, ja genetiikkaa (apolipoproteiini E). Väitöskirja koostuu neljästä alkuperäisestä tutkimuksesta, jotka on julkaistu kansainvälisissä tieteellisissä lehdissä. Ensimmäinen osatutkimus keskittyi Alzheimerin taudin varhaiseen vaiheeseen. Tutkimuksessa käytettiin ohjattua koneoppimisen menetelmää Disease State Index (DSI, taudin tilan indeksi) ennustamaan, kenellä subjektiivisesti koettu kognition heikkeneminen etenee taudin vakavampaan vaiheeseen eli lievään kognition heikentymiseen (mild cognitive impairment, MCI) tai dementiaan. Tutkimuksen aineisto koostui 647 henkilöstä kolmesta eurooppalaisesta muisti- klinikkakohortista. Kun yhdistettiin useita eri muuttujia DSI-menetelmällä, ROC- käyrän (engl. Receiver Operating Characteristic curve) alle jäävä pinta-ala (AUC) oli 0.81 ja tasapainotettu tarkkuus oli 74%. Negatiivinen ennustearvo oli korkea (93%) ja positiivinen ennustearvo oli matala (38%). Kun DSI-malli validoitiin erillisellä testikohortilla, mallin AUC huononi 11%. Lisäanalyysit osoittivat, että useat erot kohorttien välillä voivat selittää suorituskyvyn alenemista. Toinen osatutkimus keskittyi taudin myöhäisempään vaiheeseen. DSI-menetelmällä analysoitiin pitkittäistä dataa, joka koostui 273 henkilön MCI-kohortista. Kohortti hankittiin Alzheimer’s Disease and Neuroimaging (ADNI 1) tietokannasta. DSI-arvojen muutokset ajan kuluessa olivat erilaiset niillä, joiden tauti eteni Alzheimerin taudin dementiaksi, ja niillä, joilla tauti pysyi MCI-vaiheessa. Lisäksi huomattiin, että stabiilina pysynyt MCI-ryhmä koostui kahdesta aliryhmästä: ensimmäisessä ryhmässä DSI-arvot pysyivät vakaina ja toisessa ryhmässä DSI-arvot kohosivat. Tämä indikoi, että toisessa ryhmässä tauti saattaa edetä dementiaksi tulevaisuudessa. Näiden analyysien lisäksi DSI:in oleellisesti liittyvä Disease State Fingerprint (DSF, taudin tilan sormenjälki) -visualisointimenetelmä laajennettiin pitkittäiselle datalle. Kolmas osatutkimus ennusti hippokampuksen surkastumista 24 kuukauden ai- kana lähtötilanteen mittausten perusteella. Tutkimuskohortti koostui henkilöistä, joilla oli normaali kognitio, MCI tai Alzheimerin taudin dementia, ja se hankittiin ADNI 1 (n=530) ja Australian Imaging Biomarkers and Lifestyle Flagship Study of Ageing (AIBL, n=176) tutkimuksista. Useita eri datatyyppejä sisältävät mallit ennustivat hippokampuksen surkastumista tarkemmin kuin pelkistä MRI-muuttujista koostuvat mallit. Kuitenkin molemmat mallit aliarvioivat todellista surkastumista erityisesti suuremmilla surkastumisnopeuksilla, aliarviointi oli suurempaa pelkästään MRI-muuttujiin perustuvilla malleilla. Kun ennustettiin kaksiluokkaista vastemuuttujaa, eli nopea vs. hidas surkastuminen, mallien tarkkuus oli 79-87%. MRI-mallien suorituskyky oli hyvä, kun testauksessa käytettiin erillistä AIBL-aineistoa. Viimeinen osatutkimus keskittyi Alzheimerin taudin viimeisimpiin vaiheisiin. Siinä tutkittiin, mitkä tautiin liittyvät tekijät ovat yhteydessä kuolleisuuteen potilailla, joilla oli Alzheimerin taudin dementia. Aineisto koostui 616 henkilöstä Amsterdam Dementia Cohort -aineistosta. Iällä ja sukupuolella vakioidun Coxin suhteellisen vaaran mallin mukaan vanhempi ikä, miessukupuoli, huonommat pisteet kognitiivisessa toimintakyvyssä, ja aivojen kuoriosien ja mediaalisen ohimolohkon surkastuminen olivat yhteydessä kuolleisuuteen. Optimaalinen muuttujien yhdistelmä sisälsi iän, sukupuolen, tulokset kahdesta kognitiivisesta testistä (digit span backward, Trail Ma- king Test A), mediaalisen ohimolohkon surkastumisen ja selkäydinnestenäytteestä mitatun kohdasta 181 (treoniini) fosforyloidun tau-proteiinin määrän. Yhteenvetona todetaan, että datalähtöisillä menetelmillä voidaan ennustaa ja seu- rata Alzheimerin taudin etenemistä varhaisesta vaiheesta myöhäiseen vaiheeseen. Yhdistämällä useita eri datatyyppejä saadaan parempia tuloksia kuin käyttämällä vain yhtä datatyyppiä. Tulokset korostavat myös, että datalähtöiset menetelmät on tärkeä arvioida erillisellä aineistolla, jota ei ole käytetty menetelmien kehittämiseen. Lisäksi näiden menetelmien käyttöönotto eri ympäristöissä tai maissa saattaa vaatia potilaan tutkimusmenetelmien ja diagnoosikriteereiden harmonisointia.Alzheimer’s disease (AD), the most common form of dementia, is a slowly progressing neurodegenerative disease, which cannot be cured yet. However, certain medications and lifestyle interventions can delay progression of the disease and its symptoms, thereby positively influencing both quality of life of patients as well as cost- effectiveness of healthcare. Early diagnosis of AD is important because such interventions should be started already at an early phase of the disease to have the best effect. However, early diagnosis is challenging because pathological changes in the brain occur years before the clinical symptoms become visible. In addition, the re- search during the past years has produced information from a large number of different tests and biomarkers that can potentially contribute to diagnosis and prognosis of AD. This excessive amount of data can cause information overload for clinicians, thus hampering the clinicians’ decision making. Data-driven analysis and visualization methods may help with interpretation and utilization of large amounts of heterogeneous patient data and support the clinicians’ decision-making process. Furthermore, the methods may aid in identifying suitable patients for clinical drug trials. The aim of the work described in this thesis was to develop and validate data- driven methods for predicting and monitoring progression of Alzheimer’s disease at the different phases of the disease spectrum, starting from normal cognition and ending to death, using data from neuropsychological and cognitive tests, magnetic resonance imaging (MRI), cerebrospinal fluid samples (CSF), comorbidities, and genetics (apolipoprotein E). The thesis consists of four original studies published as international journal articles. The first study focused on the early phase of AD. A supervised machine learning method called Disease State Index (DSI) was utilized to predict who of the individuals with subjective cognitive decline (SCD) will progress to a more severe condition, i.e., mild cognitive impairment (MCI) or dementia. The study population included 647 subjects from three different memory clinic-based cohorts in Europe. When all data modalities were combined, the area under the receiver operating characteristic curve (AUC) was 0.81 and balanced accuracy was 74%. Negative predictive value was high (93%), whereas positive predictive value was low (38%). Performance of the DSI method in terms of AUC decreased by 11% when validated with an in- dependent test set. Additional analyses suggested that several differences between the cohorts may explain the decrease in the performance. The second study focused on a more advanced disease stage. The DSI method was applied to longitudinal data collected from an MCI cohort of 273 subjects obtained from the Alzheimer’s Disease and Neuroimaging (ADNI 1) study. Longitudinal profiles of the DSI values differed between the subjects progressing to dementia due to AD and subjects remaining as MCI. In addition, two subgroups were found in the group remaining as MCI: one group with stable DSI values over time and another group with increasing DSI values, suggesting the latter group may progress to dementia due to AD in the future. This study also extended the Disease State Fingerprint (DSF) data visualization method for longitudinal data. The third study predicted hippocampal atrophy over 24 months using baseline data and penalized linear regression. The cohorts consisted of subjects with normal cognition, MCI, and dementia due to AD and were obtained from the ADNI 1 (n=530) and the Australian Imaging Biomarkers and Lifestyle Flagship Study of Ageing (AIBL, n=176) studies. The models including different data modalities per- formed better than the models including only MRI features. However, both models underestimated the real change at higher atrophy rate levels, the MRI-only models showing a greater underestimation. When predicting dichotomized outcome, i.e., fast vs. slow atrophy, the models obtained a prediction accuracy of 79-87%. The MRI-only models performed well when evaluated with an independent validation cohort (AIBL). The last study focused on the latest phase of AD by identifying which disease- related determinants are associated with mortality in patients with dementia due to AD. The cohort included 616 patients from the Amsterdam Dementia Cohort. Age- and sex-adjusted Cox proportional hazards models revealed that older age, male sex, and worse scores on cognitive functioning, as well as more severe medial temporal lobe and global cortical atrophy were associated with an increased risk of mortality. An optimal combination of variables comprised age, sex, performance on digit span backward test and Trail Making Test A, medial temporal lobe atrophy, and tau phosphorylated at threonine 181 in CSF. In conclusion, data-driven methods can be used for predicting and monitoring progression of AD from the mildest stages to the more advanced stages. Combining information from several data modalities provides better prediction performance than individual data modalities alone. The results also highlight the importance of the validation of the methods with independent validation cohorts. Introduction of these methods to different environments and countries may require harmonization of patient examination methods and diagnostic criteria

Detección del trastorno específico del lenguaje en niños mediante el análisis acústico de sus voces

El síndrome específico del lenguaje, también conocido por sus siglas en inglés SLI (Specific Language Impairment), es un síndrome que se estima que afecta en torno al 7 u 8 por ciento de la población total de niños en el mundo. Este síndrome, se caracteriza por que aquel niño que lo sufre tiene dificultades en el aprendizaje del lenguaje sin tener ninguna otra deficiencia que pueda desembocar en problemas en el habla o lingüísticos. La problemática que existe en la actualidad para diagnosticar el SLI es que no se basa en medidas objetivas, sino que se diagnostica subjetivamente por parte de pediatras y pedagogos expertos en el tema. El objetivo de este trabajo es que se pueda crear un sistema basado en el aprendizaje máquina que sea capaz de determinar con la mayor probabilidad de acierto posible la existencia ono del SLI en niños mediante el análisis de sus voces. Este sistema se ha desarrollado a partir de una base de datos con audios de niños con y sin el síndrome específico del lenguaje. El sistema consta básicamente de dos etapas: extracción de características acústicas y clasificador. En la primera etapa, se extraen un conjunto de parámetros acústicos que representan las características más relevantes de la voz de cada niño. En concreto, se han utilizado los parámetros mel-cepstrales (Mel Frequency Cepstrum Coefficients, MFCC) y se han probado varias variantes, como la inclusión de la log-energía y de los parámetros delta-MFCC, los cuales son las derivadas de los parámetros MFCC y modelan su evolución temporal. La segunda etapa consiste en un clasificador binario basado en máquinas de vectores soporte (Support Vector Machine, SVM) con diferentes funciones Kernel. En cuanto a la parte experimental, se han realizado varios conjuntos de pruebas en distintas condiciones: dependencia e independencia de locutor, y audios limpios y contaminados con ruido. Para cuantificar el funcionamiento del sistema, se han utilizado las medidas de precisión, recall y F-score. El sistema ha obtenido altas prestaciones con habla limpia, tanto para el caso dependiente como independiente de locutor. Con respecto al habla ruidosa, como era de esperar, se observa una degradación del funcionamiento del sistema a bajas relaciones señal a ruido (Signal-to-Noise Ratio, SNR), especialmente para el caso independiente de locutor. No obstante, para SNRs medias y altas, se obtiene un F-score superior a 0.9 para el caso independiente de locutor y con la utilización de los parámetros MFCC y sus derivadas y el Kernel gaussiano.Specific Language Impairment (SLI) is a syndrome that is estimated to affect about 7 to 8 percent of the world's total child population. This syndrome is characterized by the fact that a child who suffers from it has difficulties in learning language without having any other impairment that could lead to problems in speech or language. The current problem in diagnosing SLI is that it is not based on objective measures, but is diagnosed subjectively by paediatricians and pedagogues who are experts in the subject. The aim of this work is to create develop a system based on machine learning techniques that is capable of determining with the greatest probability of success the existence or not of SLI in children through the analysis of their voices. This system has been developed from a database with audios of children with and without the specific language syndrome. It basically consists of two stages: extraction of acoustic characteristics and classifier. In the first stage, a set of acoustic parameters that represent the most relevant characteristics of each child's voice are extracted. Specifically, Mel Frequency Cepstrum Coefficients (MFCC) have been used and several variants have been tested, such as the inclusion of log-energy and delta-MFCC parameters, which are the derivatives of MFCCs and model their temporal evolution. The second stage consists of a binary classifier based on Support Vector Machines (SVM) with different Kernel functions. As for the experimental part, several sets of tests have been carried out under different conditions: dependence and independence of the speaker, and clean and noise-contaminated audios. In order to quantify the performance of the system, precision, recall and F-score measurements have been used. The system has obtained high performance with clean speech, both for the dependent and independent speaker cases. With respect to noisy speech, as was to be expected, a degradation of the functioning of the system at low signal-to-noise ratio (SNR) is observed, especially for the independent speaker case. However, for medium and high SNRs, an F-score higher than 0.9 is obtained for the independent speaker case and with the use of the MFCC parameters and their derivatives and the Gaussian kernel.Ingeniería de Sistemas Audiovisuale

Voice inactivity ranking for enhancement of speech on microphone arrays

Motivated by the problem of improving the performance of speech enhancement algorithms in non-stationary acoustic environments with low SNR, a framework is proposed for identifying signal frames of noisy speech that are unlikely to contain voice activity. Such voice-inactive frames can then be incorporated into an adaptation strategy to improve the performance of existing speech enhancement algorithms. This adaptive approach is applicable to single-channel as well as multi-channel algorithms for noisy speech. In both cases, the adaptive versions of the enhancement algorithms are observed to improve SNR levels by 20dB, as indicated by PESQ and WER criteria. In advanced speech enhancement algorithms, it is often of interest to identify some regions of the signal that have a high likelihood of being noise only i.e. no speech present. This is in contrast to advanced speech recognition, speaker recognition, and pitch tracking algorithms in which we are interested in identifying all regions that have a high likelihood of containing speech, as well as regions that have a high likelihood of not containing speech. In other terms, this would mean minimizing the false positive and false negative rates, respectively. In the context of speech enhancement, the identification of some speech-absent regions prompts the minimization of false positives while setting an acceptable tolerance on false negatives, as determined by the performance of the enhancement algorithm. Typically, Voice Activity Detectors (VADs) are used for identifying speech absent regions for the application of speech enhancement. In recent years a myriad of Deep Neural Network (DNN) based approaches have been proposed to improve the performance of VADs at low SNR levels by training on combinations of speech and noise. Training on such an exhaustive dataset is combinatorically explosive. For this dissertation, we propose a voice inactivity ranking framework, where the identification of voice-inactive frames is performed using a machine learning (ML) approach that only uses clean speech utterances for training and is robust to high levels of noise. In the proposed framework, input frames of noisy speech are ranked by ‘voice inactivity score’ to acquire definitely speech inactive (DSI) frame-sequences. These DSI regions serve as a noise estimate and are adaptively used by the underlying speech enhancement algorithm to enhance speech from a speech mixture. The proposed voice-inactivity ranking framework was used to perform speech enhancement in single-channel and multi-channel systems. In the context of microphone arrays, the proposed framework was used to determine parameters for spatial filtering using adaptive beamformers. We achieved an average Word Error Rate (WER) improvement of 50% at SNR levels below 0dB compared to the noisy signal, which is 7±2.5% more than the framework where state-of-the-art VAD decision was used for spatial filtering. For monaural signals, we propose a multi-frame multiband spectral-subtraction (MF-MBSS) speech enhancement system utilizing the voice inactivity framework to compute and update the noise statistics on overlapping frequency bands. The proposed MF-MBSS not only achieved an average PESQ improvement of 16% with a maximum improvement of 56% when compared to the state-of-the-art Spectral Subtraction but also a 5 ± 1.5% improvement in the Word Error Rate (WER) of the spatially filtered output signal, in non-stationary acoustic environments

1995-1996 Bulletin

Volume 106, Number 4. Scanned from the copy held in the Registrar\u27s Office.https://ecommons.udayton.edu/bulletin/1044/thumbnail.jp

Widespread dysregulation of mRNA splicing implicates RNA processing in the development and progression of Huntington's disease

Background In Huntington's disease (HD), a CAG repeat expansion mutation in the Huntingtin (HTT) gene drives a gain-of-function toxicity that disrupts mRNA processing. Although dysregulation of gene splicing has been shown in human HD post-mortem brain tissue, post-mortem analyses are likely confounded by cell type composition changes in late-stage HD, limiting the ability to identify dysregulation related to early pathogenesis. Methods To investigate gene splicing changes in early HD, we performed alternative splicing analyses coupled with a proteogenomics approach to identify early CAG length-associated splicing changes in an established isogenic HD cell model. Findings We report widespread neuronal differentiation stage- and CAG length-dependent splicing changes, and find an enrichment of RNA processing, neuronal function, and epigenetic modification-related genes with mutant HTT-associated splicing. When integrated with a proteomics dataset, we identified several of these differential splicing events at the protein level. By comparing with human post-mortem and mouse model data, we identified common patterns of altered splicing from embryonic stem cells through to post-mortem striatal tissue. Interpretation We show that widespread splicing dysregulation in HD occurs in an early cell model of neuronal development. Importantly, we observe HD-associated splicing changes in our HD cell model that were also identified in human HD striatum and mouse model HD striatum, suggesting that splicing-associated pathogenesis possibly occurs early in neuronal development and persists to later stages of disease. Together, our results highlight splicing dysregulation in HD which may lead to disrupted neuronal function and neuropathology. Funding This research is supported by the Lee Kong Chian School of Medicine, Nanyang Technological University Singapore Nanyang Assistant Professorship Start-Up Grant, the Singapore Ministry of Education under its Singapore Ministry of Education Academic Research Fund Tier 1 (RG23/22), the BC Children's Hospital Research Institute Investigator Grant Award (IGAP), and a Scholar Award from the Michael Smith Health Research BC