71 research outputs found
Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition
Automatic recognition of disordered speech remains a highly challenging task
to date. The underlying neuro-motor conditions, often compounded with
co-occurring physical disabilities, lead to the difficulty in collecting large
quantities of impaired speech required for ASR system development. This paper
presents novel variational auto-encoder generative adversarial network
(VAE-GAN) based personalized disordered speech augmentation approaches that
simultaneously learn to encode, generate and discriminate synthesized impaired
speech. Separate latent features are derived to learn dysarthric speech
characteristics and phoneme context representations. Self-supervised
pre-trained Wav2vec 2.0 embedding features are also incorporated. Experiments
conducted on the UASpeech corpus suggest the proposed adversarial data
augmentation approach consistently outperformed the baseline speed perturbation
and non-VAE GAN augmentation methods with trained hybrid TDNN and End-to-end
Conformer systems. After LHUC speaker adaptation, the best system using VAE-GAN
based augmentation produced an overall WER of 27.78% on the UASpeech test set
of 16 dysarthric speakers, and the lowest published WER of 57.31% on the subset
of speakers with "Very Low" intelligibility.Comment: Submitted to ICASSP 202
Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation
Automatic recognition of dysarthric speech remains a highly challenging task
to date. Neuro-motor conditions and co-occurring physical disabilities create
difficulty in large-scale data collection for ASR system development. Adapting
SSL pre-trained ASR models to limited dysarthric speech via data-intensive
parameter fine-tuning leads to poor generalization. To this end, this paper
presents an extensive comparative study of various data augmentation approaches
to improve the robustness of pre-trained ASR model fine-tuning to dysarthric
speech. These include: a) conventional speaker-independent perturbation of
impaired speech; b) speaker-dependent speed perturbation, or GAN-based
adversarial perturbation of normal, control speech based on their time
alignment against parallel dysarthric speech; c) novel Spectral basis GAN-based
adversarial data augmentation operating on non-parallel data. Experiments
conducted on the UASpeech corpus suggest GAN-based data augmentation
consistently outperforms fine-tuned Wav2vec2.0 and HuBERT models using no data
augmentation and speed perturbation across different data expansion operating
points by statistically significant word error rate (WER) reductions up to
2.01% and 0.96% absolute (9.03% and 4.63% relative) respectively on the
UASpeech test set of 16 dysarthric speakers. After cross-system outputs
rescoring, the best system produced the lowest published WER of 16.53% (46.47%
on very low intelligibility) on UASpeech.Comment: To appear at IEEE ICASSP 202
Functional specialization and interaction in the amygdala-hippocampus circuit during working memory processing
Both the hippocampus and amygdala are involved in working memory (WM) processing. However, their specific role in WM is still an open question. Here, we simultaneously recorded intracranial EEG from the amygdala and hippocampus of epilepsy patients while performing a WM task, and compared their representation patterns during the encoding and maintenance periods. By combining multivariate representational analysis and connectivity analyses with machine learning methods, our results revealed a functional specialization of the amygdala-hippocampal circuit: The mnemonic representations in the amygdala were highly distinct and decreased from encoding to maintenance. The hippocampal representations, however, were more similar across different items but remained stable in the absence of the stimulus. WM encoding and maintenance were associated with bidirectional information flow between the amygdala and the hippocampus in low-frequency bands (1-40 Hz). Furthermore, the decoding accuracy on WM load was higher by using representational features in the amygdala during encoding and in the hippocampus during maintenance, and by using information flow from the amygdala during encoding and that from the hippocampus during maintenance, respectively. Taken together, our study reveals that WM processing is associated with functional specialization and interaction within the amygdala-hippocampus circuit
Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition
Automatic recognition of disordered and elderly speech remains highly
challenging tasks to date due to data scarcity. Parameter fine-tuning is often
used to exploit the large quantities of non-aged and healthy speech pre-trained
models, while neural architecture hyper-parameters are set using expert
knowledge and remain unchanged. This paper investigates hyper-parameter
adaptation for Conformer ASR systems that are pre-trained on the Librispeech
corpus before being domain adapted to the DementiaBank elderly and UASpeech
dysarthric speech datasets. Experimental results suggest that hyper-parameter
adaptation produced word error rate (WER) reductions of 0.45% and 0.67% over
parameter-only fine-tuning on DBank and UASpeech tasks respectively. An
intuitive correlation is found between the performance improvements by
hyper-parameter domain adaptation and the relative utterance length ratio
between the source and target domain data.Comment: 5 pages, 3 figures, 3 tables, accepted by Interspeech202
Confidence Score Based Speaker Adaptation of Conformer Speech Recognition Systems
Speaker adaptation techniques provide a powerful solution to customise
automatic speech recognition (ASR) systems for individual users. Practical
application of unsupervised model-based speaker adaptation techniques to data
intensive end-to-end ASR systems is hindered by the scarcity of speaker-level
data and performance sensitivity to transcription errors. To address these
issues, a set of compact and data efficient speaker-dependent (SD) parameter
representations are used to facilitate both speaker adaptive training and
test-time unsupervised speaker adaptation of state-of-the-art Conformer ASR
systems. The sensitivity to supervision quality is reduced using a confidence
score-based selection of the less erroneous subset of speaker-level adaptation
data. Two lightweight confidence score estimation modules are proposed to
produce more reliable confidence scores. The data sparsity issue, which is
exacerbated by data selection, is addressed by modelling the SD parameter
uncertainty using Bayesian learning. Experiments on the benchmark 300-hour
Switchboard and the 233-hour AMI datasets suggest that the proposed confidence
score-based adaptation schemes consistently outperformed the baseline
speaker-independent (SI) Conformer model and conventional non-Bayesian, point
estimate-based adaptation using no speaker data selection. Similar consistent
performance improvements were retained after external Transformer and LSTM
language model rescoring. In particular, on the 300-hour Switchboard corpus,
statistically significant WER reductions of 1.0%, 1.3%, and 1.4% absolute
(9.5%, 10.9%, and 11.3% relative) were obtained over the baseline SI Conformer
on the NIST Hub5'00, RT02, and RT03 evaluation sets respectively. Similar WER
reductions of 2.7% and 3.3% absolute (8.9% and 10.2% relative) were also
obtained on the AMI development and evaluation sets.Comment: IEEE/ACM Transactions on Audio, Speech, and Language Processin
Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition
Accurate recognition of cocktail party speech containing overlapping
speakers, noise and reverberation remains a highly challenging task to date.
Motivated by the invariance of visual modality to acoustic signal corruption,
an audio-visual multi-channel speech separation, dereverberation and
recognition approach featuring a full incorporation of visual information into
all system components is proposed in this paper. The efficacy of the video
input is consistently demonstrated in mask-based MVDR speech separation,
DNN-WPE or spectral mapping (SpecM) based speech dereverberation front-end and
Conformer ASR back-end. Audio-visual integrated front-end architectures
performing speech separation and dereverberation in a pipelined or joint
fashion via mask-based WPD are investigated. The error cost mismatch between
the speech enhancement front-end and ASR back-end components is minimized by
end-to-end jointly fine-tuning using either the ASR cost function alone, or its
interpolation with the speech enhancement loss. Experiments were conducted on
the mixture overlapped and reverberant speech data constructed using simulation
or replay of the Oxford LRS2 dataset. The proposed audio-visual multi-channel
speech separation, dereverberation and recognition systems consistently
outperformed the comparable audio-only baseline by 9.1% and 6.2% absolute
(41.7% and 36.0% relative) word error rate (WER) reductions. Consistent speech
enhancement improvements were also obtained on PESQ, STOI and SRMR scores.Comment: IEEE/ACM Transactions on Audio, Speech, and Language Processin
The cortical surface area of the insula mediates the effect of DBH rs7040170 on novelty seeking
Novelty seeking (NS) is a personality trait important for adaptive functioning, but an excessive level of NS has been linked to psychiatric disorders such as ADHD and substance abuse. Previous research has investigated separately the neural and genetic bases of the NS trait, but results were mixed and neural and genetic bases have yet to be examined within the same study. In this study, we examined the interrelationships among the dopamine beta-hydroxylase (DBH) gene, brain structure, and the NS trait in 359 healthy Han Chinese subjects. We focused on the DBH gene because it encodes a key enzyme for dopamine metabolism, NS is believed to be related to the dopaminergic system and has been reported associated with DBH variation. Results showed a significant positive association between the cortical surface area of the left insula and NS score. Furthermore, the DBH genetic polymorphism at the SNP rs7040170 was strongly associated with both the surface area of the left insula and NS score, with G carriers having a larger left insula surface area and a higher NS score than AA homozygotes. Subsequent path analysis suggested that the insula partially mediated the association between the DBH gene and the NS trait. Our data provided the first evidence for the involvement of the insula in the dopamine-NS relationship. Future studies of molecular mechanisms underlying the NS personality trait and related psychiatric disorders should consider the mediation effect of the neural structure
Polygenic risk for Alzheimer's disease influences precuneal volume in two independent general populations
Alzheimer's disease (AD) is heritable with complex genetic underpinnings. Based on previous results from large-scale genome-wide association studies, recent studies found an association between the polygenic risk score (PGRS) of AD and the structure of some preselected brain regions, but the effects of AD PGRS on all voxels of the brain have not been fully investigated. In the present study, we examined the voxel-wise effect of AD PGRS on the entire brain and the influence of AD PGRS on cognitive function in 2 independent healthy young cohorts. In both cohorts, an elevated AD PGRS was associated with a smaller precuneal volume, and the effect remained after excluding the APOE genotype. No correlation was found between AD PGRS and any cognitive measure in either sample. Finding a negative correlation between the AD PGRS and the precuneal volume could help to elucidate the mechanism of the genetic risk for AD and could provide a potential biomarker for early detection and possible interventions in AD
Stable Modality-Specific Activity Flows As Reflected by the Neuroenergetic Approach to the fMRI Weighted Maps
This article uses the ideas of neuroenergetic and neural field theories to detect stimulation-driven energy flows in the brain during face and auditory word processing. In this analysis, energy flows are thought to create the stable gradients of the fMRI weighted summary images. The sources, from which activity spreads in the brain during face processing, were detected in the occipital cortex. The following direction of energy flows in the frontal cortex was described: the right inferior frontal = >the left inferior frontal = >the triangular part of the left inferior frontal cortex = >the left operculum. In the left operculum, a localized circuit was described. For auditory word processing, the sources of activity flows were detected bilaterally in the middle superior temporal regions, they were also detected in the left posterior superior temporal cortex. Thus, neuroenergetic assumptions may give a novel perspective for the analysis of neuroimaging data
Case report: Reversible splenial lesion syndrome caused by diquat poisoning
Diquat (DQ), chemically known as 1,1 ‘-ethylene-2,2’ -bipyridine, is a non-selective herbicide for leaf removal and drying. It has toxic effects on central nervous system cells, and toxic neurological lesions include axonal degeneration and pontine myelolysis. At the same time, DQ can also affect the activity of dopaminergic nerve cells through oxidative stress, causing degeneration and reducing dopamine uptake. With the increasing application of DQ in agricultural production, the clinical reports of neurotoxicity caused by acute DQ poisoning are also increasing. At present, DQ rapid-phase-related toxic encephalopathy mainly involves the pons, midbrain, basal ganglia, thalamus and other brain regions. However, this case is unusual in that the lesion mainly involved the splenium of the corpus callosum. It is also the first time to be reported
- …