13,965 research outputs found
Towards Ultrasound Tongue Image prediction from EEG during speech production
Previous initial research has already been carried out to propose
speech-based BCI using brain signals (e.g.~non-invasive EEG and invasive sEEG /
ECoG), but there is a lack of combined methods that investigate non-invasive
brain, articulation, and speech signals together and analyze the cognitive
processes in the brain, the kinematics of the articulatory movement and the
resulting speech signal. In this paper, we describe our multimodal
(electroencephalography, ultrasound tongue imaging, and speech) analysis and
synthesis experiments, as a feasibility study. We extend the analysis of brain
signals recorded during speech production with ultrasound-based articulation
data. From the brain signal measured with EEG, we predict ultrasound images of
the tongue with a fully connected deep neural network. The results show that
there is a weak but noticeable relationship between EEG and ultrasound tongue
images, i.e. the network can differentiate articulated speech and neutral
tongue position.Comment: accepted at Interspeech 202
Hierarchical Deep Feature Learning For Decoding Imagined Speech From EEG
We propose a mixed deep neural network strategy, incorporating parallel
combination of Convolutional (CNN) and Recurrent Neural Networks (RNN),
cascaded with deep autoencoders and fully connected layers towards automatic
identification of imagined speech from EEG. Instead of utilizing raw EEG
channel data, we compute the joint variability of the channels in the form of a
covariance matrix that provide spatio-temporal representations of EEG. The
networks are trained hierarchically and the extracted features are passed onto
the next network hierarchy until the final classification. Using a publicly
available EEG based speech imagery database we demonstrate around 23.45%
improvement of accuracy over the baseline method. Our approach demonstrates the
promise of a mixed DNN approach for complex spatial-temporal classification
problems.Comment: Accepted in AAAI 2019 under Student Abstract and Poster Progra
- …