1,678 research outputs found
Exploring efficient neural architectures for linguistic-acoustic mapping in text-to-speech
Conversion from text to speech relies on the accurate mapping from linguistic to acoustic symbol sequences, for which current practice employs recurrent statistical models such as recurrent neural networks. Despite the good performance of such models (in terms of low distortion in the generated speech), their recursive structure with intermediate affine transformations tends to make them slow to train and to sample from. In this work, we explore two different mechanisms that enhance the operational efficiency of recurrent neural networks, and study their performance–speed trade-off. The first mechanism is based on the quasi-recurrent neural network, where expensive affine transformations are removed from temporal connections and placed only on feed-forward computational directions. The second mechanism includes a module based on the transformer decoder network, designed without recurrent connections but emulating them with attention and positioning codes. Our results show that the proposed decoder networks are competitive in terms of distortion when compared to a recurrent baseline, whilst being significantly faster in terms of CPU and GPU inference time. The best performing model is the one based on the quasi-recurrent mechanism, reaching the same level of naturalness as the recurrent neural network based model with a speedup of 11.2 on CPU and 3.3 on GPU.Peer ReviewedPostprint (published version
Modeling neural dynamics during speech production using a state space variational autoencoder
Characterizing the neural encoding of behavior remains a challenging task in
many research areas due in part to complex and noisy spatiotemporal dynamics of
evoked brain activity. An important aspect of modeling these neural encodings
involves separation of robust, behaviorally relevant signals from background
activity, which often contains signals from irrelevant brain processes and
decaying information from previous behavioral events. To achieve this
separation, we develop a two-branch State Space Variational AutoEncoder (SSVAE)
model to individually describe the instantaneous evoked foreground signals and
the context-dependent background signals. We modeled the spontaneous
speech-evoked brain dynamics using smoothed Gaussian mixture models. By
applying the proposed SSVAE model to track ECoG dynamics in one participant
over multiple hours, we find that the model can predict speech-related dynamics
more accurately than other latent factor inference algorithms. Our results
demonstrate that separately modeling the instantaneous speech-evoked and slow
context-dependent brain dynamics can enhance tracking performance, which has
important implications for the development of advanced neural encoding and
decoding models in various neuroscience sub-disciplines.Comment: 5 page
Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes
Recurrent neural networks are nowadays successfully used in an abundance of
applications, going from text, speech and image processing to recommender
systems. Backpropagation through time is the algorithm that is commonly used to
train these networks on specific tasks. Many deep learning frameworks have
their own implementation of training and sampling procedures for recurrent
neural networks, while there are in fact multiple other possibilities to choose
from and other parameters to tune. In existing literature this is very often
overlooked or ignored. In this paper we therefore give an overview of possible
training and sampling schemes for character-level recurrent neural networks to
solve the task of predicting the next token in a given sequence. We test these
different schemes on a variety of datasets, neural network architectures and
parameter settings, and formulate a number of take-home recommendations. The
choice of training and sampling scheme turns out to be subject to a number of
trade-offs, such as training stability, sampling time, model performance and
implementation effort, but is largely independent of the data. Perhaps the most
surprising result is that transferring hidden states for correctly initializing
the model on subsequences often leads to unstable training behavior depending
on the dataset.Comment: 23 pages, 11 figures, 4 table
A Systematic Survey on Deep Generative Models for Graph Generation
Graphs are important data representations for describing objects and their
relationships, which appear in a wide diversity of real-world scenarios. As one
of a critical problem in this area, graph generation considers learning the
distributions of given graphs and generating more novel graphs. Owing to its
wide range of applications, generative models for graphs have a rich history,
which, however, are traditionally hand-crafted and only capable of modeling a
few statistical properties of graphs. Recent advances in deep generative models
for graph generation is an important step towards improving the fidelity of
generated graphs and paves the way for new kinds of applications. This article
provides an extensive overview of the literature in the field of deep
generative models for the graph generation. Firstly, the formal definition of
deep generative models for the graph generation as well as preliminary
knowledge is provided. Secondly, two taxonomies of deep generative models for
unconditional, and conditional graph generation respectively are proposed; the
existing works of each are compared and analyzed. After that, an overview of
the evaluation metrics in this specific domain is provided. Finally, the
applications that deep graph generation enables are summarized and five
promising future research directions are highlighted
- …