20 research outputs found
Parallel implementation of Artificial Neural Network training for speech recognition
In this paper we describe the implementation of a complete ANN training procedure using the block
mode back-propagation learning algorithm for sequential patterns – such as the observation feature vectors
of a speech recognition system – exploiting the high performance SIMD architecture of GPU using
CUDA and its C-like language interface. We also compare the speed-up obtained implementing the training
procedure only taking advantage of the multi-thread capabilities of multi-core processors. In our
implementation we take into account all the peculiar aspects of training large scale sequential patterns,
in particular, the re-segmentation of the training sentences, the block size for the feed-forward and for
the back-propagation steps, and the transfer of huge amount of data from host memory to the GPU card.
Our approach has been tested by training acoustic models for large vocabulary speech recognition
tasks, showing a six times reduction of the time required to train real-world large size networks with
respect to an already optimized implementation using the Intel MKL libraries.
Thanks to these optimizations and to the support of the GPU, the training time for language having a
huge set of training sentences (about one million for Italian) can be reduced from approximately a month
to 5 days