3 research outputs found
Very Deep Convolutional Neural Networks for Robust Speech Recognition
This paper describes the extension and optimization of our previous work on
very deep convolutional neural networks (CNNs) for effective recognition of
noisy speech in the Aurora 4 task. The appropriate number of convolutional
layers, the sizes of the filters, pooling operations and input feature maps are
all modified: the filter and pooling sizes are reduced and dimensions of input
feature maps are extended to allow adding more convolutional layers.
Furthermore appropriate input padding and input feature map selection
strategies are developed. In addition, an adaptation framework using joint
training of very deep CNN with auxiliary features i-vector and fMLLR features
is developed. These modifications give substantial word error rate reductions
over the standard CNN used as baseline. Finally the very deep CNN is combined
with an LSTM-RNN acoustic model and it is shown that state-level weighted log
likelihood score combination in a joint acoustic model decoding scheme is very
effective. On the Aurora 4 task, the very deep CNN achieves a WER of 8.81%,
further 7.99% with auxiliary feature joint training, and 7.09% with LSTM-RNN
joint decoding.Comment: accepted by SLT 201
Deep Order Statistic Networks
Recently, Maxout networks have demonstrated state-of-the-art per-formance on several machne learning tasks, which has fueled ag-gressive research on Maxout networks and generalizations thereof. In this work, we propose the utilization of order statistics as a gen-eralization of the max non-linearity. A particularly general exam-ple of an order-statistic non-linearity is the ”sortout ” non-linearity, which outputs all input activations, but in sorted order. Such Order-statistic networks (OSNs), in contrast with other recently proposed generalizations of Maxout networks, leave the determination of the interpolation weights on the activations to the network, and remain conditionally linear given the input, and so are well suited for power-ful model aggregation techniques such as dropout, drop connect, and annealed dropout. Experimental results demonstrate that the use of order statistics rather than Maxout networks can lead to substantial improvements in the word error rate (WER) performance of auto-matic speech recognition systems