36,901 research outputs found
Metric Learning for Temporal Sequence Alignment
In this paper, we propose to learn a Mahalanobis distance to perform
alignment of multivariate time series. The learning examples for this task are
time series for which the true alignment is known. We cast the alignment
problem as a structured prediction task, and propose realistic losses between
alignments for which the optimization is tractable. We provide experiments on
real data in the audio to audio context, where we show that the learning of a
similarity measure leads to improvements in the performance of the alignment
task. We also propose to use this metric learning framework to perform feature
selection and, from basic audio features, build a combination of these with
better performance for the alignment
Improving End-to-End Speech Recognition with Policy Learning
Connectionist temporal classification (CTC) is widely used for maximum
likelihood learning in end-to-end speech recognition models. However, there is
usually a disparity between the negative maximum likelihood and the performance
metric used in speech recognition, e.g., word error rate (WER). This results in
a mismatch between the objective function and metric during training. We show
that the above problem can be mitigated by jointly training with maximum
likelihood and policy gradient. In particular, with policy learning we are able
to directly optimize on the (otherwise non-differentiable) performance metric.
We show that joint training improves relative performance by 4% to 13% for our
end-to-end model as compared to the same model learned through maximum
likelihood. The model achieves 5.53% WER on Wall Street Journal dataset, and
5.42% and 14.70% on Librispeech test-clean and test-other set, respectively
Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
In this work, we address the task of weakly-supervised human action
segmentation in long, untrimmed videos. Recent methods have relied on expensive
learning models, such as Recurrent Neural Networks (RNN) and Hidden Markov
Models (HMM). However, these methods suffer from expensive computational cost,
thus are unable to be deployed in large scale. To overcome the limitations, the
keys to our design are efficiency and scalability. We propose a novel action
modeling framework, which consists of a new temporal convolutional network,
named Temporal Convolutional Feature Pyramid Network (TCFPN), for predicting
frame-wise action labels, and a novel training strategy for weakly-supervised
sequence modeling, named Iterative Soft Boundary Assignment (ISBA), to align
action sequences and update the network in an iterative fashion. The proposed
framework is evaluated on two benchmark datasets, Breakfast and Hollywood
Extended, with four different evaluation metrics. Extensive experimental
results show that our methods achieve competitive or superior performance to
state-of-the-art methods.Comment: CVPR 201
- …