424 research outputs found
Adversarial Data Programming: Using GANs to Relax the Bottleneck of Curated Labeled Data
Paucity of large curated hand-labeled training data for every
domain-of-interest forms a major bottleneck in the deployment of machine
learning models in computer vision and other fields. Recent work (Data
Programming) has shown how distant supervision signals in the form of labeling
functions can be used to obtain labels for given data in near-constant time. In
this work, we present Adversarial Data Programming (ADP), which presents an
adversarial methodology to generate data as well as a curated aggregated label
has given a set of weak labeling functions. We validated our method on the
MNIST, Fashion MNIST, CIFAR 10 and SVHN datasets, and it outperformed many
state-of-the-art models. We conducted extensive experiments to study its
usefulness, as well as showed how the proposed ADP framework can be used for
transfer learning as well as multi-task learning, where data from two domains
are generated simultaneously using the framework along with the label
information. Our future work will involve understanding the theoretical
implications of this new framework from a game-theoretic perspective, as well
as explore the performance of the method on more complex datasets.Comment: CVPR 2018 main conference pape
DualLip: A System for Joint Lip Reading and Generation
Lip reading aims to recognize text from talking lip, while lip generation
aims to synthesize talking lip according to text, which is a key component in
talking face generation and is a dual task of lip reading. In this paper, we
develop DualLip, a system that jointly improves lip reading and generation by
leveraging the task duality and using unlabeled text and lip video data. The
key ideas of the DualLip include: 1) Generate lip video from unlabeled text
with a lip generation model, and use the pseudo pairs to improve lip reading;
2) Generate text from unlabeled lip video with a lip reading model, and use the
pseudo pairs to improve lip generation. We further extend DualLip to talking
face generation with two additionally introduced components: lip to face
generation and text to speech generation. Experiments on GRID and TCD-TIMIT
demonstrate the effectiveness of DualLip on improving lip reading, lip
generation, and talking face generation by utilizing unlabeled data.
Specifically, the lip generation model in our DualLip system trained with
only10% paired data surpasses the performance of that trained with the whole
paired data. And on the GRID benchmark of lip reading, we achieve 1.16%
character error rate and 2.71% word error rate, outperforming the
state-of-the-art models using the same amount of paired data.Comment: Accepted by ACM Multimedia 202
- …