4,455 research outputs found
MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Face Images
This paper is aimed at creating extremely small and fast convolutional neural
networks (CNN) for the problem of facial expression recognition (FER) from
frontal face images. To this end, we employed the popular knowledge
distillation (KD) method and identified two major shortcomings with its use: 1)
a fine-grained grid search is needed for tuning the temperature hyperparameter
and 2) to find the optimal size-accuracy balance, one needs to search for the
final network size (or the compression rate). On the other hand, KD is proved
to be useful for model compression for the FER problem, and we discovered that
its effects gets more and more significant with the decreasing model size. In
addition, we hypothesized that translation invariance achieved using
max-pooling layers would not be useful for the FER problem as the expressions
are sensitive to small, pixel-wise changes around the eye and the mouth.
However, we have found an intriguing improvement on generalization when
max-pooling is used. We conducted experiments on two widely-used FER datasets,
CK+ and Oulu-CASIA. Our smallest model (MicroExpNet), obtained using knowledge
distillation, is less than 1MB in size and works at 1851 frames per second on
an Intel i7 CPU. Despite being less accurate than the state-of-the-art,
MicroExpNet still provides significant insights for designing a
microarchitecture for the FER problem.Comment: International Conference on Image Processing Theory, Tools and
Applications (IPTA) 2019 camera ready version. Codes are available at:
https://github.com/cuguilke/microexpne
Learning to Parse and Translate Improves Neural Machine Translation
There has been relatively little attention to incorporating linguistic prior
to neural machine translation. Much of the previous work was further
constrained to considering linguistic prior on the source side. In this paper,
we propose a hybrid model, called NMT+RNNG, that learns to parse and translate
by combining the recurrent neural network grammar into the attention-based
neural machine translation. Our approach encourages the neural machine
translation model to incorporate linguistic prior during training, and lets it
translate on its own afterward. Extensive experiments with four language pairs
show the effectiveness of the proposed NMT+RNNG.Comment: Accepted as a short paper at the 55th Annual Meeting of the
Association for Computational Linguistics (ACL 2017
Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
The key challenge of multi-domain translation lies in simultaneously encoding
both the general knowledge shared across domains and the particular knowledge
distinctive to each domain in a unified model. Previous work shows that the
standard neural machine translation (NMT) model, trained on mixed-domain data,
generally captures the general knowledge, but misses the domain-specific
knowledge. In response to this problem, we augment NMT model with additional
domain transformation networks to transform the general representations to
domain-specific representations, which are subsequently fed to the NMT decoder.
To guarantee the knowledge transformation, we also propose two complementary
supervision signals by leveraging the power of knowledge distillation and
adversarial learning. Experimental results on several language pairs, covering
both balanced and unbalanced multi-domain translation, demonstrate the
effectiveness and universality of the proposed approach. Encouragingly, the
proposed unified model achieves comparable results with the fine-tuning
approach that requires multiple models to preserve the particular knowledge.
Further analyses reveal that the domain transformation networks successfully
capture the domain-specific knowledge as expected.Comment: AAAI 202
- …