13,502 research outputs found
Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation
The recent advances in deep learning have made it possible to generate
photo-realistic images by using neural networks and even to extrapolate video
frames from an input video clip. In this paper, for the sake of both furthering
this exploration and our own interest in a realistic application, we study
image-to-video translation and particularly focus on the videos of facial
expressions. This problem challenges the deep neural networks by another
temporal dimension comparing to the image-to-image translation. Moreover, its
single input image fails most existing video generation methods that rely on
recurrent models. We propose a user-controllable approach so as to generate
video clips of various lengths from a single face image. The lengths and types
of the expressions are controlled by users. To this end, we design a novel
neural network architecture that can incorporate the user input into its skip
connections and propose several improvements to the adversarial training method
for the neural network. Experiments and user studies verify the effectiveness
of our approach. Especially, we would like to highlight that even for the face
images in the wild (downloaded from the Web and the authors' own photos), our
model can generate high-quality facial expression videos of which about 50\%
are labeled as real by Amazon Mechanical Turk workers.Comment: 10 page
Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
Graph networks are a new machine learning (ML) paradigm that supports both
relational reasoning and combinatorial generalization. Here, we develop
universal MatErials Graph Network (MEGNet) models for accurate property
prediction in both molecules and crystals. We demonstrate that the MEGNet
models outperform prior ML models such as the SchNet in 11 out of 13 properties
of the QM9 molecule data set. Similarly, we show that MEGNet models trained on
crystals in the Materials Project substantially outperform prior
ML models in the prediction of the formation energies, band gaps and elastic
moduli of crystals, achieving better than DFT accuracy over a much larger data
set. We present two new strategies to address data limitations common in
materials science and chemistry. First, we demonstrate a physically-intuitive
approach to unify four separate molecular MEGNet models for the internal energy
at 0 K and room temperature, enthalpy and Gibbs free energy into a single free
energy MEGNet model by incorporating the temperature, pressure and entropy as
global state inputs. Second, we show that the learned element embeddings in
MEGNet models encode periodic chemical trends and can be transfer-learned from
a property model trained on a larger data set (formation energies) to improve
property models with smaller amounts of data (band gaps and elastic moduli)
- …