Image-to-Image Translation with Conditional Adversarial Networks

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.Comment: Website: https://phillipi.github.io/pix2pix/, CVPR 201

Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation

The recent advances in deep learning have made it possible to generate photo-realistic images by using neural networks and even to extrapolate video frames from an input video clip. In this paper, for the sake of both furthering this exploration and our own interest in a realistic application, we study image-to-video translation and particularly focus on the videos of facial expressions. This problem challenges the deep neural networks by another temporal dimension comparing to the image-to-image translation. Moreover, its single input image fails most existing video generation methods that rely on recurrent models. We propose a user-controllable approach so as to generate video clips of various lengths from a single face image. The lengths and types of the expressions are controlled by users. To this end, we design a novel neural network architecture that can incorporate the user input into its skip connections and propose several improvements to the adversarial training method for the neural network. Experiments and user studies verify the effectiveness of our approach. Especially, we would like to highlight that even for the face images in the wild (downloaded from the Web and the authors' own photos), our model can generate high-quality facial expression videos of which about 50\% are labeled as real by Amazon Mechanical Turk workers.Comment: 10 page

A deep-learning approach for high-speed Fourier ptychographic microscopy

We demonstrate a new convolutional neural network architecture to perform Fourier ptychographic Microscopy (FPM) reconstruction, which achieves high-resolution phase recovery with considerably less data than standard FPM.https://www.researchgate.net/profile/Thanh_Nguyen68/publication/325829575_A_deep-learning_approach_for_high-speed_Fourier_ptychographic_microscopy/links/5b2beec20f7e9b0df5ba4872/A-deep-learning-approach-for-high-speed-Fourier-ptychographic-microscopy.pdfhttps://www.researchgate.net/profile/Thanh_Nguyen68/publication/325829575_A_deep-learning_approach_for_high-speed_Fourier_ptychographic_microscopy/links/5b2beec20f7e9b0df5ba4872/A-deep-learning-approach-for-high-speed-Fourier-ptychographic-microscopy.pdfhttps://www.researchgate.net/profile/Thanh_Nguyen68/publication/325829575_A_deep-learning_approach_for_high-speed_Fourier_ptychographic_microscopy/links/5b2beec20f7e9b0df5ba4872/A-deep-learning-approach-for-high-speed-Fourier-ptychographic-microscopy.pdfhttps://www.researchgate.net/profile/Thanh_Nguyen68/publication/325829575_A_deep-learning_approach_for_high-speed_Fourier_ptychographic_microscopy/links/5b2beec20f7e9b0df5ba4872/A-deep-learning-approach-for-high-speed-Fourier-ptychographic-microscopy.pdfhttps://www.researchgate.net/profile/Thanh_Nguyen68/publication/325829575_A_deep-learning_approach_for_high-speed_Fourier_ptychographic_microscopy/links/5b2beec20f7e9b0df5ba4872/A-deep-learning-approach-for-high-speed-Fourier-ptychographic-microscopy.pdfhttps://www.researchgate.net/profile/Thanh_Nguyen68/publication/325829575_A_deep-learning_approach_for_high-speed_Fourier_ptychographic_microscopy/links/5b2beec20f7e9b0df5ba4872/A-deep-learning-approach-for-high-speed-Fourier-ptychographic-microscopy.pdfPublished versio

Building Footprint Generation Using Improved Generative Adversarial Networks

Building footprint information is an essential ingredient for 3-D reconstruction of urban models. The automatic generation of building footprints from satellite images presents a considerable challenge due to the complexity of building shapes. In this work, we have proposed improved generative adversarial networks (GANs) for the automatic generation of building footprints from satellite images. We used a conditional GAN with a cost function derived from the Wasserstein distance and added a gradient penalty term. The achieved results indicated that the proposed method can significantly improve the quality of building footprint generation compared to conditional generative adversarial networks, the U-Net, and other networks. In addition, our method nearly removes all hyperparameters tuning.Comment: 5 page