Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval

Free-hand sketch-based image retrieval (SBIR) is a specific cross-view retrieval task, in which queries are abstract and ambiguous sketches while the retrieval database is formed with natural images. Work in this area mainly focuses on extracting representative and shared features for sketches and natural images. However, these can neither cope well with the geometric distortion between sketches and images nor be feasible for large-scale SBIR due to the heavy continuous-valued distance computation. In this paper, we speed up SBIR by introducing a novel binary coding method, named \textbf{Deep Sketch Hashing} (DSH), where a semi-heterogeneous deep architecture is proposed and incorporated into an end-to-end binary coding framework. Specifically, three convolutional neural networks are utilized to encode free-hand sketches, natural images and, especially, the auxiliary sketch-tokens which are adopted as bridges to mitigate the sketch-image geometric distortion. The learned DSH codes can effectively capture the cross-view similarities as well as the intrinsic semantic correlations between different categories. To the best of our knowledge, DSH is the first hashing work specifically designed for category-level SBIR with an end-to-end deep architecture. The proposed DSH is comprehensively evaluated on two large-scale datasets of TU-Berlin Extension and Sketchy, and the experiments consistently show DSH's superior SBIR accuracies over several state-of-the-art methods, while achieving significantly reduced retrieval time and memory footprint.Comment: This paper will appear as a spotlight paper in CVPR201

Learning Residual Images for Face Attribute Manipulation

Face attributes are interesting due to their detailed description of human faces. Unlike prior researches working on attribute prediction, we address an inverse and more challenging problem called face attribute manipulation which aims at modifying a face image according to a given attribute value. Instead of manipulating the whole image, we propose to learn the corresponding residual image defined as the difference between images before and after the manipulation. In this way, the manipulation can be operated efficiently with modest pixel modification. The framework of our approach is based on the Generative Adversarial Network. It consists of two image transformation networks and a discriminative network. The transformation networks are responsible for the attribute manipulation and its dual operation and the discriminative network is used to distinguish the generated images from real images. We also apply dual learning to allow transformation networks to learn from each other. Experiments show that residual images can be effectively learned and used for attribute manipulations. The generated images remain most of the details in attribute-irrelevant areas

Observational Study Of the Quasi-Periodic Fast Propagating Magnetosonic Waves and the Associated Flare on 2011 May 30

On 2011 May 30, quasi-periodic fast propagating (QFP) magnetosonic waves accompanied by a C2.8 flare were directly imaged by the Atomospheric Imaging Assembly instrument on board the Solar Dynamics Observatory. The QFP waves successively emanated from the flare kernel, they propagated along a cluster of open coronal loops with a phase speed of 834 km/s during the flare's rising phase, and the multiple arc-shaped wave trains can be fitted with a series of concentric circles. We generate the k-omega diagram of the Fourier power and find a straight ridge that represents the dispersion relation of the waves. Along the ridge, we find a lot of prominent nodes which represent the available frequencies of the QFP waves. On the other hand, the frequencies of the flare are also obtained by analyzing the flare light curves using the wavelet technique. The results indicate that almost all the main frequencies of the flare are consistent with those of the QFP waves. This suggests that the flare and the QFP waves were possibly excited by a common physical origin. On the other hand, a few low frequencies revealed by the k-omega diagram can not be found in the accompanying flare. We propose that these low frequencies were possibly due to the leakage of the pressure-driven p-mode oscillations from the photosphere into the low corona, which should be a noticeable mechanism for driving the QFP waves observed in the corona.Comment: Published in Ap