Automatic Generation of Grounded Visual Questions

In this paper, we propose the first model to be able to generate visually grounded questions with diverse types for a single image. Visual question generation is an emerging topic which aims to ask questions in natural language based on visual input. To the best of our knowledge, it lacks automatic methods to generate meaningful questions with various types for the same visual input. To circumvent the problem, we propose a model that automatically generates visually grounded questions with varying types. Our model takes as input both images and the captions generated by a dense caption model, samples the most probable question types, and generates the questions in sequel. The experimental results on two real world datasets show that our model outperforms the strongest baseline in terms of both correctness and diversity with a wide margin.Comment: VQ

The evolution of magnetic structure driven by a synthetic spin-orbit coupling in two-component Bose-Hubbard model

We study the evolution of magnetic structure driven by a synthetic spin-orbit coupling in a one-dimensional two-component Bose-Hubbard model. In addition to the Mott insulator-superfluid transition, we found in Mott insulator phases a transition from a gapped ferromagnetic phase to a gapless chiral phase by increasing the strength of spin-orbit coupling. Further increasing the spin-orbit coupling drives a transition from the gapless chiral phase to a gapped antiferromagnetic phase. These magnetic structures persist in superfluid phases. In particular, in the chiral Mott insulator and chiral superfluid phases, incommensurability is observed in characteristic correlation functions. These unconventional Mott insulator phase and superfluid phase demonstrate the novel effects arising from the competition between the kinetic energy and the spin-orbit coupling.Comment: 9 fig; English polished, note adde

Balanced Sparsity for Efficient DNN Inference on GPU

In trained deep neural networks, unstructured pruning can reduce redundant weights to lower storage cost. However, it requires the customization of hardwares to speed up practical inference. Another trend accelerates sparse model inference on general-purpose hardwares by adopting coarse-grained sparsity to prune or regularize consecutive weights for efficient computation. But this method often sacrifices model accuracy. In this paper, we propose a novel fine-grained sparsity approach, balanced sparsity, to achieve high model accuracy with commercial hardwares efficiently. Our approach adapts to high parallelism property of GPU, showing incredible potential for sparsity in the widely deployment of deep learning services. Experiment results show that balanced sparsity achieves up to 3.1x practical speedup for model inference on GPU, while retains the same high model accuracy as fine-grained sparsity