Multimodal Storytelling via Generative Adversarial Imitation Learning

Deriving event storylines is an effective summarization method to succinctly organize extensive information, which can significantly alleviate the pain of information overload. The critical challenge is the lack of widely recognized definition of storyline metric. Prior studies have developed various approaches based on different assumptions about users' interests. These works can extract interesting patterns, but their assumptions do not guarantee that the derived patterns will match users' preference. On the other hand, their exclusiveness of single modality source misses cross-modality information. This paper proposes a method, multimodal imitation learning via generative adversarial networks(MIL-GAN), to directly model users' interests as reflected by various data. In particular, the proposed model addresses the critical challenge by imitating users' demonstrated storylines. Our proposed model is designed to learn the reward patterns given user-provided storylines and then applies the learned policy to unseen data. The proposed approach is demonstrated to be capable of acquiring the user's implicit intent and outperforming competing methods by a substantial margin with a user study.Comment: IJCAI 201

Patent Citation Dynamics Modeling via Multi-Attention Recurrent Networks

Modeling and forecasting forward citations to a patent is a central task for the discovery of emerging technologies and for measuring the pulse of inventive progress. Conventional methods for forecasting these forward citations cast the problem as analysis of temporal point processes which rely on the conditional intensity of previously received citations. Recent approaches model the conditional intensity as a chain of recurrent neural networks to capture memory dependency in hopes of reducing the restrictions of the parametric form of the intensity function. For the problem of patent citations, we observe that forecasting a patent's chain of citations benefits from not only the patent's history itself but also from the historical citations of assignees and inventors associated with that patent. In this paper, we propose a sequence-to-sequence model which employs an attention-of-attention mechanism to capture the dependencies of these multiple time sequences. Furthermore, the proposed model is able to forecast both the timestamp and the category of a patent's next citation. Extensive experiments on a large patent citation dataset collected from USPTO demonstrate that the proposed model outperforms state-of-the-art models at forward citation forecasting

Lasing on nonlinear localized waves in curved geometry

The use of geometrical constraints opens many new perspectives in photonics and in fundamental studies of nonlinear waves. By implementing surface structures in vertical cavity surface emitting lasers as manifolds for curved space, we experimentally study the impacts of geometrical constraints on nonlinear wave localization. We observe localized waves pinned to the maximal curvature in an elliptical-ring, and confirm the reduction in the localization length of waves by measuring near and far field patterns, as well as the corresponding dispersion relation. Theoretically, analyses based on a dissipative model with a parabola curve give good agreement remarkably to experimental measurement on the transition from delocalized to localized waves. The introduction of curved geometry allows to control and design lasing modes in the nonlinear regime.Comment: 6 pages, 6 figure