Learning Multimodal Word Representation via Dynamic Fusion Methods

Multimodal models have been proven to outperform text-based models on learning semantic word representations. Almost all previous multimodal models typically treat the representations from different modalities equally. However, it is obvious that information from different modalities contributes differently to the meaning of words. This motivates us to build a multimodal model that can dynamically fuse the semantic representations from different modalities according to different types of words. To that end, we propose three novel dynamic fusion methods to assign importance weights to each modality, in which weights are learned under the weak supervision of word association pairs. The extensive experiments have demonstrated that the proposed methods outperform strong unimodal baselines and state-of-the-art multimodal models.Comment: To be appear in AAAI-1

Combinatorial cobordism maps in hat Heegaard Floer theory

In a previous paper, Sarkar and the third author gave a combinatorial description of the hat version of Heegaard Floer homology for three-manifolds. Given a cobordism between two connected three-manifolds, there is an induced map between their Heegaard Floer homologies. Assume that the first homology group of each boundary component surjects onto the first homology group of the cobordism (modulo torsion). Under this assumption, we present a procedure for finding the rank of the induced Heegaard Floer map combinatorially, in the hat version.Comment: 34 pages, 31 figures; discussion now limited to cobordisms satisfying a homological assumption; Section 4 completely rewritten; various other revisions; this version to appear in Duke Math.