Multimodal Convolutional Neural Networks for Matching Image and Sentence

In this paper, we propose multimodal convolutional neural networks (m-CNNs) for matching image and sentence. Our m-CNN provides an end-to-end framework with convolutional architectures to exploit image representation, word composition, and the matching relations between the two modalities. More specifically, it consists of one image CNN encoding the image content, and one matching CNN learning the joint representation of image and sentence. The matching CNN composes words to different semantic fragments and learns the inter-modal relations between image and the composed fragments at different levels, thus fully exploit the matching relations between image and sentence. Experimental results on benchmark databases of bidirectional image and sentence retrieval demonstrate that the proposed m-CNNs can effectively capture the information necessary for image and sentence matching. Specifically, our proposed m-CNNs for bidirectional image and sentence retrieval on Flickr30K and Microsoft COCO databases achieve the state-of-the-art performances.Comment: Accepted by ICCV 201

Western Modernity interwoven with Chinese traditions:

As one of the first Chinese cities opened to Western trade in mid 19th century, Shanghai soon became the preferred place of residence for foreign merchants and entrepreneurs in East Asia. Shanghai’s economy thrived and the people made this development possible came from various provinces of China, from the principal nations of Europe, the United States, Japan, and other Asian countries already colonised by the West. The relative importance of these communities varied from one period to another, but the barriers that separated them - languages, customs, and interests - all contributed to the fragmentation of the local society. This paper examines how different practices came to be grafted onto traditional systems and how Shanghai provided many opportunities for intercultural contacts between groups of people from vastly different backgrounds. I shall demonstrate that a new cosmopolitan society, largely modeled upon the Western modernity but interwoven with traditions from various parts of China, was rapidly emerging in Shanghai in the late nineteenth century, together with a distinctive “Shanghai identity” that shaped both Chinese and foreigners