Exploring Interaction Between Images and Texts for Web Image Categorization

With the rapid development of technologies for fast access to the Internet and the popularization of digital cameras, enormous digital images are posted and shared online everyday. Simultaneously, web images are usually organized by topics of events and are often assigned appropriate topic-related text descriptions. Given a set of images along with corresponding texts, a challenging problem is how to utilize the available information to perform image retrieval tasks, such as image classification and image clustering. Previous works on image categorization focus on either adopting text or image features, or simply combining these two types of information together. In this paper, we propose two novel approaches (Dynamic Weighting and Region-based Semantic Concept Integration) to categorize the images under the “supervision” of topic-related text descriptions; In addition, we provide a comparative experimental investigation on utilizing text and image information to tackle image classification. Empirical experiments on a manually collected image dataset (consisting of images related to the events after disasters) demonstrate the efficacy of our proposed classification methods