Empirical study of multi-label classification methods for image annotation and retrieval

This paper presents an empirical study of multi-label classification methods, and gives suggestions for multi-label classification that are effective for automatic image annotation applications. The study shows that triple random ensemble multi-label classification algorithm (TREMLC) outperforms among its counterparts, especially on scene image dataset. Multi-label k-nearest neighbor (ML-kNN) and binary relevance (BR) learning algorithms perform well on Corel image dataset. Based on the overall evaluation results, examples are given to show label prediction performance for the algorithms using selected image examples. This provides an indication of the suitability of different multi-label classification methods for automatic image annotation under different problem settings.<br /

Conditional Graphical Lasso for Multi-label Image Classification

© 2016 IEEE. Multi-label image classification aims to predict multiple labels for a single image which contains diverse content. By utilizing label correlations, various techniques have been developed to improve classification performance. However, current existing methods either neglect image features when exploiting label correlations or lack the ability to learn image-dependent conditional label structures. In this paper, we develop conditional graphical Lasso (CGL) to handle these challenges. CGL provides a unified Bayesian framework for structure and parameter learning conditioned on image features. We formulate the multi-label prediction as CGL inference problem, which is solved by a mean field variational approach. Meanwhile, CGL learning is efficient due to a tailored proximal gradient procedure by applying the maximum a posterior (MAP) methodology. CGL performs competitively for multi-label image classification on benchmark datasets MULAN scene, PASCAL VOC 2007 and PASCAL VOC 2012, compared with the state-of-the-art multi-label classification algorithms

Multi-level adaptive active learning for scene classification

Semantic scene classification is a challenging problem in computer vision. In this paper, we present a novel multi-level active learning approach to reduce the human annotation effort for training robust scene classification models. Different from most existing active learning methods that can only query labels for selected instances at the target categorization level, i.e., the scene class level, our approach establishes a semantic framework that predicts scene labels based on a latent object-based semantic representation of images, and is capable to query labels at two different levels, the target scene class level (abstractive high level) and the latent object class level (semantic middle level). Specifically, we develop an adaptive active learning strategy to perform multi-level label query, which maintains the default label query at the target scene class level, but switches to the latent object class level whenever an "unexpected" target class label is returned by the labeler. We conduct experiments on two standard scene classification datasets to investigate the efficacy of the proposed approach. Our empirical results show the proposed adaptive multi-level active learning approach can outperform both baseline active learning methods and a state-of-the-art multi-level active learning method

Multi-Label Self-Supervised Learning with Scene Images

Self-supervised learning (SSL) methods targeting scene images have seen a rapid growth recently, and they mostly rely on either a dedicated dense matching mechanism or a costly unsupervised object discovery module. This paper shows that instead of hinging on these strenuous operations, quality image representations can be learned by treating scene/multi-label image SSL simply as a multi-label classification problem, which greatly simplifies the learning framework. Specifically, multiple binary pseudo-labels are assigned for each input image by comparing its embeddings with those in two dictionaries, and the network is optimized using the binary cross entropy loss. The proposed method is named Multi-Label Self-supervised learning (MLS). Visualizations qualitatively show that clearly the pseudo-labels by MLS can automatically find semantically similar pseudo-positive pairs across different images to facilitate contrastive learning. MLS learns high quality representations on MS-COCO and achieves state-of-the-art results on classification, detection and segmentation benchmarks. At the same time, MLS is much simpler than existing methods, making it easier to deploy and for further exploration.Comment: ICCV202

Multi-label learning by Image-to-Class distance for scene classification and image annotation

In multi-label learning, an image containing multiple objects can be assigned to multiple labels, which makes it more chal-lenging than traditional multi-class classification task where an image is assigned to only one label. In this paper, we propose a multi-label learning framework based on Image-to-Class (I2C) distance, which is recently shown useful for image classification. We adjust this I2C distance to cater for the multi-label problem by learning a weight attached to each local feature patch and formulating it into a large margin optimization problem. For each image, we constrain its weighted I2C distance to the relevant class to be much less than its distance to other irrelevant class, by the use of a margin in the optimization problem. Label ranks are generated under this learned I2C distance framework for a query image. Thereafter, we employ the label correlation in-formation to split the label rank for predicting the label(s) of this query image. The proposed method is evaluated in the applications of scene classification and automatic image annotation using both the natural scene dataset and Mi-crosoft Research Cambridge (MSRC) dataset. Experiment results show better performance of our method compared to previous multi-label learning algorithms

Study on Multi-label Image Classification Based on Sample Distribution Loss

Different from the data distribution in general image classification scenarios,in the scenario of multi label image classification,the sample number distribution among different label categories is unbalanced,and a small number of head categories often account for the majority of sample size.However,due to the correlation between multiple labels,and the distribution of diffi-cult samples under multiple labels is also related to the data distribution and category distribution,the re-sampling and other methods for solving the data imbalance in the single label problem cannot be effectively applied in the multi label scenario.This paper proposes a classification method based on the loss of sample distribution in multi label image scene and deep learning.Firs-tly,the unbalanced distribution of multi label data is set with category correlation,and the loss is re-used,and the dynamic lear-ning method is used to prevent the excessive alienation of distribution.Then,the asymmetric sample learning loss is designed,and different learning abilities for positive and negative samples and difficult samples are set.At the same time,the information loss is reduced by softening the sample learning weight.Experiments on related data sets show that the algorithm has achieved good results in solving the sample learning problem in the scene of uneven distribution of multi-label data

A Comparative Study of Deep Learning Loss Functions for Multi-Label Remote Sensing Image Classification

This paper analyzes and compares different deep learning loss functions in the framework of multi-label remote sensing (RS) image scene classification problems. We consider seven loss functions: 1) cross-entropy loss; 2) focal loss; 3) weighted cross-entropy loss; 4) Hamming loss; 5) Huber loss; 6) ranking loss; and 7) sparseMax loss. All the considered loss functions are analyzed for the first time in RS. After a theoretical analysis, an experimental analysis is carried out to compare the considered loss functions in terms of their: 1) overall accuracy; 2) class imbalance awareness (for which the number of samples associated to each class significantly varies); 3) convexibility and differentiability; and 4) learning efficiency (i.e., convergence speed). On the basis of our analysis, some guidelines are derived for a proper selection of a loss function in multi-label RS scene classification problems.Comment: Accepted at IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2020. For code visit: https://gitlab.tubit.tu-berlin.de/rsim/RS-MLC-Losse