The Emerging Trends of Multi-Label Learning

Exabytes of data are generated daily by humans, leading to the growing need for new efforts in dealing with the grand challenges for multi-label learning brought by big data. For example, extreme multi-label classification is an active and rapidly growing research area that deals with classification tasks with an extremely large number of classes or labels; utilizing massive data with limited supervision to build a multi-label classification model becomes valuable for practical applications, etc. Besides these, there are tremendous efforts on how to harvest the strong learning capability of deep learning to better capture the label dependencies in multi-label learning, which is the key for deep learning to address real-world classification tasks. However, it is noted that there has been a lack of systemic studies that focus explicitly on analyzing the emerging trends and new challenges of multi-label learning in the era of big data. It is imperative to call for a comprehensive survey to fulfill this mission and delineate future research directions and new applications.Comment: Accepted to TPAMI 202

Online Metric Learning for Multi-Label Classification

Existing research into online multi-label classification, such as online sequential multi-label extreme learning machine (OSML-ELM) and stochastic gradient descent (SGD), has achieved promising performance. However, these works lack an analysis of loss function and do not consider label dependency. Accordingly, to fill the current research gap, we propose a novel online metric learning paradigm for multi-label classification. More specifically, we first project instances and labels into a lower dimension for comparison, then leverage the large margin principle to learn a metric with an efficient optimization algorithm. Moreover, we provide theoretical analysis on the upper bound of the cumulative loss for our method. Comprehensive experiments on a number of benchmark multi-label datasets validate our theoretical approach and illustrate that our proposed online metric learning (OML) algorithm outperforms state-of-the-art methods

Online Metric Learning for Multi-Label Classification

Online multi-label classification has been widely used in various real world applications, such as twitter, facebookpost, instagram, video search and RSS feeds. With the proliferation of multi-label classification, significant researchefforts have been devoted. This thesis studies some approaches on online multi-label classification.Existing researches on online multi-label classification, such as online sequential multi-label extreme learningmachine (OSML-ELM) and stochastic gradient descent (SGD), have shown a promising performance on multi-labelclassification. However, these works lack analysis of loss function and do not consider the label dependency. To fillthe gap of current research, we propose a novel online metric learning paradigm for multi-label classification.Specifically, we first project instances and labels into a lower dimension for comparison, and then leverage thelarge margin principle to learn a metric with an efficient optimization algorithm. Moreover, we provide the theoreticalanalysis on the upper bound of the cumulative loss for our method. Comprehensive experiments on a number ofbenchmark multi-label datasets validate our theoretical studies and illustrate that our proposed online metriclearning (OML) algorithm outperforms state-of-the-art approaches

Distributed Online Metric Learning for Multi-Label Classification

The multi-label classification problem is an important area of research. In this problem setting, many well-developed batched learning algorithms have been proposed. However, traditional batched learning algorithms are not efficient enough due to the costly pre-processing and integration of the data. Online learning algorithms are powerful in dealing with these challenges but still lack practical data distribution consideration. In real-world scenarios, data is usually generated in a distributed manner. Therefore, the training data needs to be transferred to a centralised server in traditional centralised online algorithms before further operations. However, this transfer is both inefficient and comes with a security risk. An online multi-label learning algorithm for distributed scenarios is still a research gap. To address this research gap, we propose a novel distributed online metric learning algorithm based on the distributed least mean squares method and metric learning algorithm. It can achieve the performance of a centralised classification algorithm without the need to transfer training data. We provide convergence analysis for our proposed algorithm and an experimental analysis of the scalability of the proposed DOML algorithm. The experimental results show that our proposed algorithm achieves the expected outcomes. The loss of each node can converge to a stable value after tens of iterations, which confirms the fast convergence of our proposed DOML algorithm. Meanwhile, our proposed DOML algorithm achieves comparable accuracies to traditional batch learning and online metric learning (OML) algorithms using the same datasets. In addition, for scalability, our proposed DOML algorithm converges slower in the larger network environment, but the connectivity of the networks has no significant effect on the final training results