Zero initialized active learning with spectral clustering using Hungarian method

Supervised machine learning tasks often require a large number of labeled training data to set up a model, and then prediction - for example the classification - is carried out based on this model. Nowadays tremendous amount of data is available on the web or in data warehouses, although only a portion of those data is annotated and the labeling process can be tedious, expensive and time consuming. Active learning tries to overcome this problem by reducing the labeling cost through allowing the learning system to iteratively select the data from which it learns. In special case of active learning, the process starts from zero initialized scenario, where the labeled training dataset is empty, and therefore only unsupervised methods can be performed. In this paper a novel query strategy framework is presented for this problem, called Clustering Based Balanced Sampling Framework (CBBSF), which is not only select the initial labeled training dataset, but uniformly selects the items among the categories to get a balanced labeled training dataset. The framework includes an assignment technique to implicitly determine the class membership probabilities. Assignment solution is updated during CBBSF iterations, hence it simulates supervised machine learning more accurately as the process progresses. The proposed Spectral Clustering Based Sampling (SCBS) query startegy realizes the CBBSF framework, and therefore it is applicable in the special zero initialized situation. This selection approach uses ClusterGAN (Clustering using Generative Adversarial Networks) integrated in the spectral clustering algorithm and then it selects an unlabeled instance depending on the class membership probabilities. Global and local versions of SCBS were developed, furthermore, most confident and minimal entropy measures were calculated, thus four different SCBS variants were examined in total. Experimental evaluation was conducted on the MNIST dataset, and the results showed that SCBS outperforms the state-of-the-art zero initialized active learning query strategies

SEVEN: Deep Semi-supervised Verification Networks

Verification determines whether two samples belong to the same class or not, and has important applications such as face and fingerprint verification, where thousands or millions of categories are present but each category has scarce labeled examples, presenting two major challenges for existing deep learning models. We propose a deep semi-supervised model named SEmi-supervised VErification Network (SEVEN) to address these challenges. The model consists of two complementary components. The generative component addresses the lack of supervision within each category by learning general salient structures from a large amount of data across categories. The discriminative component exploits the learned general features to mitigate the lack of supervision within categories, and also directs the generative component to find more informative structures of the whole data manifold. The two components are tied together in SEVEN to allow an end-to-end training of the two components. Extensive experiments on four verification tasks demonstrate that SEVEN significantly outperforms other state-of-the-art deep semi-supervised techniques when labeled data are in short supply. Furthermore, SEVEN is competitive with fully supervised baselines trained with a larger amount of labeled data. It indicates the importance of the generative component in SEVEN.Comment: 7 pages, 2 figures, accepted to the 2017 International Joint Conference on Artificial Intelligence (IJCAI-17

toward zero shot entity recognition in task oriented conversational agents

n/

OBEBS (Optimally Balanced Entropy-Based Sampling)

In active learning, Optimally Balanced Entropy-Based Sampling (OBEBS) method is a selection strategy from unlabelled data. At active zero-shot learning there is not enough information for supervised machine learning method, thus, our sampling strategy was based on unsupervised learning (clustering). The cluster membership likelihoods of the items were essential for the algorithm to connect the clusters and the classes; i.e. to find assignment between them. For best assignment, Hungarian algorithm was used. We developed and implemented adaptive assignment variants of OBEBS method in the software

OBEBS (Optimally Balanced Entropy-Based Sampling)

In active learning, Optimally Balanced Entropy-Based Sampling (OBEBS) method is a selection strategy from unlabelled data. At active zero-shot learning there is not enough information for supervised machine learning method, thus, our sampling strategy was based on unsupervised learning (clustering). The cluster membership likelihoods of the items were essential for the algorithm to connect the clusters and the classes; i.e. to find assignment between them. For best assignment, Hungarian algorithm was used. We developed and implemented adaptive assignment variants of OBEBS method in the software