Universum-inspired Supervised Contrastive Learning

As an effective data augmentation method, Mixup synthesizes an extra amount of samples through linear interpolations. Despite its theoretical dependency on data properties, Mixup reportedly performs well as a regularizer and calibrator contributing reliable robustness and generalization to deep model training. In this paper, inspired by Universum Learning which uses out-of-class samples to assist the target tasks, we investigate Mixup from a largely under-explored perspective - the potential to generate in-domain samples that belong to none of the target classes, that is, universum. We find that in the framework of supervised contrastive learning, Mixup-induced universum can serve as surprisingly high-quality hard negatives, greatly relieving the need for large batch sizes in contrastive learning. With these findings, we propose Universum-inspired supervised Contrastive learning (UniCon), which incorporates Mixup strategy to generate Mixup-induced universum as universum negatives and pushes them apart from anchor samples of the target classes. We extend our method to the unsupervised setting, proposing Unsupervised Universum-inspired contrastive model (Un-Uni). Our approach not only improves Mixup with hard labels, but also innovates a novel measure to generate universum data. With a linear classifier on the learned representations, UniCon shows state-of-the-art performance on various datasets. Specially, UniCon achieves 81.7% top-1 accuracy on CIFAR-100, surpassing the state of art by a significant margin of 5.2% with a much smaller batch size, typically, 256 in UniCon vs. 1024 in SupCon using ResNet-50. Un-Uni also outperforms SOTA methods on CIFAR-100. The code of this paper is released on https://github.com/hannaiiyanggit/UniCon.Comment: Accepted by IEEE Transactions on Image Processin

Breast Cancer Classification by Gene Expression Analysis using Hybrid Feature Selection and Hyper-heuristic Adaptive Universum Support Vector Machine

Comprehensive assessments of the molecular characteristics of breast cancer from gene expression patterns can aid in the early identification and treatment of tumor patients. The enormous scale of gene expression data obtained through microarray sequencing increases the difficulty of training the classifier due to large-scale features. Selecting pivotal gene features can minimize high dimensionality and the classifier complexity with improved breast cancer detection accuracy. However, traditional filter and wrapper-based selection methods have scalability and adaptability issues in handling complex gene features. This paper presents a hybrid feature selection method of Mutual Information Maximization - Improved Moth Flame Optimization (MIM-IMFO) for gene selection along with an advanced Hyper-heuristic Adaptive Universum Support classification model Vector Machine (HH-AUSVM) to improve cancer detection rates. The hybrid gene selection method is developed by performing filter-based selection using MIM in the first stage followed by the wrapper method in the second stage, to obtain the pivotal features and remove the inappropriate ones. This method improves standard MFO by a hybrid exploration/exploitation phase to accomplish a better trade-off between exploration and exploitation phases. The classifier HH-AUSVM is formulated by integrating the Adaptive Universum learning approach to the hyper- heuristics-based parameter optimized SVM to tackle the class samples imbalance problem. Evaluated on breast cancer gene expression datasets from Mendeley Data Repository, this proposed MIM-IMFO gene selection-based HH-AUSVM classification approach provided better breast cancer detection with high accuracies of 95.67%, 96.52%, 97.97% and 95.5% and less processing time of 4.28, 3.17, 9.45 and 6.31 seconds, respectively

Distance Matrix Approach to Content Image Retrieval

As the volume of image data and the need of using it in various applications is growing significantly in the last days it brings a necessity of retrieval efficiency and effectiveness. Unfortunately, existing indexing methods are not applicable to a wide range of problem-oriented fields due to their operating time limitations and strong dependency on the traditional descriptors extracted from the image. To meet higher requirements, a novel distance-based indexing method for region-based image retrieval has been proposed and investigated. The method creates premises for considering embedded partitions of images to carry out the search with different refinement or roughening level and so to seek the image meaningful content

A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning

Current deep learning research is dominated by benchmark evaluation. A method is regarded as favorable if it empirically performs well on the dedicated test set. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving sets of benchmark data are investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten due to the iterative parameter updates. However, comparison of individual methods is nevertheless treated in isolation from real world application and typically judged by monitoring accumulated test set performance. The closed world assumption remains predominant. It is assumed that during deployment a model is guaranteed to encounter data that stems from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown instances and break down in the face of corrupted data. In this work we argue that notable lessons from open set recognition, the identification of statistically deviating data outside of the observed dataset, and the adjacent field of active learning, where data is incrementally queried such that the expected performance gain is maximized, are frequently overlooked in the deep learning era. Based on these forgotten lessons, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Our results show that this not only benefits each individual paradigm, but highlights the natural synergies in a common framework. We empirically demonstrate improvements when alleviating catastrophic forgetting, querying data in active learning, selecting task orders, while exhibiting robust open world application where previously proposed methods fail.Comment: 32 page

Ambient and intrinsic triangulations and topological methods in cosmology

The thesis consist of two parts, one part concerns triangulations the other the structure of the universe. 1 Images in films such as Shrek or Frozen and in computer games are often made using small triangles. Subdividing a figure (such as Shrek) into small triangles is called triangulating. This may be done in two different ways. The first method makes use of straight triangles and is used most often. Because computer power is limited, we want to use as few triangles as possible, while maintaining the quality of the image. This means that one has to choose the triangles in a clever manner. Much is known about the choice of triangles if the surface is convex (egg-shaped). This thesis contributes to our understanding of non-convex surfaces. The second and new method uses curved triangles that follow the surface. The triangles we use are determined by the intrinsic geometry of the surface and are called intrinsic triangles. 2 Shortly after the Big Bang the universe was very hot and dense. Quantum mechanical effects introduced structure into the matter distribution in the early universe. The universe expanded according the laws of General Relativity and the matter cooled down. After the matter in the universe had cooled down, clusters of galaxies formed out of the densest regions. These clusters of galaxies are connected by stringy structures consisting of galaxies. This thesis contributes to the understanding of this intricate structure

Ambient and intrinsic triangulations and topological methods in cosmology

The thesis consist of two parts, one part concerns triangulations the other the structure of the universe. 1 Images in films such as Shrek or Frozen and in computer games are often made using small triangles. Subdividing a figure (such as Shrek) into small triangles is called triangulating. This may be done in two different ways. The first method makes use of straight triangles and is used most often. Because computer power is limited, we want to use as few triangles as possible, while maintaining the quality of the image. This means that one has to choose the triangles in a clever manner. Much is known about the choice of triangles if the surface is convex (egg-shaped). This thesis contributes to our understanding of non-convex surfaces. The second and new method uses curved triangles that follow the surface. The triangles we use are determined by the intrinsic geometry of the surface and are called intrinsic triangles. 2 Shortly after the Big Bang the universe was very hot and dense. Quantum mechanical effects introduced structure into the matter distribution in the early universe. The universe expanded according the laws of General Relativity and the matter cooled down. After the matter in the universe had cooled down, clusters of galaxies formed out of the densest regions. These clusters of galaxies are connected by stringy structures consisting of galaxies. This thesis contributes to the understanding of this intricate structure