Learnt quasi-transitive similarity for retrieval from large collections of faces

We are interested in identity-based retrieval of face sets from large unlabelled collections acquired in uncontrolled environments. Given a baseline algorithm for measuring the similarity of two face sets, the meta-algorithm introduced in this paper seeks to leverage the structure of the data corpus to make the best use of the available baseline. In particular, we show how partial transitivity of inter-personal similarity can be exploited to improve the retrieval of particularly challenging sets which poorly match the query under the baseline measure. We: (i) describe the use of proxy sets as a means of computing the similarity between two sets, (ii) introduce transitivity meta-features based on the similarity of salient modes of appearance variation between sets, (iii) show how quasi-transitivity can be learnt from such features without any labelling or manual intervention, and (iv) demonstrate the effectiveness of the proposed methodology through experiments on the notoriously challenging YouTube database.Postprin

Reimagining the central challenge of face recognition : turning a problem into an advantage

High inter-personal similarity has been universally acknowledged as the principal challenge of automatic face recognition since the earliest days of research in this area. The challenge is particularly prominent when images or videos are acquired in largely unconstrained conditions ‘in the wild’, and intra-personal variability due to illumination, pose, occlusions, and a variety of other confounds is extreme. Counter to the general consensus and intuition, in this paper I demonstrate that in some contexts, high inter-personal similarity can be used to advantage, i.e. it can help improve recognition performance. I start by a theoretical introduction of this key conceptual novelty which I term ‘quasi-transitive similarity’, describe an approach that implements it in practice, and demonstrate its effectiveness empirically. The results on a most challenging real-world data set show impressive performance, and open avenues to future research on different technical approaches which make use of this novel idea.PostprintPeer reviewe

Descriptor transition tables for object retrieval using unconstrained cluttered video acquired using a consumer level handheld mobile device

Visual recognition and vision based retrieval of objects from large databases are tasks with a wide spectrum of potential applications. In this paper we propose a novel recognition method from video sequences suitable for retrieval from databases acquired in highly unconstrained conditions e.g. using a mobile consumer-level device such as a phone. On the lowest level, we represent each sequence as a 3D mesh of densely packed local appearance descriptors. While image plane geometry is captured implicitly by a large overlap of neighbouring regions from which the descriptors are extracted, 3D information is extracted by means of a descriptor transition table, learnt from a single sequence for each known gallery object. These allow us to connect local descriptors along the 3rd dimension (which corresponds to viewpoint changes), thus resulting in a set of variable length Markov chains for each video. The matching of two sets of such chains is formulated as a statistical hypothesis test, whereby a subset of each is chosen to maximize the likelihood that the corresponding video sequences show the same object. The effectiveness of the proposed algorithm is empirically evaluated on the Amsterdam Library of Object Images and a new highly challenging video data set acquired using a mobile phone. On both data sets our method is shown to be successful in recognition in the presence of background clutter and large viewpoint changes.Postprin

Baseline fusion for image an pattern recognition - what not to do (and how to do better)

(Special issue on Computer Vision and Pattern Recognition).The ever-increasing demand for a reliable inference capable of handling unpredictable challenges of practical application in the real world has made research on information fusion of major importance; indeed, this challenge is pervasive in a whole range of image understanding tasks. In the development of the most common type—score-level fusion algorithms—it is virtually universally desirable to have as a reference starting point a simple and universally sound baseline benchmark which newly developed approaches can be compared to. One of the most pervasively used methods is that of weighted linear fusion. It has cemented itself as the default off-the-shelf baseline owing to its simplicity of implementation, interpretability, and surprisingly competitive performance across a widest range of application domains and information source types. In this paper I argue that despite this track record, weighted linear fusion is not a good baseline on the grounds that there is an equally simple and interpretable alternative—namely quadratic mean-based fusion—which is theoretically more principled and which is more successful in practice. I argue the former from first principles and demonstrate the latter using a series of experiments on a diverse set of fusion problems: classification using synthetically generated data, computer vision-based object recognition, arrhythmia detection, and fatality prediction in motor vehicle accidents. On all of the aforementioned problems and in all instances, the proposed fusion approach exhibits superior performance over linear fusion, often increasing class separation by several orders of magnitude.Publisher PDFPeer reviewe

Machine learning based detection of Kepler objects of interest

The authors would like to thank CNPq-Brazil and the University of St Andrews for their kind supportPostprintPeer reviewe

Employing domain specific discriminative information to address inherent limitations of the LBP descriptor in face recognition

The local binary patern (LBP) descriptor and its derivatives have a demonstrated track record of good performance in face recognition. Nevertheless the original descriptor, the framework within which it is employed, and the aforementioned improvements of these in the existing literature, all suffer from a number of inherent limitations. In this work we highlight these and propose novel ways of addressing them in a principled fashion. Specifically, we introduce (i) gradient based weighting of local descriptor contributions to region based histograms as a means of avoiding data smoothing by non-discriminative image loci, and (ii) Gaussian fuzzy region membership as a means of achieving robustness to registration errors. Importantly, the nature of these contributions allows the proposed techniques to be combined with the existing extensions to the LBP descriptor thus making them universally recommendable. Effectiveness is demonstrated on the notoriously challenging Extended Yale B face corpus.Postprin

A Siamese transformer network for zero-shot ancient coin classification

Ancient numismatics, the study of ancient coins, has in recent years become an attractive domain for the application of computer vision and machine learning. Though rich in research problems, the predominant focus in this area to date has been on the task of attributing a coin from an image, that is of identifying its issue. This may be considered the cardinal problem in the field and it continues to challenge automatic methods. In the present paper, we address a number of limitations of previous work. Firstly, the existing methods approach the problem as a classification task. As such, they are unable to deal with classes with no or few exemplars (which would be most, given over 50,000 issues of Roman Imperial coins alone), and require retraining when exemplars of a new class become available. Hence, rather than seeking to learn a representation that distinguishes a particular class from all the others, herein we seek a representation that is overall best at distinguishing classes from one another, thus relinquishing the demand for exemplars of any specific class. This leads to our adoption of the paradigm of pairwise coin matching by issue, rather than the usual classification paradigm, and the specific solution we propose in the form of a Siamese neural network. Furthermore, while adopting deep learning, motivated by its successes in the field and its unchallenged superiority over classical computer vision approaches, we also seek to leverage the advantages that transformers have over the previously employed convolutional neural networks, and in particular their non-local attention mechanisms, which ought to be particularly useful in ancient coin analysis by associating semantically but not visually related distal elements of a coin’s design. Evaluated on a large data corpus of 14,820 images and 7605 issues, using transfer learning and only a small training set of 542 images of 24 issues, our Double Siamese ViT model is shown to surpass the state of the art by a large margin, achieving an overall accuracy of 81%. Moreover, our further investigation of the results shows that the majority of the method’s errors are unrelated to the intrinsic aspects of the algorithm itself, but are rather a consequence of unclean data, which is a problem that can be easily addressed in practice by simple pre-processing and quality checking.Publisher PDFPeer reviewe

Face Image Retrieval with Landmark Detection and Semantic Concepts Extraction

This thesis proposes various novel approaches for improving the performances of automatic facial landmarks detection system based on the concept of pictorial tree structure model. Furthermore, a robust glasses landmark detection system is also proposed as glasses are commonly used. These proposed approaches are employed to develop an automatic semantic based face images retrieval system. The experiment results demonstrate significant improvements of all the proposed approaches towards accuracy and efficiency

Topological Foundations of Cognitive Science

A collection of papers presented at the First International Summer Institute in Cognitive Science, University at Buffalo, July 1994, including the following papers: ** Topological Foundations of Cognitive Science, Barry Smith ** The Bounds of Axiomatisation, Graham White ** Rethinking Boundaries, Wojciech Zelaniec ** Sheaf Mereology and Space Cognition, Jean Petitot ** A Mereotopological Definition of 'Point', Carola Eschenbach ** Discreteness, Finiteness, and the Structure of Topological Spaces, Christopher Habel ** Mass Reference and the Geometry of Solids, Almerindo E. Ojeda ** Defining a 'Doughnut' Made Difficult, N .M. Gotts ** A Theory of Spatial Regions with Indeterminate Boundaries, A.G. Cohn and N.M. Gotts ** Mereotopological Construction of Time from Events, Fabio Pianesi and Achille C. Varzi ** Computational Mereology: A Study of Part-of Relations for Multi-media Indexing, Wlodek Zadrozny and Michelle Ki

Spot the Odd Song Out: Similarity Model Adaptation and Analysis using Relative Human Ratings

Understanding how listeners relate and compare pieces of music is a fundamental challenge in music research as well as for commercial applications: Today’s large-scale applications for music recommendation and exploration utilise various models for similarity prediction to satisfy users’ expectations. Perceived similarity is specific to the individual and influenced by a number of factors such as cultural background and age. Thus, adapting a generic model to human similarity data is useful for personalisation and can help to better understand such differences. This thesis presents new and state-of-the-art machine learning techniques for modelling music similarity and their first evaluation on relative music similarity data. We expand the scope for future research with methods for similarity data collection and a new dataset. In particular, our models are evaluated on their ability to “spot the odd song out” of three given songs. While a few methods are readily available, others had to be adapted for their first application to such data. We explore the potential for learning generalisable similarity measures, presenting algorithms for metrics and neural networks. A generic modelling workflow is presented and implemented. We report the first evaluation of the methods on the MagnaTagATune dataset showing learning is possible and pointing out particularities of algorithms and feature types. The best results with up to 74% performance on test sets were achieved with a combination of acoustic and cultural features, but model training proved most powerful when only acoustic information is available. To assess the generalisability of the findings, we provide a first systematic analysis of the dataset itself. We also identify a bias in standard sampling methods for cross-validation with similarity data and present a new method for unbiased evaluation, providing use cases for the different validation strategies. Furthermore, we present an online game that collects a new similarity dataset, including participant attributes such as age, location, language and music background. It is based on our extensible framework which manages storage of participant input, context information as well as selection of presented samples. The collected data enables a more specific adaptation of music similarity by including user attributes into similarity models. Distinct similarity models are learnt from geographically defined user groups in a first experiment towards the more complex task of culture-aware similarity modelling. In order to improve training of the specific models on small datasets, we implement the concept of transfer learning for music similarity models