Discriminant feature extraction by generalized difference subspace

This paper reveals the discriminant ability of the orthogonal projection of data onto a generalized difference subspace (GDS) both theoretically and experimentally. In our previous work, we have demonstrated that GDS projection works as the quasi-orthogonalization of class subspaces. Interestingly, GDS projection also works as a discriminant feature extraction through a similar mechanism to the Fisher discriminant analysis (FDA). A direct proof of the connection between GDS projection and FDA is difficult due to the significant difference in their formulations. To avoid the difficulty, we first introduce geometrical Fisher discriminant analysis (gFDA) based on a simplified Fisher criterion. gFDA can work stably even under few samples, bypassing the small sample size (SSS) problem of FDA. Next, we prove that gFDA is equivalent to GDS projection with a small correction term. This equivalence ensures GDS projection to inherit the discriminant ability from FDA via gFDA. Furthermore, we discuss two useful extensions of these methods, 1) nonlinear extension by kernel trick, 2) the combination of convolutional neural network (CNN) features. The equivalence and the effectiveness of the extensions have been verified through extensive experiments on the extended Yale B+, CMU face database, ALOI, ETH80, MNIST and CIFAR10, focusing on the SSS problem

Identity From Variation : Representations of Faces Derived From Multiple Instances

Research in face recognition has tended to focus on discriminating between individuals, or "telling people apart." It has recently become clear that it is also necessary to understand how images of the same person can vary, or "telling people together." Learning a new face, and tracking its representation as it changes from unfamiliar to familiar, involves an abstraction of the variability in different images of that person's face. Here, we present an application of principal components analysis computed across different photos of the same person. We demonstrate that people vary in systematic ways, and that this variability is idiosyncratic-the dimensions of variability in one face do not generalize well to another. Learning a new face therefore entails learning how that face varies. We present evidence for this proposal and suggest that it provides an explanation for various effects in face recognition. We conclude by making a number of testable predictions derived from this framework

Adaptation of K-means-type algorithms to the Grassmann manifold, An

2019 Spring.Includes bibliographical references.The Grassmann manifold provides a robust framework for analysis of high-dimensional data through the use of subspaces. Treating data as subspaces allows for separability between data classes that is not otherwise achieved in Euclidean space, particularly with the use of the smallest principal angle pseudometric. Clustering algorithms focus on identifying similarities within data and highlighting the underlying structure. To exploit the properties of the Grassmannian for unsupervised data analysis, two variations of the popular K-means algorithm are adapted to perform clustering directly on the manifold. We provide the theoretical foundations needed for computations on the Grassmann manifold and detailed derivations of the key equations. Both algorithms are then thoroughly tested on toy data and two benchmark data sets from machine learning: the MNIST handwritten digit database and the AVIRIS Indian Pines hyperspectral data. Performance of algorithms is tested on manifolds of varying dimension. Unsupervised classification results on the benchmark data are compared to those currently found in the literature

Comparing sets of data sets on the Grassmann and flag manifolds with applications to data analysis in high and low dimensions

Includes bibliographical references.2020 Summer.This dissertation develops numerical algorithms for comparing sets of data sets utilizing shape and orientation of data clouds. Two key components for "comparing" are the distance measure between data sets and correspondingly the geodesic path in between. Both components will play a core role which connects two parts of this dissertation, namely data analysis on the Grassmann manifold and flag manifold. For the first part, we build on the well known geometric framework for analyzing and optimizing over data on the Grassmann manifold. To be specific, we extend the classical self-organizing mappings to the Grassamann manifold to visualize sets of high dimensional data sets in 2D space. We also propose an optimization problem on the Grassmannian to recover missing data. In the second part, we extend the geometric framework to the flag manifold to encode the variability of nested subspaces. There we propose a numerical algorithm for computing a geodesic path and distance between nested subspaces. We also prove theorems to show how to reduce the dimension of the algorithm for practical computations. The approach is shown to have advantages for analyzing data when the number of data points is larger than the number of features

Numerical algebraic geometry approach to polynomial optimization, The

2017 Summer.Includes bibliographical references.Numerical algebraic geometry (NAG) consists of a collection of numerical algorithms, based on homotopy continuation, to approximate the solution sets of systems of polynomial equations arising from applications in science and engineering. This research focused on finding global solutions to constrained polynomial optimization problems of moderate size using NAG methods. The benefit of employing a NAG approach to nonlinear optimization problems is that every critical point of the objective function is obtained with probability-one. The NAG approach to global optimization aims to reduce computational complexity during path tracking by exploiting structure that arises from the corresponding polynomial systems. This thesis will consider applications to systems biology and life sciences where polynomials solve problems in model compatibility, model selection, and parameter estimation. Furthermore, these techniques produce mathematical models of large data sets on non-euclidean manifolds such as a disjoint union of Grassmannians. These methods will also play a role in analyzing the performance of existing local methods for solving polynomial optimization problems

Reconhecimento de caras com componentes principais

Mestrado em Engenharia Electrónica e TelecomunicaçõesThe purpose of this dissertation was to analyze the image processing method known as Principal Component Analysis (PCA) and its performance when applied to face recognition. This algorithm spans a subspace (called facespace) where the faces in a database are represented with a reduced number of features (called feature vectors). The study focused on performing various exhaustive tests to analyze in what conditions it is best to apply PCA. First, a facespace was spanned using the images of all the people in the database. We obtained then a new representation of each image by projecting them onto this facespace. We measured the distance between the projected test image with the other projections and determined that the closest test-train couple (k-Nearest Neighbour) was the recognized subject. This first way of applying PCA was tested with the Leave{One{Out test. This test takes an image in the database for test and the rest to build the facespace, and repeats the process until all the images have been used as test image once, adding up the successful recognitions as a result. The second test was to perform an 8{Fold Cross{Validation, which takes ten images as eligible test images (there are 10 persons in the database with eight images each) and uses the rest to build the facespace. All test images are tested for recognition in this fold, and the next fold is carried out, until all eight folds are complete, showing a different set of results. The other way to use PCA we used was to span what we call Single Person Facespaces (SPFs, a group of subspaces, each spanned with images of a single person) and measure subspace distance using the theory of principal angles. Since the database is small, a way to synthesize images from the existing ones was explored as a way to overcoming low successful recognition rates. All of these tests were performed for a series of thresholds (a variable which selected the number of feature vectors the facespaces were built with, i.e. the facespaces' dimension), and for the database after being preprocessed in two different ways in order to reduce statistically redundant information. The results obtained throughout the tests were within what expected from what can be read in literature: success rates of around 85% in some cases. Special mention needs to be made on the great result improvement between SPFs before and after extending the database with synthetic images. The results revealed that using PCA to project the images in the group facespace is very accurate for face recognition, even when having a small number of samples per subject. Comparing personal facespaces is more effective when we can synthesize images or have a natural way of acquiring new images of the subject, like for example using video footage. The tests and results were obtained with a custom software with user interface, designed and programmed by the author of this dissertation.O propósito desta Dissertação foi a aplicação da Analise em Componentes Principais (PCA, de acordo com as siglas em inglês), em sistemas para reconhecimento de faces. Esta técnica permite calcular um subespaço (chamado facespace, onde as imagens de uma base de dados são representadas por um número reduzido de características (chamadas feature vectors). O estudo realizado centrou-se em vários testes para analisar quais são as condições óptimas para aplicar o PCA. Para começar, gerou-se um faces- pace utilizando todas as imagens da base de dados. Obtivemos uma nova representação de cada imagem, após a projecção neste espaço, e foram medidas as distâncias entre as projecções da imagem de teste e as de treino. A dupla de imagens de teste-treino mais próximas determina o sujeito reconhecido (classificador vizinhos mais próximos). Esta primeira forma de aplicar o PCA, e o respectivo classificador, foi avaliada com as estratégias Leave{One{Out e 8{Fold Cross{Validation. A outra forma de utilizar o PCA foi gerando subespaços individuais (designada por SPF, Single Person Facespace), onde cada subespaço era gerado com imagens de apenas uma pessoa, para a seguir medir a distância entre estes espaços utilizando o conceito de ângulos principais. Como a base de dados era pequena, foi explorada uma forma de sintetizar novas imagens a partir das já existentes. Todos estes teste foram feitos para uma série de limiares (uma variável threshold que determinam o número de feature vectors com os que o faces- pace é construído) e diferentes formas de pre-processamento. Os resultados obtidos estavam dentro do esperado: taxas de acerto aproximadamente iguais a 85% em alguns casos. Pode destacar-se uma grande melhoria na taxa de reconhecimento após a inclusão de imagens sintéticas na base de dados. Os resultados revelaram que o uso do PCA para projectar imagens no subespaço da base de dados _e viável em sistemas de reconhecimento de faces, principalmente se comparar subespaço individuais no caso de base de dados com poucos exemplares em que _e possível sintetizar imagens ou em sistemas com captura de vídeo

Low rank representations of matrices using nuclear norm heuristics

2014 Summer.The pursuit of low dimensional structure from high dimensional data leads in many instances to the finding the lowest rank matrix among a parameterized family of matrices. In its most general setting, this problem is NP-hard. Different heuristics have been introduced for approaching the problem. Among them is the nuclear norm heuristic for rank minimization. One aspect of this thesis is the application of the nuclear norm heuristic to the Euclidean distance matrix completion problem. As a special case, the approach is applied to the graph embedding problem. More generally, semi-definite programming, convex optimization, and the nuclear norm heuristic are applied to the graph embedding problem in order to extract invariants such as the chromatic number, Rn embeddability, and Borsuk-embeddability. In addition, we apply related techniques to decompose a matrix into components which simultaneously minimize a linear combination of the nuclear norm and the spectral norm. In the case when the Euclidean distance matrix is the distance matrix for a complete k-partite graph it is shown that the nuclear norm of the associated positive semidefinite matrix can be evaluated in terms of the second elementary symmetric polynomial evaluated at the partition. We prove that for k-partite graphs the maximum value of the nuclear norm of the associated positive semidefinite matrix is attained in the situation when we have equal number of vertices in each set of the partition. We use this result to determine a lower bound on the chromatic number of the graph. Finally, we describe a convex optimization approach to decomposition of a matrix into two components using the nuclear norm and spectral norm