Verificação facial em duas etapas para dispositivos móveis

Orientadores: Jacques Wainer, Fernanda Alcântara AndalóDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Dispositivos móveis, como smartphones e tablets, se tornaram mais populares e acessíveis nos últimos anos. Como consequência de sua ubiquidade, esses aparelhos guardam diversos tipos de informações pessoais (fotos, conversas de texto, coordenadas GPS, dados bancários, entre outros) que só devem ser acessadas pelo dono do dispositivo. Apesar de métodos baseados em conhecimento, como senhas numéricas ou padrões, ainda estejam entre as principais formas de assegurar a identidade do usuário, traços biométricos tem sido utilizados para garantir uma autenticação mais segura e prática. Entre eles, reconhecimento facial ganhou atenção nos últimos anos devido aos recentes avanços nos dispositivos de captura de imagens e na crescente disponibilidade de fotos em redes sociais. Aliado a isso, o aumento de recursos computacionais, com múltiplas CPUs e GPUs, permitiu o desenvolvimento de modelos mais complexos e robustos, como redes neurais profundas. Porém, apesar da evolução das capacidades de dispositivos móveis, os métodos de reconhecimento facial atuais ainda não são desenvolvidos considerando as características do ambiente móvel, como processamento limitado, conectividade instável e consumo de bateria. Neste trabalho, nós propomos um método de verificação facial otimizado para o ambiente móvel. Ele consiste em um procedimento em dois níveis que combina engenharia de características (histograma de gradientes orientados e análise de componentes principais por regiões) e uma rede neural convolucional para verificar se o indivíduo presente em uma imagem corresponde ao dono do dispositivo. Nós também propomos a \emph{Hybrid-Fire Convolutional Neural Network}, uma arquitetura ajustada para dispositivos móveis que processa informação de pares de imagens. Finalmente, nós apresentamos uma técnica para adaptar o limiar de aceitação do método proposto para imagens com características diferentes daquelas presentes no treinamento, utilizando a galeria de imagens do dono do dispositivo. A solução proposta se compara em acurácia aos métodos de reconhecimento facial do estado da arte, além de possuir um modelo 16 vezes menor e 4 vezes mais rápido ao processar uma imagem em smartphones modernos. Por último, nós também organizamos uma base de dados composta por 2873 selfies de 56 identidades capturadas em condições diversas, a qual esperamos que ajude pesquisas futuras realizadas neste cenárioAbstract: Mobile devices, such as smartphones and tablets, had their popularity and affordability greatly increased in recent years. As a consequence of their ubiquity, these devices now carry all sorts of personal data (\emph{e.g.} photos, text conversations, GPS coordinates, banking information) that should be accessed only by the device's owner. Even though knowledge-based procedures, such as entering a PIN or drawing a pattern, are still the main methods to secure the owner's identity, recently biometric traits have been employed for a more secure and effortless authentication. Among them, face recognition has gained more attention in past years due to recent improvements in image-capturing devices and the availability of images in social networks. In addition to that, the increase in computational resources, with multiple CPUs and GPUs, enabled the design of more complex and robust models, such as deep neural networks. Although the capabilities of mobile devices have been growing in past years, most recent face recognition techniques are still not designed considering the mobile environment's characteristics, such as limited processing power, unstable connectivity and battery consumption. In this work, we propose a facial verification method optimized to the mobile environment. It consists of a two-tiered procedure that combines hand-crafted features (histogram of oriented gradients and local region principal component analysis) and a convolutional neural network to verify if the person depicted in a picture corresponds to the device owner. We also propose \emph{Hybrid-Fire Convolutional Neural Network}, an architecture tweaked for mobile devices that process encoded information of a pair of face images. Finally, we expose a technique to adapt our method's acceptance thresholds to images with different characteristics than those present during training, by using the device owner's enrolled gallery. The proposed solution performs a par to the state-of-the-art face recognition methods, while having a model 16 times smaller and 4 times faster when processing an image in recent smartphone models. Finally, we have collected a new dataset of selfie pictures comprising 2873 images from 56 identities with varied capture conditions, that hopefully will support future researches in this scenarioMestradoCiência da ComputaçãoMestre em Ciência da Computaçã

A Review of Techniques on Gait-Based Person Re-Identification

Copyright (c) 2023 Babak Rahi, Maozhen Li and Man Qi. Person re-identification at a distance across multiple non-overlapping cameras has been an active research area for years. In the past ten years, short-term Person re-identification techniques have made great strides in accuracy using only appearance features in limited environments. However, massive intra-class variations and inter-class confusion limit their ability to be used in practical applications. Moreover, appearance consistency can only be assumed in a short time span from one camera to the other. Since the holistic appearance will change drastically over days and weeks, the technique, as mentioned above, will be ineffective. Practical applications usually require a long-term solution in which the subject's appearance and clothing might have changed after the elapse of a significant period. Facing these problems, soft biometric features such as Gait has stirred much interest in the past years. Nevertheless, even Gait can vary with illness, ageing and emotional states, walking surfaces, shoe types, clothes types, carried objects (by the subject) and even environment clutters. Therefore, Gait is considered as a temporal cue that could provide biometric motion information. On the other hand, the shape of the human body could be viewed as a spatial signal which can produce valuable information. So extracting discriminative features from both spatial and temporal domains would benefit this research. This article examines the main approaches used in gait analysis for re-identification over the past decade. We identify several relevant dimensions of the problem and provide a taxonomic analysis of current research. We conclude by reviewing the performance levels achievable with current technology and providing a perspective on the most challenging and promising research directions.This research received no external funding

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

The automatic detection of eye positions, their temporal consistency, and their mapping into a line of sight in the real world (to find where a person is looking at) is reported in the scientific literature as gaze tracking. This has become a very hot topic in the field of computer vision during the last decades, with a surprising and continuously growing number of application fields. A very long journey has been made from the first pioneering works, and this continuous search for more accurate solutions process has been further boosted in the last decade when deep neural networks have revolutionized the whole machine learning area, and gaze tracking as well. In this arena, it is being increasingly useful to find guidance through survey/review articles collecting most relevant works and putting clear pros and cons of existing techniques, also by introducing a precise taxonomy. This kind of manuscripts allows researchers and technicians to choose the better way to move towards their application or scientific goals. In the literature, there exist holistic and specifically technological survey documents (even if not updated), but, unfortunately, there is not an overview discussing how the great advancements in computer vision have impacted gaze tracking. Thus, this work represents an attempt to fill this gap, also introducing a wider point of view that brings to a new taxonomy (extending the consolidated ones) by considering gaze tracking as a more exhaustive task that aims at estimating gaze target from different perspectives: from the eye of the beholder (first-person view), from an external camera framing the beholder’s, from a third-person view looking at the scene where the beholder is placed in, and from an external view independent from the beholder

Métodos objetivos estadísticos de valoración de la magnitud de interés en base a las trayectorias de dirección de visionado

Aplicat embargament des de la data de defensa fins el dia 1 de gener de 2022In this work, a new method is presented to quantify the attention paid by people to the different objects in their environment. The new metric models the eye to determine the attention given and quantifies that attention at all points in an area of interest. This metric is compared with the current one based on time and it is justified that ours is more in line with human perception. For the calculation of said attention, the concept of oriented trajectory is introduced as the set of positions and orientation angles of the head, of each person of interest and in the time that is of interest. We will justify that only with this data can such care be determined. In the presented method, top view cameras are used as a system to have the highest performance with the minimum number of cameras. Likewise, two methods are analyzed: the 3D method that uses depth information, and a less precise method, 2D, that only uses imaging cameras. This thesis also presents a way of calculating the time metric, a method that is widely used today to verify how many people and in how long an ad has been seen. The form presented by our method allows reducing the number of cameras required, and therefore it is advantageous in terms of the resources required for its implementation. Finally, the results are verified using a camera in the front part of the head simulating the eye and an IMU sensor that measures the angles of the head. In this way, the attention relationship of the objects detected by the camera is determined, and the same attention relationship of the objects obtained by the proposed method.En este trabajo se presenta un nuevo método para cuantificar la atención prestada por las personas en los diferentes objetos de su entorno. La nueva métrica modeliza el ojo para determinar la atención prestada y cuantifica dicha atención en todos los puntos de una zona de interés. Se compara esta métrica con la actual basada en tiempo y se justifica que la nuestra se ajusta más a la percepción humana. Para el cálculo de dicha atención se introduce el concepto de trayectoria orientada como el conjunto de posiciones y de ángulos de orientación de la cabeza, de cada persona de interés y en el tiempo que sea de interés. Justificaremos que solo con estos datos se puede determinar dicha atención. En el método presentado se utilizan cámaras cenitales como sistema de tener las mayores prestaciones con el mínimo número de cámaras. Así mismo se analizan dos métodos: el método 3D que utiliza la información de profundidad, y un método menos preciso, el 2D, que solo utiliza cámaras de imagen. Esta tesis también presenta una forma de calcular la métrica de tiempos, método que se utiliza ampliamente en la actualidad para verificar cuantas personas y en cuánto tiempo se ha visto un anuncio La forma presentada por nuestro método permite reducir el número de cámaras necesarias, y por tanto es ventajosa en cuanto a los recursos que requiere para su implementación. Finalmente se verifican los resultados utilizando una cámara en la parte frontal de la cabeza simulando al ojo y un sensor IMU que mide los ángulos de la cabeza. De esta manera se determina la relación de atención de los objetos detectados por la cámara, y la misma relación de atención de los objetos obtenidos por el método propuesto.Postprint (published version