Vision-based techniques for gait recognition

Global security concerns have raised a proliferation of video surveillance devices. Intelligent surveillance systems seek to discover possible threats automatically and raise alerts. Being able to identify the surveyed object can help determine its threat level. The current generation of devices provide digital video data to be analysed for time varying features to assist in the identification process. Commonly, people queue up to access a facility and approach a video camera in full frontal view. In this environment, a variety of biometrics are available - for example, gait which includes temporal features like stride period. Gait can be measured unobtrusively at a distance. The video data will also include face features, which are short-range biometrics. In this way, one can combine biometrics naturally using one set of data. In this paper we survey current techniques of gait recognition and modelling with the environment in which the research was conducted. We also discuss in detail the issues arising from deriving gait data, such as perspective and occlusion effects, together with the associated computer vision challenges of reliable tracking of human movement. Then, after highlighting these issues and challenges related to gait processing, we proceed to discuss the frameworks combining gait with other biometrics. We then provide motivations for a novel paradigm in biometrics-based human recognition, i.e. the use of the fronto-normal view of gait as a far-range biometrics combined with biometrics operating at a near distance

Cardiovascular information for improving biometric recognition

Mención Internacional en el título de doctorThe improvements of the last two decades in data modeling and computing have lead to new biometric modalities. The Electrocardiogram (ECG) modality is part of them, and has been mainly researched by using public databases related to medical training. Despite of being useful for initial approaches, they are not representative of a real biometric environment. In addition, publishing and creating a new database is none trivial due to human resources and data protection laws. The main goal of this thesis is to successfully use ECG as a biometric signal while getting closer to the real case scenario. Every experiment considers low computational calculations and transformations to help in potential portability. The core experiments in this work come from a private database with different positions, time and heart rate scenarios. An initial segmentation evaluation is achieved with the help of fiducial point detection which determines the QRS selection as the input data for all the experiments. The approach of training a model per user (open-set) is tested with different machine learning algorithms, only getting an acceptable result with Gaussian Mixture Models (GMM). However, the concept of training all users in one model (closed-set) shows more potential with Linear Discriminant Analysis (LDA), whose results were improved in 40%. The results with LDA are also tested as a multi-modality technique, decreasing the Equal Error Rate (EER) of fingerprint verification in up to 70.64% with score fusion, and reaching 0% in Protection Attack Detection (PAD). The Multilayer Perceptron (MLP) algorithm enhances these results in verification while applying the first differentiation to the signal. The network optimization is achieved with EER as an observation metric, and improves the results of LDA in 22% for the worst case scenario, and decreases the EER to 0% in the best case. Complexity is added creating a Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) based network, BioECG. The tuning process is achieved without extra feature transformation and is evaluated through accuracy, aiming for good identification. The inclusion of a second day of enrollment in improves results from MLP, reaching the overall lowest results of 0.009%–1.352% in EER. Throughout the use of good quality signals, position changes did not noticeably impact the verification. In addition, collecting data in a different day or in a different hour did not clearly affect the performance. Moreover, modifying the verification process based on attempts, improves the overall results, up to reach a 0% EER when applying BioECG. Finally, to get closer to a real scenario, a smartband prototype is used to collect new databases. A private database with limited scenarios but controlled data, and another local database with a wider range of scenarios and days, and with a more relaxed use of the device. Applying the concepts of first differentiation and MLP, these signals required the Stationary Wavelet Transform (SWT) and new fiducial point detection to improve their results. The first database gave subtle chances of being used in identification with up to 78.2% accuracy, but the latter was completely discarded for this purpose. These realistic experiments show the impact of a low fidelity sensor, even considering the same modifications in previous successful experiments with better quality data, reaching up to 13.530% EER. In the second database, results reach a range of 0.068%–31.669% EER. This type of sensor is affected by heart rate changes, but also by position variations, given its sensitivity to movement.Las mejoras en modelado de datos y computación de las últimas dos décadas, han llevado a la creación de nuevas modalidades biométricas. La modalidad de electrocardiograma (ECG) es una de ellas, la cual se ha investigado usando bases de datos públicas que fueron creadas para entrenamiento de profesional médico. Aunque estos datos han sido útiles para los estados iniciales de la modalidad, no son representativos de un entorno biométrico real. Además, publicar y crear bases de datos nuevas son problemas no triviales debido a los recursos humanos y las leyes de protección de datos. El principal objetivo de esta tesis es usar exitosamente datos de ECG como señales biométricas a la vez que nos acercamos a un escenario realista. Cada experimento considera cálculos y transformadas de bajo coste computacional para ayudar en su potencial uso en aparatos móviles. Los principales experimentos de este trabajo se producen con una base de datos privada con diferentes escenarios en términos de postura, tiempo y frecuencia cardíaca. Con ella se evalúan las diferentes seleccións del complejo QRS mediante detección de puntos fiduciales, lo cual servirá como datos de entrada para el resto de experimentos. El enfoque de entrenar un modelo por usuario (open-set) se prueba con diferentes algoritmos de aprendizaje máquina (machine learning), obteniendo resultados aceptables únicamente mediante el uso de modelos de mezcla de Gaussianas (Gaussian Mixture Models, GMM). Sin embargo, el concepto de entrenar un modelo con todos los usuarios (closed-set) demuestra mayor potencial con Linear Discriminant Analysis (Análisis de Discriminante Lineal, LDA), cuyos resultados mejoran en un 40%. Los resultados de LDA también se utilizan como técnica multi-modal, disminuyendo la Equal Error Rate (Tasa de Igual Error, EER) de la verificación mediante huella en hasta un 70.64% con fusión de puntuación, y llegando a un sistema con un 0% de EER en Detección de Ataques de Presentación (Presentation Attack Detection, PAD). El algoritmo de Perceptrón Multicapa (Multilayer Perceptron, MLP) mejora los resultados previos en verificación aplicando la primera derivada a la señal. La optimización de la red se consigue en base a su EER, mejora la de LDA en hasta un 22% en el peor caso, y la lleva hasta un 0% en el mejor caso. Se añade complejidad creando una red neural convolucional (Convolutional Neural Network, CNN) con una red de memoria a largo-corto plazo (Long-Short Term Memory, LSTM), llamada BioECG. El proceso de ajuste de hiperparámetros se lleva acabo sin transformaciones y se evalúa observando la accuracy (precisión), para mejorar la identificación. Sin embargo, incluir un segundo día de registro (enrollment) con BioECG, estos resultados mejoran hasta un 74% para el peor caso, llegando a los resultados más bajos hasta el momento con 0.009%–1.352% en la EER. Durante el uso de señales de buena calidad, los cambios de postura no afectaron notablemente a la verificación. Además, adquirir los datos en días u horas diferentes tampoco afectó claramente a los resultados. Asimismo, modificar el proceso de verificación en base a intentos también produce mejoría en todos los resultados, hasta el punto de llegar a un 0% de EER cuando se aplica BioECG. Finalmente, para acercarnos al caso más realista, se usa un prototipo de pulsera para capturar nuevas bases de datos. Una base de datos privada con escenarios limitados pero datos más controlados, y otra base de datos local con más espectro de escenarios y días y un uso del dispositivo más relajado. Para estos datos se aplican los conceptos de primera diferenciación en MLP, cuyas señales requieren la Transformada de Wavelet Estacionaria (Stationary Wavelet Transform, SWT) y un detector de puntos fiduciales para mejorar los resultados. La primera base de datos da opciones a ser usada para identificación con un máximo de precisión del 78.2%, pero la segunda se descartó completamente para este propósito. Estos experimentos más realistas demuestran el impact de tener un sensor de baja fidelidad, incluso considerando las mismas modificaciones que previamente tuvieron buenos resultados en datos mejores, llegando a un 13.530% de EER. En la segunda base de datos, los resultados llegan a un rango de 0.068%–31.669% en EER. Este tipo de sensor se ve afectado por las variaciones de frecuencia cardíaca, pero también por el cambio de posición, dado que es más sensible al movimiento.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Cristina Conde Vilda.- Secretario: Mariano López García.- Vocal: Young-Bin Know

Seamless Multimodal Biometrics for Continuous Personalised Wellbeing Monitoring

Artificially intelligent perception is increasingly present in the lives of every one of us. Vehicles are no exception, (...) In the near future, pattern recognition will have an even stronger role in vehicles, as self-driving cars will require automated ways to understand what is happening around (and within) them and act accordingly. (...) This doctoral work focused on advancing in-vehicle sensing through the research of novel computer vision and pattern recognition methodologies for both biometrics and wellbeing monitoring. The main focus has been on electrocardiogram (ECG) biometrics, a trait well-known for its potential for seamless driver monitoring. Major efforts were devoted to achieving improved performance in identification and identity verification in off-the-person scenarios, well-known for increased noise and variability. Here, end-to-end deep learning ECG biometric solutions were proposed and important topics were addressed such as cross-database and long-term performance, waveform relevance through explainability, and interlead conversion. Face biometrics, a natural complement to the ECG in seamless unconstrained scenarios, was also studied in this work. The open challenges of masked face recognition and interpretability in biometrics were tackled in an effort to evolve towards algorithms that are more transparent, trustworthy, and robust to significant occlusions. Within the topic of wellbeing monitoring, improved solutions to multimodal emotion recognition in groups of people and activity/violence recognition in in-vehicle scenarios were proposed. At last, we also proposed a novel way to learn template security within end-to-end models, dismissing additional separate encryption processes, and a self-supervised learning approach tailored to sequential data, in order to ensure data security and optimal performance. (...)Comment: Doctoral thesis presented and approved on the 21st of December 2022 to the University of Port

Generalized Hidden Filter Markov Models Applied to Speaker Recognition

Classification of time series has wide Air Force, DoD and commercial interest, from automatic target recognition systems on munitions to recognition of speakers in diverse environments. The ability to effectively model the temporal information contained in a sequence is of paramount importance. Toward this goal, this research develops theoretical extensions to a class of stochastic models and demonstrates their effectiveness on the problem of text-independent (language constrained) speaker recognition. Specifically within the hidden Markov model architecture, additional constraints are implemented which better incorporate observation correlations and context, where standard approaches fail. Two methods of modeling correlations are developed, and their mathematical properties of convergence and reestimation are analyzed. These differ in modeling correlation present in the time samples and those present in the processed features, such as Mel frequency cepstral coefficients. The system models speaker dependent phonemes, making use of word dictionary grammars, and recognition is based on normalized log-likelihood Viterbi decoding. Both closed set identification and speaker verification using cohorts are performed on the YOHO database. YOHO is the only large scale, multiple-session, high-quality speech database for speaker authentication and contains over one hundred speakers stating combination locks. Equal error rates of 0.21% for males and 0.31% for females are demonstrated. A critical error analysis using a hypothesis test formulation provides the maximum number of errors observable while still meeting the goal error rates of 1% False Reject and 0.1% False Accept. Our system achieves this goal

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

Learning representations of multivariate time series with missing data

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this recordLearning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical dimensionality reduction approaches are designed for vectorial data and cannot deal explicitly with missing values. In this work, we propose a novel autoencoder architecture based on recurrent neural networks to generate compressed representations of MTS. The proposed model can process inputs characterized by variable lengths and it is specifically designed to handle missing data. Our autoencoder learns fixed-length vectorial representations, whose pairwise similarities are aligned to a kernel function that operates in input space and that handles missing values. This allows to learn good representations, even in the presence of a significant amount of missing data. To show the effectiveness of the proposed approach, we evaluate the quality of the learned representations in several classification tasks, including those involving medical data, and we compare to other methods for dimensionality reduction. Successively, we design two frameworks based on the proposed architecture: one for imputing missing data and another for one-class classification. Finally, we analyze under what circumstances an autoencoder with recurrent layers can learn better compressed representations of MTS than feed-forward architectures.Norwegian Research Counci

An exploration of dynamic biometric performance using device interaction and wearable technologies

With the growth of mobile technologies and internet transactions, privacy issues and identity check became a hot topic in the past decades. Mobile biometrics provided a new level of security in addition to passwords and PIN, with a multitude of modalities to authenticate subjects. This thesis explores the verification performance of behavioural biometric modalities, as previous studies in literature proved them to be effective in identifying individual behaviours and guarantee robust continuous authentication. In addition, it addresses open issues such as single sample authentication, quality measurements for behavioural data, and fast electrocardiogram capture and biometric verification. The scope of this project is to assess the performance and stability of authentication models for mobile and wearable devices, with ceremony based tasks and a framework that includes behavioural and electrocardiogram biometrics. The results from the experiments suggest that a fast verification, appliable on real life scenarios (e.g. login or transaction request), with a single sample request and the considered modalities (Swipe gestures, PIN dynamics and electrocardiogram recording) can be performed with a stable performance. In addition, the novel fusion method implemented greatly reduced the authentication error. As additional contribution, this thesis introduces to a novel pre-processing algorithm for faulty Swipe data removal. Lastly, a theoretical framework comprised of three different modalities is proposed, based on the results of the various experiments conducted in this study. It's reasonable to state that the findings presented in this thesis will contribute to the enhancement of identity verification on mobile and wearable technologies

Biometric Systems

Biometric authentication has been widely used for access control and security systems over the past few years. The purpose of this book is to provide the readers with life cycle of different biometric authentication systems from their design and development to qualification and final application. The major systems discussed in this book include fingerprint identification, face recognition, iris segmentation and classification, signature verification and other miscellaneous systems which describe management policies of biometrics, reliability measures, pressure based typing and signature verification, bio-chemical systems and behavioral characteristics. In summary, this book provides the students and the researchers with different approaches to develop biometric authentication systems and at the same time includes state-of-the-art approaches in their design and development. The approaches have been thoroughly tested on standard databases and in real world applications

Geometric Expression Invariant 3D Face Recognition using Statistical Discriminant Models

Currently there is no complete face recognition system that is invariant to all facial expressions. Although humans find it easy to identify and recognise faces regardless of changes in illumination, pose and expression, producing a computer system with a similar capability has proved to be particularly di cult. Three dimensional face models are geometric in nature and therefore have the advantage of being invariant to head pose and lighting. However they are still susceptible to facial expressions. This can be seen in the decrease in the recognition results using principal component analysis when expressions are added to a data set. In order to achieve expression-invariant face recognition systems, we have employed a tensor algebra framework to represent 3D face data with facial expressions in a parsimonious space. Face variation factors are organised in particular subject and facial expression modes. We manipulate this using single value decomposition on sub-tensors representing one variation mode. This framework possesses the ability to deal with the shortcomings of PCA in less constrained environments and still preserves the integrity of the 3D data. The results show improved recognition rates for faces and facial expressions, even recognising high intensity expressions that are not in the training datasets. We have determined, experimentally, a set of anatomical landmarks that best describe facial expression e ectively. We found that the best placement of landmarks to distinguish di erent facial expressions are in areas around the prominent features, such as the cheeks and eyebrows. Recognition results using landmark-based face recognition could be improved with better placement. We looked into the possibility of achieving expression-invariant face recognition by reconstructing and manipulating realistic facial expressions. We proposed a tensor-based statistical discriminant analysis method to reconstruct facial expressions and in particular to neutralise facial expressions. The results of the synthesised facial expressions are visually more realistic than facial expressions generated using conventional active shape modelling (ASM). We then used reconstructed neutral faces in the sub-tensor framework for recognition purposes. The recognition results showed slight improvement. Besides biometric recognition, this novel tensor-based synthesis approach could be used in computer games and real-time animation applications