Human Gait Recognition Subject to Different Covariate Factors in a Multi-View Environment

Human gait recognition system identifies individuals based on their biometric traits. A human’s biometric features can be grouped into physiologic or behavioral traits. Biometric traits, such as the face [1], ears [2], iris [3], finger prints, passwords, and tokens, require highly accurate recognition and a well-controlled human interaction to be effective. In contrast, behavioral traits such as voice, signature, and gait do not require any human interaction and can be collected in a hidden and non-invasive mode with a camera system at a low resolution. In comparison with other physiological traits, one of the main advantages of gait analysis is the collection of data from a certain distance. However, gait is less powerful than physiological traits, yet it still has widespread application in surveillance for unfavorable situations. From traditional algorithms to deep learning models, a gait survey provides a detailed history of gait recognition

Person recognition based on deep gait: a survey.

Gait recognition, also known as walking pattern recognition, has expressed deep interest in the computer vision and biometrics community due to its potential to identify individuals from a distance. It has attracted increasing attention due to its potential applications and non-invasive nature. Since 2014, deep learning approaches have shown promising results in gait recognition by automatically extracting features. However, recognizing gait accurately is challenging due to the covariate factors, complexity and variability of environments, and human body representations. This paper provides a comprehensive overview of the advancements made in this field along with the challenges and limitations associated with deep learning methods. For that, it initially examines the various gait datasets used in the literature review and analyzes the performance of state-of-the-art techniques. After that, a taxonomy of deep learning methods is presented to characterize and organize the research landscape in this field. Furthermore, the taxonomy highlights the basic limitations of deep learning methods in the context of gait recognition. The paper is concluded by focusing on the present challenges and suggesting several research directions to improve the performance of gait recognition in the future

Geometric Invariance In The Analysis Of Human Motion In Video Data

Human motion analysis is one of the major problems in computer vision research. It deals with the study of the motion of human body in video data from different aspects, ranging from the tracking of body parts and reconstruction of 3D human body configuration, to higher level of interpretation of human action and activities in image sequences. When human motion is observed through video camera, it is perspectively distorted and may appear totally different from different viewpoints. Therefore it is highly challenging to establish correct relationships between human motions across video sequences with different camera settings. In this work, we investigate the geometric invariance in the motion of human body, which is critical to accurately understand human motion in video data regardless of variations in camera parameters and viewpoints. In human action analysis, the representation of human action is a very important issue, and it usually determines the nature of the solutions, including their limits in resolving the problem. Unlike existing research that study human motion as a whole 2D/3D object or a sequence of postures, we study human motion as a sequence of body pose transitions. We also decompose a human body pose further into a number of body point triplets, and break down a pose transition into the transition of a set of body point triplets. In this way the study of complex non-rigid motion of human body is reduced to that of the motion of rigid body point triplets, i.e. a collection of planes in motion. As a result, projective geometry and linear algebra can be applied to explore the geometric invariance in human motion. Based on this formulation, we have discovered the fundamental ratio invariant and the eigenvalue equality invariant in human motion. We also propose solutions based on these geometric invariants to the problems of view-invariant recognition of human postures and actions, as well as analysis of human motion styles. These invariants and their applicability have been validated by experimental results supporting that their effectiveness in understanding human motion with various camera parameters and viewpoints

Towards Practical and Secure Channel Impulse Response-based Physical Layer Key Generation

Der derzeitige Trend hin zu “smarten” Geräten bringt eine Vielzahl an Internet-fähigen und verbundenen Geräten mit sich. Die entsprechende Kommunikation dieser Geräte muss zwangsläufig durch geeignete Maßnahmen abgesichert werden, um die datenschutz- und sicherheitsrelevanten Anforderungen an die übertragenen Informationen zu erfüllen. Jedoch zeigt die Vielzahl an sicherheitskritischen Vorfällen im Kontext von “smarten” Geräten und des Internets der Dinge auf, dass diese Absicherung der Kommunikation derzeit nur unzureichend umgesetzt wird. Die Ursachen hierfür sind vielfältig: so werden essentielle Sicherheitsmaßnahmen im Designprozess mitunter nicht berücksichtigt oder auf Grund von Preisdruck nicht realisiert. Darüber hinaus erschwert die Beschaffenheit der eingesetzten Geräte die Anwendung klassischer Sicherheitsverfahren. So werden in diesem Kontext vorrangig stark auf Anwendungsfälle zugeschnittene Lösungen realisiert, die auf Grund der verwendeten Hardware meist nur eingeschränkte Rechen- und Energieressourcen zur Verfügung haben. An dieser Stelle können die Ansätze und Lösungen der Sicherheit auf physikalischer Schicht (physical layer security, PLS) eine Alternative zu klassischer Kryptografie bieten. Im Kontext der drahtlosen Kommunikation können hier die Eigenschaften des Übertragungskanals zwischen zwei legitimen Kommunikationspartnern genutzt werden, um Sicherheitsprimitive zu implementieren und damit Sicherheitsziele zu realisieren. Konkret können etwa reziproke Kanaleigenschaften verwendet werden, um einen Vertrauensanker in Form eines geteilten, symmetrischen Geheimnisses zu generieren. Dieses Verfahren wird Schlüsselgenerierung basierend auf Kanalreziprozität (channel reciprocity based key generation, CRKG) genannt. Auf Grund der weitreichenden Verfügbarkeit wird dieses Verfahren meist mit Hilfe der Kanaleigenschaft des Empfangsstärkenindikators (received signal strength indicator, RSSI) realisiert. Dies hat jedoch den Nachteil, dass alle physikalischen Kanaleigenschaften auf einen einzigen Wert heruntergebrochen werden und somit ein Großteil der verfügbaren Informationen vernachlässigt wird. Dem gegenüber steht die Verwendung der vollständigen Kanalzustandsinformationen (channel state information, CSI). Aktuelle technische Entwicklungen ermöglichen es zunehmend, diese Informationen auch in Alltagsgeräten zur Verfügung zu stellen und somit für PLS weiterzuverwenden. In dieser Arbeit analysieren wir Fragestellungen, die sich aus einem Wechsel hin zu CSI als verwendetes Schlüsselmaterial ergeben. Konkret untersuchen wir CSI in Form von Ultrabreitband-Kanalimpulsantworten (channel impulse response, CIR). Für die Untersuchungen haben wir initial umfangreiche Messungen vorgenommen und damit analysiert, in wie weit die grundlegenden Annahmen von PLS und CRKG erfüllt sind und die CIRs sich grundsätzlich für die Schlüsselgenerierung eignen. Hier zeigen wir, dass die CIRs der legitimen Kommunikationspartner eine höhere Ähnlichkeit als die eines Angreifers aufzeigen und das somit ein Vorteil gegenüber diesem auf der physikalischen Schicht besteht, der für die Schlüsselgenerierung ausgenutzt werden kann. Basierend auf den Ergebnissen der initialen Untersuchung stellen wir dann grundlegende Verfahren vor, die notwendig sind, um die Ähnlichkeit der legitimen Messungen zu verbessern und somit die Schlüsselgenerierung zu ermöglichen. Konkret werden Verfahren vorgestellt, die den zeitlichen Versatz zwischen reziproken Messungen entfernen und somit die Ähnlichkeit erhöhen, sowie Verfahren, die das in den Messungen zwangsläufig vorhandene Rauschen entfernen. Gleichzeitig untersuchen wir, inwieweit die getroffenen fundamentalen Sicherheitsannahmen aus Sicht eines Angreifers erfüllt sind. Zu diesem Zweck präsentieren, implementieren und analysieren wir verschiedene praktische Angriffsmethoden. Diese Verfahren umfassen etwa Ansätze, bei denen mit Hilfe von deterministischen Kanalmodellen oder durch ray tracing versucht wird, die legitimen CIRs vorherzusagen. Weiterhin untersuchen wir Machine Learning Ansätze, die darauf abzielen, die legitimen CIRs direkt aus den Beobachtungen eines Angreifers zu inferieren. Besonders mit Hilfe des letzten Verfahrens kann hier gezeigt werden, dass große Teile der CIRs deterministisch vorhersagbar sind. Daraus leitet sich der Schluss ab, dass CIRs nicht ohne adäquate Vorverarbeitung als Eingabe für Sicherheitsprimitive verwendet werden sollten. Basierend auf diesen Erkenntnissen entwerfen und implementieren wir abschließend Verfahren, die resistent gegen die vorgestellten Angriffe sind. Die erste Lösung baut auf der Erkenntnis auf, dass die Angriffe aufgrund von vorhersehbaren Teilen innerhalb der CIRs möglich sind. Daher schlagen wir einen klassischen Vorverarbeitungsansatz vor, der diese deterministisch vorhersagbaren Teile entfernt und somit das Eingabematerial absichert. Wir implementieren und analysieren diese Lösung und zeigen ihre Effektivität sowie ihre Resistenz gegen die vorgeschlagenen Angriffe. In einer zweiten Lösung nutzen wir die Fähigkeiten des maschinellen Lernens, indem wir sie ebenfalls in das Systemdesign einbringen. Aufbauend auf ihrer starken Leistung bei der Mustererkennung entwickeln, implementieren und analysieren wir eine Lösung, die lernt, die zufälligen Teile aus den rohen CIRs zu extrahieren, durch die die Kanalreziprozität definiert wird, und alle anderen, deterministischen Teile verwirft. Damit ist nicht nur das Schlüsselmaterial gesichert, sondern gleichzeitig auch der Abgleich des Schlüsselmaterials, da Differenzen zwischen den legitimen Beobachtungen durch die Merkmalsextraktion effizient entfernt werden. Alle vorgestellten Lösungen verzichten komplett auf den Austausch von Informationen zwischen den legitimen Kommunikationspartnern, wodurch der damit verbundene Informationsabfluss sowie Energieverbrauch inhärent vermieden wird

Bridging Domain Gaps for Cross-Spectrum and Long-Range Face Recognition Using Domain Adaptive Machine Learning

Face recognition technology has witnessed significant advancements in recent decades, enabling its widespread adoption in various applications such as security, surveillance, and biometrics applications. However, one of the primary challenges faced by existing face recognition systems is their limited performance when presented with images from different modalities or domains( such as infrared to visible, long range to close range, nighttime to daytime, profile to f rontal, e tc.) Additionally, advancements in camera sensors, analytics beyond the visible spectrum, and the increasing size of cross-modal datasets have led to a particular interest in cross-modal learning for face recognition in the biometrics and computer vision community. Despite a relatively large gap between source and target domains, existing approaches reduce or bridge such domain gaps by either synthesizing face imagery in the target domain using face imagery from the source domain, or by learning cross-modal image representations that are robust to both the source and the target domain. Therefore, this dissertation presents the design and implementation of a novel domain adaptation framework leveraging robust image representations to achieve state-of-the art performance in cross-spectrum and long-range face recognition. The proposed methods use machine learning and deep learning techniques to (1) efficiently ex tract an d le arn do main-invariant embedding from face imagery, (2) learn a mapping from the source to the target domain, and (3) evaluate the proposed framework on several cross-modal face datasets

Identification and evaluation of biomarkers for Huntington’s disease

Huntington’s disease (HD) is a devastating, incurable inherited neurodegenerative disorder that commonly affects adults in mid-life. Despite encouraging results from in vitro and animal trials, disease-modifying therapeutic trials in HD are limited by a lack of tools to track disease progression. HD is clinically heterogeneous, and current clinical rating scales lack sensitivity and specificity, particularly over relatively short time periods. Improvements in the precision of objective measurement of disease progression in HD could lead to state markers (biomarkers) better able to predict onset, detect progression and measure the effects of therapeutic intervention. Biomarkers capable of detecting disease-related changes in premanifest gene carriers will be essential for clinical trials of treatments to delay onset. Imaging, clinical and cognitive assessment as well as laboratory markers have all been proposed as biomarkers, but few measures have been quantified over short time intervals or shown to be predictive of clinical change over longer periods. A robust panel of biomarkers from a number of modalities will be necessary to progress to interventional clinical trials of disease-modifying therapies in HD, using biomarkers to measure the success or failure of an intervention. Such cross-validation requires simultaneous multimodal biomarker evaluation within a suitable cohort of subjects studied longitudinally. This thesis describes a multi-modal approach to the discovery and evaluation of potential biomarkers for Huntington's disease in a large cohort of human volunteers. After reviewing the relevant features of Huntington's disease and current state of biomarker research in Huntington's disease, several approaches to, and outcomes from, biomarker discovery and evaluation are described, including proteomic profiling, targeted ELISA, multiplex inflammatory profiling and measurement of whole-brain atrophy by longitudinal magnetic resonance imaging. The thesis draws together these different approaches and summarises the contributions to both biomarker research and our understanding of the neurobiology of HD that the work has generated

Multimodal Biometric Systems for Personal Identification and Authentication using Machine and Deep Learning Classifiers

Multimodal biometrics, using machine and deep learning, has recently gained interest over single biometric modalities. This interest stems from the fact that this technique improves recognition and, thus, provides more security. In fact, by combining the abilities of single biometrics, the fusion of two or more biometric modalities creates a robust recognition system that is resistant to the flaws of individual modalities. However, the excellent recognition of multimodal systems depends on multiple factors, such as the fusion scheme, fusion technique, feature extraction techniques, and classification method. In machine learning, existing works generally use different algorithms for feature extraction of modalities, which makes the system more complex. On the other hand, deep learning, with its ability to extract features automatically, has made recognition more efficient and accurate. Studies deploying deep learning algorithms in multimodal biometric systems tried to find a good compromise between the false acceptance and the false rejection rates (FAR and FRR) to choose the threshold in the matching step. This manual choice is not optimal and depends on the expertise of the solution designer, hence the need to automatize this step. From this perspective, the second part of this thesis details an end-to-end CNN algorithm with an automatic matching mechanism. This thesis has conducted two studies on face and iris multimodal biometric recognition. The first study proposes a new feature extraction technique for biometric systems based on machine learning. The iris and facial features extraction is performed using the Discrete Wavelet Transform (DWT) combined with the Singular Value Decomposition (SVD). Merging the relevant characteristics of the two modalities is used to create a pattern for an individual in the dataset. The experimental results show the robustness of our proposed technique and the efficiency when using the same feature extraction technique for both modalities. The proposed method outperformed the state-of-the-art and gave an accuracy of 98.90%. The second study proposes a deep learning approach using DensNet121 and FaceNet for iris and faces multimodal recognition using feature-level fusion and a new automatic matching technique. The proposed automatic matching approach does not use the threshold to ensure a better compromise between performance and FAR and FRR errors. However, it uses a trained multilayer perceptron (MLP) model that allows people’s automatic classification into two classes: recognized and unrecognized. This platform ensures an accurate and fully automatic process of multimodal recognition. The results obtained by the DenseNet121-FaceNet model by adopting feature-level fusion and automatic matching are very satisfactory. The proposed deep learning models give 99.78% of accuracy, and 99.56% of precision, with 0.22% of FRR and without FAR errors. The proposed and developed platform solutions in this thesis were tested and vali- dated in two different case studies, the central pharmacy of Al-Asria Eye Clinic in Dubai and the Abu Dhabi Police General Headquarters (Police GHQ). The solution allows fast identification of the persons authorized to access the different rooms. It thus protects the pharmacy against any medication abuse and the red zone in the military zone against the unauthorized use of weapons

Neuromuscular Control Modeling: from Physics to Data-Driven Approaches

Il controllo neurale della postura umana è stato investigato a partire da un punto di vista fisico. Il paradigma di controllo intermittente è stato usato allo scopo di capire il peso di quest'ultimo nella generazione delle traiettorie del centro di pressione. Un primo contributo di questo lavoro rigurda quindi l'analisi del centro di pressione generato dal suddetto modello biomeccanico attraverso l'extended detrended fluctuation analysis, recentemente proposta in letteratura. Le proprietà di correlazione a lungo termine e disomogeneità sono risultate strettamente legate al guadagno derivativo del modello di controllo intermittente e anche al grado di intermittenza. Il paradigma di controllo è stato poi esteso verso un sistema biomeccanico più complesso, cioè un pendolo inverso doppio link con controllo intermittente alla caviglia. I contributi più significativi hanno riguardato la modellazione matematica del centro di pressione per una struttura multi-link e la verifica della sua plausibilità fisiologica. Si è poi preso in considerazione il caso della postura perturbata, integrando aspetti cinematici, dinamici e relativi all'attività muscolare. A tal fine, si è utilizzato sia un approccio fisico che basato su dati per l'identificazione dei modelli a struttura variabile Si sono prese in considerazione differenti condizioni di sperimentali, e in tutti i casi l'approccio utilizzato ha garantito un adeguato grado di interpretabilità riguardo il ruolo del sistema nervoso centrale nella regolazione del postura eretta in condizioni perturbate. La seconda parte della tesi ha riguardato la caratterizzazione del controllo motorio attraverso il segnale elettromiografico di superfice. Il primo contributo ha riguardato l'identifcazione dell'onset muscolare in condizioni di basso rapporto segnale rumore, sfruttando operatori energetico di tipo Teager-Kaiser al fine del precondizionamento del segnale mioelettrico. La versioe estesa di questo tipo di operatori è risultata particolarmente utile al miglioramento delle performance di numerosi algoritmi di detection. Si è poi proseguito con l'utilizzo di tali segnali al fine della classificazione dei gesti dell'arto superiore. In particoalre si è prerso in considerazione il problema della motion intention detection dei principali movimenti della spalla , utilizzando sia descrittori del segnale elettromiografico nel dominio del tempo e della frequenza. Quest'ultimo aspetto risulta essere un elemento di novità nel contesto scientifico in quanto si sono considerati il riconoscimento l'intezioni di movimento di otto gesti della spalla con particolare attenzione al ruolo dei descrittori del segnale per la classificazione. Infine, con approcci simili, si è preso in considerazione il problema del riconoscimento della scrittura manuale a partire dal dato elettromiografico. Tale aspetto risulta poco investigato sotto la prospettiva della pattern recognition mioelettrica, ma la sua valenza è data dalla crescente richiesta di interfacce uomo-macchina per compiti riabilitativi che coinvolgono una componente cognitiva significativa, Inoltre, vista la tendenza ad investigare il ruolo del polso per il prelievo del segnale elettromiografico al fine della realizzazione delle suddette interfacce, si è analizzato l'utilizzo dei segnali elettromiografici del polso rispetto a quelli dell'avambraccio al fine di predirre le cifre scritte dall'utente, noto che l'avambraccio risulta essere la zona di prelievo più comunemente utilizzata.The biomechanics and the neural control of the human stance was investigated starting from a physical point of view. In particular the intermittent motor control paradigm was investigated with the aim of understanding how such paradigm mirrors in the center of pressure (COP) trajectories. A first contribution given in this work of thesis regards the analysis of COP generated from intermittent controlled inverted pendulum through the extended detrended fluctuation analysis, which was recently introduced in the literature. It has been found that the long-term correlation and inohmogeniety properties of the COP time series strictly depend on the derivative gain term of the intermittent controller and on the degree of intermittency of the control action. Thus, , another contribution provided in this work of thesis regards the use of a more complex biomechanical model of the stance, e.g. a double-link inverted pendulum intermittently controlled at the ankle. In terms of novelty, it deserves to be pointed out the results regarding the mechanical modeling of the COP for a multi-link structure, and the assessment of its physiological plausibility. . On the other hand, when the perturbed posture motor task was taken into account, there was the need to enlarge the perspective, integrating kinematic, dynamic and muscle activity data. The idea of employing different sources of information to develop models of the CNS represents an important element that was investigated using tools related to hybrid system identification theory. Subjects underwent to impulsive support base translations in three different conditions: considering eyes open, closed, and performing mental counting. Although such data were essentially analyzed through a data-driven approach, the identified models guaranteed physical interpretations of the role played by the CNS in the three different conditions. The second main core of this thesis regards the characterization of the motor control using the surface electromyographic (sEMG) signals. A first contribution given in this work regarded the muscle onset detection considering low SNR scenarios. In this framework, energy operators such as the Teager-Kaiser energy operator (TKEO) and its extended version (ETKEO) were investigated as signal preconditioning steps before the application of state of the art onset detection algorithms. The latter have been significantly boosted when ETKEO was used with respect to TKEO. The use of extended energy operators for the sEMG signal preprocessing constitutes a novel element in this field that can be also further investigated in future studies. From the sEMG muscle, one can also predict which movement the subject is going to perform. This aspect can be enclosed in the motion intention detection (MID) field. In this thesis a MID problem was investigated by taking into account two important aspects: as first the study was centered on the shoulder joint movements. Secondly, the MID problem was faced under a pattern recognition perspective. This allowed to verify whether methodologies encountered in the myoelectric hand gesture recognition can be transferred in the affine field of MID In contrast to what reported in the literature, where MID problems generally consider only few movements, in this work of thesis up to eight shoulder movements have been investigated. Myoelectric pattern recognition architectures were also used in the assessment of the ten hand-written digits. Despite the handwriting can be considered a hand movement that involves fine muscular control actions, it has not been consistently investigated in the field of sEMG based hand gesture recognition. Further, since the literature supports the change from forearm to wrist in order to acquire EMG data for hand gesture recognition, it was investigated whether such exchange can be performed when a challenging classification task, as the handwriting recognition has to be performed