A Multimodal and Multi-Algorithmic Architecture for Data Fusion in Biometric Systems

Software di autenticazione basato su tratti biometric

Signal processing and machine learning techniques for human verification based on finger textures

PhD ThesisIn recent years, Finger Textures (FTs) have attracted considerable attention as potential biometric characteristics. They can provide robust recognition performance as they have various human-speci c features, such as wrinkles and apparent lines distributed along the inner surface of all ngers. The main topic of this thesis is verifying people according to their unique FT patterns by exploiting signal processing and machine learning techniques. A Robust Finger Segmentation (RFS) method is rst proposed to isolate nger images from a hand area. It is able to detect the ngers as objects from a hand image. An e cient adaptive nger segmentation method is also suggested to address the problem of alignment variations in the hand image called the Adaptive and Robust Finger Segmentation (ARFS) method. A new Multi-scale Sobel Angles Local Binary Pattern (MSALBP) feature extraction method is proposed which combines the Sobel direction angles with the Multi-Scale Local Binary Pattern (MSLBP). Moreover, an enhanced method called the Enhanced Local Line Binary Pattern (ELLBP) is designed to e ciently analyse the FT patterns. As a result, a powerful human veri cation scheme based on nger Feature Level Fusion with a Probabilistic Neural Network (FLFPNN) is proposed. A multi-object fusion method, termed the Finger Contribution Fusion Neural Network (FCFNN), combines the contribution scores of the nger objects. The veri cation performances are examined in the case of missing FT areas. Consequently, to overcome nger regions which are poorly imaged a method is suggested to salvage missing FT elements by exploiting the information embedded within the trained Probabilistic Neural Network (PNN). Finally, a novel method to produce a Receiver Operating Characteristic (ROC) curve from a PNN is suggested. Furthermore, additional development to this method is applied to generate the ROC graph from the FCFNN. Three databases are employed for evaluation: The Hong Kong Polytechnic University Contact-free 3D/2D (PolyU3D2D), Indian Institute of Technology (IIT) Delhi and Spectral 460nm (S460) from the CASIA Multi-Spectral (CASIAMS) databases. Comparative simulation studies con rm the e ciency of the proposed methods for human veri cation. The main advantage of both segmentation approaches, the RFS and ARFS, is that they can collect all the FT features. The best results have been benchmarked for the ELLBP feature extraction with the FCFNN, where the best Equal Error Rate (EER) values for the three databases PolyU3D2D, IIT Delhi and CASIAMS (S460) have been achieved 0.11%, 1.35% and 0%, respectively. The proposed salvage approach for the missing feature elements has the capability to enhance the veri cation performance for the FLFPNN. Moreover, ROC graphs have been successively established from the PNN and FCFNN.the ministry of higher education and scientific research in Iraq (MOHESR); the Technical college of Mosul; the Iraqi Cultural Attach e; the active people in the MOHESR, who strongly supported Iraqi students

Heterogeneous Sensor Signal Processing for Inference with Nonlinear Dependence

Inferring events of interest by fusing data from multiple heterogeneous sources has been an interesting and important topic in recent years. Several issues related to inference using heterogeneous data with complex and nonlinear dependence are investigated in this dissertation. We apply copula theory to characterize the dependence among heterogeneous data. In centralized detection, where sensor observations are available at the fusion center (FC), we study copula-based fusion. We design detection algorithms based on sample-wise copula selection and mixture of copulas model in different scenarios of the true dependence. The proposed approaches are theoretically justified and perform well when applied to fuse acoustic and seismic sensor data for personnel detection. Besides traditional sensors, the access to the massive amount of social media data provides a unique opportunity for extracting information about unfolding events. We further study how sensor networks and social media complement each other in facilitating the data-to-decision making process. We propose a copula-based joint characterization of multiple dependent time series from sensors and social media. As a proof-of-concept, this model is applied to the fusion of Google Trends (GT) data and stock/flu data for prediction, where the stock/flu data serves as a surrogate for sensor data. In energy constrained networks, local observations are compressed before they are transmitted to the FC. In these cases, conditional dependence and heterogeneity complicate the system design particularly. We consider the classification of discrete random signals in Wireless Sensor Networks (WSNs), where, for communication efficiency, only local decisions are transmitted. We derive the necessary conditions for the optimal decision rules at the sensors and the FC by introducing a hidden random variable. An iterative algorithm is designed to search for the optimal decision rules. Its convergence and asymptotical optimality are also proved. The performance of the proposed scheme is illustrated for the distributed Automatic Modulation Classification (AMC) problem. Censoring is another communication efficient strategy, in which sensors transmit only informative observations to the FC, and censor those deemed uninformative . We design the detectors that take into account the spatial dependence among observations. Fusion rules for censored data are proposed with continuous and discrete local messages, respectively. Their computationally efficient counterparts based on the key idea of injecting controlled noise at the FC before fusion are also investigated. In this thesis, with heterogeneous and dependent sensor observations, we consider not only inference in parallel frameworks but also the problem of collaborative inference where collaboration exists among local sensors. Each sensor forms coalition with other sensors and shares information within the coalition, to maximize its inference performance. The collaboration strategy is investigated under a communication constraint. To characterize the influence of inter-sensor dependence on inference performance and thus collaboration strategy, we quantify the gain and loss in forming a coalition by introducing the copula-based definitions of diversity gain and redundancy loss for both estimation and detection problems. A coalition formation game is proposed for the distributed inference problem, through which the information contained in the inter-sensor dependence is fully explored and utilized for improved inference performance

Vision-based techniques for gait recognition

Global security concerns have raised a proliferation of video surveillance devices. Intelligent surveillance systems seek to discover possible threats automatically and raise alerts. Being able to identify the surveyed object can help determine its threat level. The current generation of devices provide digital video data to be analysed for time varying features to assist in the identification process. Commonly, people queue up to access a facility and approach a video camera in full frontal view. In this environment, a variety of biometrics are available - for example, gait which includes temporal features like stride period. Gait can be measured unobtrusively at a distance. The video data will also include face features, which are short-range biometrics. In this way, one can combine biometrics naturally using one set of data. In this paper we survey current techniques of gait recognition and modelling with the environment in which the research was conducted. We also discuss in detail the issues arising from deriving gait data, such as perspective and occlusion effects, together with the associated computer vision challenges of reliable tracking of human movement. Then, after highlighting these issues and challenges related to gait processing, we proceed to discuss the frameworks combining gait with other biometrics. We then provide motivations for a novel paradigm in biometrics-based human recognition, i.e. the use of the fronto-normal view of gait as a far-range biometrics combined with biometrics operating at a near distance

Hypothesis Testing and Model Estimation with Dependent Observations in Heterogeneous Sensor Networks

Advances in microelectronics, communication and signal processing have enabled the development of inexpensive sensors that can be networked to collect vital information from their environment to be used in decision-making and inference. The sensors transmit their data to a central processor which integrates the information from the sensors using a so-called fusion algorithm. Many applications of sensor networks (SNs) involve hypothesis testing or the detection of a phenomenon. Many approaches to data fusion for hypothesis testing assume that, given each hypothesis, the sensors\u27 measurements are conditionally independent. However, since the sensors are densely deployed in practice, their field of views overlap and consequently their measurements are dependent. Moreover, a sensor\u27s measurement samples may be correlated over time. Another assumption often used in data fusion algorithms is that the underlying statistical model of sensors\u27 observations is completely known. However, in practice these statistics may not be available prior to deployment and may change over the lifetime of the network due to hardware changes, aging, and environmental conditions. In this dissertation, we consider the problem of data fusion in heterogeneous SNs (SNs in which the sensors are not identical) collecting dependent data. We develop the expectation maximization algorithm for hypothesis testing and model estimation. Copula distributions are used to model the correlation in the data. Moreover, it is assumed that the distribution of the sensors\u27 measurements is not completely known. we consider both parametric and non-parametric model estimation. The proposed approach is developed for both batch and online processing. In batch processing, fusion can only be performed after a block of data samples is received from each sensor, while in online processing, fusion is performed upon arrival of each data sample. Online processing is of great interest since for many applications, the long delay required for the accumulation of data in batch processing is not acceptable. To evaluate the proposed algorithms, both simulation data and real-world datasets are used. Detection performances of the proposed algorithms are compared with well-known supervised and unsupervised learning methods as well as with similar EM-based methods, which either partially or entirely ignore the dependence in the data

Analysis Of Data Stratification In A Multi-Sensor Fingerprint Dataset Using Match Score Statistics

Biometric data is an essential feature employed in testing the performance of any real time biometric recognition system prior to its usage. The variations introduced in the match performance critically determine the authenticity of the biometric data to be able to be used in an everyday scenario for the testing of biometric verification systems. This study in totality aims at understanding the impact of data stratification of a such a biometric test dataset on the match performance of each of its stratum. In order to achieve this goal, the fingerprint dataset of the West Virginia University\u27s 2012 BioCOP has been employed which is a part of the many multimodal biometric data collection projects that the University has accomplished. This test dataset has been initially segmented based on the scanners employed in the process of data acquisition to check for the variations in match performance with reference to the acquisition device. The secondary stage of data stratification included the creation of stratum based on the demographic features of the subjects in the dataset.;The main objectives this study aims to achieve are:;• Developing a framework to assess the match score distributions of each stratum..;• Assessing the match performance of demographic strata in comparison to the total dataset..;• Statistical match performance evaluation using match score statistics..;Following the generation of genuine and imposter match score distributions , Receiver Operating Characteristic Curves (ROC) were plotted to compare the match performance of each demographic stratum with respect to the total dataset. The divergence measures KLD and JSD have been calculated which signify the amount of variation between the match score distributions of each stratum. With the help of these procedures, the task of estimating the effect of data stratification on the match performance has been accomplished which serves as a measure of understanding the impact of this fingerprint dataset when used for biometric testing purposes

Activity related biometrics for person authentication

One of the major challenges in human-machine interaction has always been the development of such techniques that are able to provide accurate human recognition, so as to other either personalized services or to protect critical infrastructures from unauthorized access. To this direction, a series of well stated and efficient methods have been proposed mainly based on biometric characteristics of the user. Despite the significant progress that has been achieved recently, there are still many open issues in the area, concerning not only the performance of the systems but also the intrusiveness of the collecting methods. The current thesis deals with the investigation of novel, activity-related biometric traits and their potential for multiple and unobtrusive authentication based on the spatiotemporal analysis of human activities. In particular, it starts with an extensive bibliography review regarding the most important works in the area of biometrics, exhibiting and justifying in parallel the transition that is performed from the classic biometrics to the new concept of behavioural biometrics. Based on previous works related to the human physiology and human motion and motivated by the intuitive assumption that different body types and different characters would produce distinguishable, and thus, valuable for biometric verification, activity-related traits, a new type of biometrics, the so-called prehension biometrics (i.e. the combined movement of reaching, grasping activities), is introduced and thoroughly studied herein. The analysis is performed via the so-called Activity hyper-Surfaces that form a dynamic movement-related manifold for the extraction of a series of behavioural features. Thereafter, the focus is laid on the extraction of continuous soft biometric features and their efficient combination with state-of-the-art biometric approaches towards increased authentication performance and enhanced security in template storage via Soft biometric Keys. In this context, a novel and generic probabilistic framework is proposed that produces an enhanced matching probability based on the modelling of the systematic error induced during the estimation of the aforementioned soft biometrics and the efficient clustering of the soft biometric feature space. Next, an extensive experimental evaluation of the proposed methodologies follows that effectively illustrates the increased authentication potential of the prehension-related biometrics and the significant advances in the recognition performance by the probabilistic framework. In particular, the prehension biometrics related biometrics is applied on several databases of ~100 different subjects in total performing a great variety of movements. The carried out experiments simulate both episodic and multiple authentication scenarios, while contextual parameters, (i.e. the ergonomic-based quality factors of the human body) are also taken into account. Furthermore, the probabilistic framework for augmenting biometric recognition via soft biometrics is applied on top of two state-of-art biometric systems, i.e. a gait recognition (> 100 subjects)- and a 3D face recognition-based one (~55 subjects), exhibiting significant advances to their performance. The thesis is concluded with an in-depth discussion summarizing the major achievements of the current work, as well as some possible drawbacks and other open issues of the proposed approaches that could be addressed in future works.Open Acces

Image and Video Forensics

Nowadays, images and videos have become the main modalities of information being exchanged in everyday life, and their pervasiveness has led the image forensics community to question their reliability, integrity, confidentiality, and security. Multimedia contents are generated in many different ways through the use of consumer electronics and high-quality digital imaging devices, such as smartphones, digital cameras, tablets, and wearable and IoT devices. The ever-increasing convenience of image acquisition has facilitated instant distribution and sharing of digital images on digital social platforms, determining a great amount of exchange data. Moreover, the pervasiveness of powerful image editing tools has allowed the manipulation of digital images for malicious or criminal ends, up to the creation of synthesized images and videos with the use of deep learning techniques. In response to these threats, the multimedia forensics community has produced major research efforts regarding the identification of the source and the detection of manipulation. In all cases (e.g., forensic investigations, fake news debunking, information warfare, and cyberattacks) where images and videos serve as critical evidence, forensic technologies that help to determine the origin, authenticity, and integrity of multimedia content can become essential tools. This book aims to collect a diverse and complementary set of articles that demonstrate new developments and applications in image and video forensics to tackle new and serious challenges to ensure media authenticity