Evaluating the Performance of a Large-Scale Facial Image Dataset Using Agglomerated Match Score Statistics

Biometrics systems are experiencing wide-spread usage in identification and access control applications. To estimate the performance of any biometric systems, their characteristics need to be analyzed to make concrete conclusions for real time usage. Performance testing of hardware or software components of either custom or state-of-the-art commercial biometric systems is typically carried out on large datasets. Several public and private datasets are used in current biometric research. West Virginia University has completed several large scale multimodal biometric data collection with an aim to create research datasets that can be used by disciplines concerning secured biometric applications. However, the demographic and image quality properties of these datasets can potentially lead to bias when they are used in performance testing of new systems. To overcome this, the characteristics of datasets used for performance testing must be well understood prior to usage.;This thesis will answer three main questions associated with this issue:;• For a single matcher, do the genuine and impostor match score distributions within specific demographics groups vary from those of the entire dataset? • What are the possible ways to compare the subset of demographic match score distributions against those of the entire dataset? • Based on these comparisons, what conclusions can be made about the characteristics of dataset?;In this work, 13,976 frontal face images from WVU\u27s 2012 Biometric collection project funded by the FBI involving 1200 individuals were used as a \u27test\u27 dataset. The goal was to evaluate performance of this dataset by generating genuine and impostor match scores distributions using a commercial matching software Further, the dataset was categorized demographically, and match score distributions were generated for these subsets in order to explore whether or not this breakdown impacted match score distributions. The match score distributions of the overall dataset were compared against each demographic cohorts.;Using statistical measures, Area under Curve (AUC) and Equal Error Rate (EER) were observed by plotting Receiver Operating Characteristics (ROC) curves to measure the performance of each demographic group with respect to overall data and also within the cohorts of demographic group. Also, Kull-back Leibler Divergence and Jensen Shannon Divergence values were calculated for each demographic cohort (age, gender and ethnicity) within the overall data. These statistical approaches provide a numerical value representing the amount of variation between two match score distributions In addition, FAR and FRR was observed to estimate the error rates. These statistical measures effectively enabled the determination of the impact of different demographic breakdown on match score distributions, and thus, helped in understanding the characteristics of dataset and how they may impact its usage in performance testing biometrics

Intersection of Longest Paths in Graph Theory and Predicting Performance in Facial Recognition

A set of subsets is said to have the Helly property if the condition that each pair of subsets has a non-empty intersection implies that the intersection of all subsets has a non-empty intersection. In 1966, Gallai noticed that the set of all longest paths of a connected graph is pairwise intersecting and asked if the set had the Helly property. While it is not true in general, a number of classes of graphs have been shown to have the property. In this dissertation, we show that K4-minor-free graphs, interval graphs, circular arc graphs, and the intersection graphs of spider graphs are classes that have this property. The accuracy of facial recognition algorithms on images taken in controlled conditions has improved significantly over the last two decades. As the focus is turning to more unconstrained or relaxed conditions and toward videos, there is a need to better understand what factors influence performance. If these factors were better understood, it would be easier to predict how well an algorithm will perform when new conditions are introduced. Previous studies have studied the effect of various factors on the verification rate (VR), but less attention has been paid to the false accept rate (FAR). In this dissertation, we study the effect various factors have on the FAR as well as the correlation between marginal FAR and VR. Using these relationships, we propose two models to predict marginal VR and demonstrate that the models predict better than using the previous global VR