Suspect identification based on descriptive facial attributes

We present a method for using human describable face attributes to perform face identification in criminal inves-tigations. To enable this approach, a set of 46 facial at-tributes were carefully defined with the goal of capturing all describable and persistent facial features. Using crowd sourced labor, a large corpus of face images were manually annotated with the proposed attributes. In turn, we train an automated attribute extraction algorithm to encode target repositories with the attribute information. Attribute extrac-tion is performed using localized face components to im-prove the extraction accuracy. Experiments are conducted to compare the use of attribute feature information, derived from crowd workers, to face sketch information, drawn by expert artists. In addition to removing the dependence on expert artists, the proposed method complements sketch-based face recognition by allowing investigators to imme-diately search face repositories without the time delay that is incurred due to sketch generation. 1

Video-to-video face matching: Establishing a baseline for unconstrained face recognition

Abstract Face recognition in video is becoming increasingly important due to the abundance of video data captured by surveillance cameras, mobile devices, Internet uploads, and other sources. Given the aggregate of facial information contained in a video (i.e., a sequence of face images or frames), video-based face recognition solutions can potentially alleviate classic challenges caused by variations in pose, illumination, and expression. However, with this increased focus on the development of algorithms specifically crafted for video-based face recognition, it is important to establish a baseline for the accuracy using state-of-theart still image matchers. Note that most commercial-offthe-shelf (COTS) offerings are still limited to single frame matching. In order to measure the accuracy of COTS face recognition systems on video data, we first investigate the effectiveness of multi-frame score-level fusion and analyze the consistency across three COTS face matchers. We demonstrate that all three COTS matchers individually are superior to previously published face recognition results on the unconstrained YouTube Faces database. Further, fusion of scores from the three COTS matchers achieves a 20% improvement in accuracy over previously published results. We encourage the use of these results as a competitive baseline for video-to-video face matching on the YouTube Faces database

Background subtraction using ensembles of classifiers with an extended feature set

The limitations of foreground segmentation in difficult environments using standard color space features often result in poor performance during autonomous tracking. This work presents a new approach for classification of foreground and background pixels in image sequences by employing an ensemble of classifiers, each operating on a different feature type such as the three RGB features, gradient magnitude and orientation features, and eight Haar features. These thirteen features are used in an ensemble classifier where each classifier operates on a single image feature. Each classifier implements a Mixture of Gaussians-based unsupervised background classification algorithm. The non-thresholded, classification decision score of each classifier are fused together by taking the average of their outputs and creating one single hypothesis. The results of using the ensemble classifier on three separate and distinct data sets are compared to using only RGB features through ROC graphs. The extended feature vector outperforms the RGB features on all three data sets, and shows a large scale improvement on two of the three data sets. The two data sets with the greatest improvements are both outdoor data sets with global illumination changes and the other has many local illumination changes. When using the entire feature set, to operate at a 90% true positive rate, the per pixel, false alarm rate is reduced five times in one data set and six times in the other data set

Background subtraction using ensembles of classifiers with an extended feature set

The limitations of foreground segmentation in difficult environments using standard color space features often result in poor performance during autonomous tracking. This work presents a new approach for classification of foreground and background pixels in image sequences by employing an ensemble of classifiers, each operating on a different feature type such as the three RGB features, gradient magnitude and orientation features, and eight Haar features. These thirteen features are used in an ensemble classifier where each classifier operates on a single image feature. Each classifier implements a Mixture of Gaussians-based unsupervised background classification algorithm. The non-thresholded, classification decision score of each classifier are fused together by taking the average of their outputs and creating one single hypothesis. The results of using the ensemble classifier on three separate and distinct data sets are compared to using only RGB features through ROC graphs. The extended feature vector outperforms the RGB features on all three data sets, and shows a large scale improvement on two of the three data sets. The two data sets with the greatest improvements are both outdoor data sets with global illumination changes and the other has many local illumination changes. When using the entire feature set, to operate at a 90% true positive rate, the per pixel, false alarm rate is reduced five times in one data set and six times in the other data set

Background subtraction in varying illuminations using an ensemble based on an enlarged feature

Abstract Image sequences with dynamic backgrounds often cause false classification of pixels. In particular, varying illuminations cause significant changes in the representation of a scene in different color spaces, which in turn results in the high levels of failure in such conditions. Because mapping to alternate color spaces has largely failed to solve this problem, a solution of using alternate image features is proposed in this paper. In particular, the use of gradient and texture features along with the original color intensities are used in an ensemble of Mixture of Gaussians background classifiers. A clear improvement is shown when using this method compared to the Mixture of Gaussians algorithm using only color intensities. In addition, this work shows that performing background subtraction using only gradient magnitude as an image feature performs at a much higher rate in varying illuminations then using color intensities. Results are generated on three separate datasets, each with unique, dynamic, illumination conditions

Face recognition across time lapse: On learning feature subspaces

There is a growing interest in understanding the impact of aging on face recognition performance, as well as designing recognition algorithms that are mostly invariant to temporal changes. While some success has been made on this front, a fundamental questions has yet to be answered: do face recognition systems that compensate for the effects of aging compromise recognition performance for faces that have not undergone any aging? The studies in this paper help confirm that age invariant systems do seem to decrease performance in non-aging scenarios. This is demonstrated by performing training experiments on the largest face aging dataset studied in the literature to date (over 200,000 images from roughly 64,000 subjects). Further experiments conducted in this research help demonstrate the impact of aging on two leading commercial face recognition systems. We also determine the regions of the face that remain the most stable over time. 1