Suspect identification based on descriptive facial attributes

We present a method for using human describable face attributes to perform face identification in criminal inves-tigations. To enable this approach, a set of 46 facial at-tributes were carefully defined with the goal of capturing all describable and persistent facial features. Using crowd sourced labor, a large corpus of face images were manually annotated with the proposed attributes. In turn, we train an automated attribute extraction algorithm to encode target repositories with the attribute information. Attribute extrac-tion is performed using localized face components to im-prove the extraction accuracy. Experiments are conducted to compare the use of attribute feature information, derived from crowd workers, to face sketch information, drawn by expert artists. In addition to removing the dependence on expert artists, the proposed method complements sketch-based face recognition by allowing investigators to imme-diately search face repositories without the time delay that is incurred due to sketch generation. 1

Background subtraction using ensembles of classifiers with an extended feature set

The limitations of foreground segmentation in difficult environments using standard color space features often result in poor performance during autonomous tracking. This work presents a new approach for classification of foreground and background pixels in image sequences by employing an ensemble of classifiers, each operating on a different feature type such as the three RGB features, gradient magnitude and orientation features, and eight Haar features. These thirteen features are used in an ensemble classifier where each classifier operates on a single image feature. Each classifier implements a Mixture of Gaussians-based unsupervised background classification algorithm. The non-thresholded, classification decision score of each classifier are fused together by taking the average of their outputs and creating one single hypothesis. The results of using the ensemble classifier on three separate and distinct data sets are compared to using only RGB features through ROC graphs. The extended feature vector outperforms the RGB features on all three data sets, and shows a large scale improvement on two of the three data sets. The two data sets with the greatest improvements are both outdoor data sets with global illumination changes and the other has many local illumination changes. When using the entire feature set, to operate at a 90% true positive rate, the per pixel, false alarm rate is reduced five times in one data set and six times in the other data set

Background subtraction using ensembles of classifiers with an extended feature set

The limitations of foreground segmentation in difficult environments using standard color space features often result in poor performance during autonomous tracking. This work presents a new approach for classification of foreground and background pixels in image sequences by employing an ensemble of classifiers, each operating on a different feature type such as the three RGB features, gradient magnitude and orientation features, and eight Haar features. These thirteen features are used in an ensemble classifier where each classifier operates on a single image feature. Each classifier implements a Mixture of Gaussians-based unsupervised background classification algorithm. The non-thresholded, classification decision score of each classifier are fused together by taking the average of their outputs and creating one single hypothesis. The results of using the ensemble classifier on three separate and distinct data sets are compared to using only RGB features through ROC graphs. The extended feature vector outperforms the RGB features on all three data sets, and shows a large scale improvement on two of the three data sets. The two data sets with the greatest improvements are both outdoor data sets with global illumination changes and the other has many local illumination changes. When using the entire feature set, to operate at a 90% true positive rate, the per pixel, false alarm rate is reduced five times in one data set and six times in the other data set

Towards automated caricature recognition

This paper addresses the problem of identifying a subject from a caricature. A caricature is a facial sketch of a subject’s face that exaggerates identifiable facial features beyond realism, while still conveying his identity. To enable this task, we propose a set of qualitative facial features that encodes the appearance of both caricatures and photographs. We utilized crowdsourcing, through Amazon’s Mechanical Turk service, to assist in the labeling of the qualitative features. Using these features, we combine logistic regression, multiple kernel learning, and support vector machines to generate a similarity score between a caricature and a facial photograph. Experiments are conducted on a dataset of 196 pairs of caricatures and photographs, which we have made publicly available. Through the development of novel feature representations and matching algorithms, this research seeks to help leverage the ability of humans to recognize caricatures to improve automatic face recognition methods. 1

The FaceSketchID system: Matching facial composites to mugshots

Abstract—Facial composites are widely used by law enforce-ment agencies to assist in the identification and apprehension of suspects involved in criminal activities. These composites, generated from witness descriptions, are posted in public places and in the media with the hope that some viewers will provide tips about the identity of the suspect. This method of identifying suspects is slow, tedious, and may not lead to the timely apprehension of a suspect. Hence, there is a need for a method that can automatically and efficiently match facial composites to large police mugshot databases. Because of this requirement, facial composite recognition is an important topic for biometrics researchers. While substantial progress has been made in non-forensic facial composite (or viewed composite) recognition over the past decade, very little work has been done using operational composites relevant to law enforcement agencies. Furthermore, no facial composite to mugshot matching systems have been documented that are readily deployable as standalone software. Thus, the contributions of this paper include: (i) an exploration of composite recognition use cases involving multiple forms of facial composites, (ii) the FaceSketchID System, a scalable and operationally deployable software system that achieves state-of-the-art matching accuracy on facial composites using two algorithms (holistic and component-based), and (iii) a study of the effects of training data on algorithm performance. We present experimental results using a large mugshot gallery that is representative of a law enforcement agency’s mugshot database. All results are compared against three state-of-the-art commercial-off-the-shelf (COTS) face recognition systems. Index Terms—Facial composite recognition, hand-drawn com-posite, software-generated composite, surveillance composite, mugshot, holistic face recognition, component-based face recog-nition I

Open Source Biometric Recognition

The biometrics community enjoys an active research field that has produced algorithms for several modalities suitable for real-world applications. Despite these developments, there exist few open source implementations of complete algorithms that are maintained by the community or deployed outside a laboratory environment. In this paper we motivate the need for more community-driven open source software in the field of biometrics and present OpenBR as a candidate to address this deficiency. We overview the OpenBR software architecture and consider still-image frontal face recognition as a case study to illustrate its strengths and capabilities. All of our work is available at www.openbiometrics.org. 1