Person Search Using Identity Attributes

Abstract

The goal of this dissertation is to develop and evaluate algorithms and a prototype system to retrieve frames depicting humans with specific identity attributes obtained from a textual description. Solving this problem requires addressing three separate subproblems, namely (i) defining the ontology of the identity and the identity-related attributes, (ii) developing and evaluating algorithms for extracting identity attributes from images, and (iii) developing and evaluating an algorithm for attribute-based person search in databases of image frames. This dissertation presents a list of methods on visual attribute classification and person search that significantly improve the accuracy over previous work. The methods presented tackle key limitations of previous work such as the class imbalance of visual attributes, or the challenge of learning discriminative representations from the textual input. By learning to retrieve the most relevant images of individuals based on textual descriptions, such techniques can have a broader impact in cases of missing children or in surveillance applications. The works introduced in this dissertation are capable of successfully identifying which images contain humans with such characteristics which could reduce dramatically the effort and the time required to identify such information. In each method a detailed overview of the benefits and limitations of each approach is introduced, extensive experimental evaluation and ablation studies are provided to analyze the impact of different modules, and further limitations have been identified that need to be addressed by future work.Computer Science, Department o