UG^2: a Video Benchmark for Assessing the Impact of Image Restoration and Enhancement on Automatic Visual Recognition

Advances in image restoration and enhancement techniques have led to discussion about how such algorithmscan be applied as a pre-processing step to improve automatic visual recognition. In principle, techniques like deblurring and super-resolution should yield improvements by de-emphasizing noise and increasing signal in an input image. But the historically divergent goals of the computational photography and visual recognition communities have created a significant need for more work in this direction. To facilitate new research, we introduce a new benchmark dataset called UG^2, which contains three difficult real-world scenarios: uncontrolled videos taken by UAVs and manned gliders, as well as controlled videos taken on the ground. Over 160,000 annotated frames forhundreds of ImageNet classes are available, which are used for baseline experiments that assess the impact of known and unknown image artifacts and other conditions on common deep learning-based object classification approaches. Further, current image restoration and enhancement techniques are evaluated by determining whether or not theyimprove baseline classification performance. Results showthat there is plenty of room for algorithmic innovation, making this dataset a useful tool going forward.Comment: Supplemental material: https://goo.gl/vVM1xe, Dataset: https://goo.gl/AjA6En, CVPR 2018 Prize Challenge: ug2challenge.or

Deep Learning for the Analysis of Latent Fingerprint Images

Latent fingerprints are fingerprint impressions unintentionally left on surfaces at a crime scene. The accuracy of latent fingerprint identification by latent fingerprint forensic examiners has been the subject of increased study, scrutiny, and commentary in the legal system and the forensic science literature. Errors in latent fingerprint matchingcan be devastating, resulting in missed opportunities to apprehend criminals or wrongful convictions of innocent people. Latent fingerprint comparison is increasingly relied upon by law enforcement to solve crime, and prosecute offenders. The increasing use of this service places new strains on the limited resources of the forensic science delivery system. Currently, latent examiners manually mark the region of interest (ROI) in latent fingerprints and use features manually identified in the ROI tosearch large databases of reference full fingerprints to identify a small number of potential matches for subsequent manual examination. Given the large size of law enforcement databases containing rolled and plain fingerprints, it is very desirable to perform latent fingerprint processing in a fully automated way.This dissertation proposes deep learning models and algorithms developed in the context of machine learning for automatic latent fingerprint image quality assessment, quality improvement, segmentation and matching. We also propose techniques that help speed-up convergence of a deep neural network and achieve a better estimation of the relation between a latent fingerprint image patch and its target class. A unified frequency domain based framework for latent fingerprint matching using image patches, as well as a novel latent fingerprint super-resolution model that uses a graph-total variation energy of latent fingerprints as a non-local regularizer for learning optimal weights for high quality image reconstruction, are also proposed. Using the deep learning models, we aim at providing an end-to-end automatic system that solves the problems inherent in latent fingerprint quality assessment, quality improvement, segmentation and matching