Mixing Biometric Data For Generating Joint Identities and Preserving Privacy

Biometrics is the science of automatically recognizing individuals by utilizing biological traits such as fingerprints, face, iris and voice. A classical biometric system digitizes the human body and uses this digitized identity for human recognition. In this work, we introduce the concept of mixing biometrics. Mixing biometrics refers to the process of generating a new biometric image by fusing images of different fingers, different faces, or different irises. The resultant mixed image can be used directly in the feature extraction and matching stages of an existing biometric system. In this regard, we design and systematically evaluate novel methods for generating mixed images for the fingerprint, iris and face modalities. Further, we extend the concept of mixing to accommodate two distinct modalities of an individual, viz., fingerprint and iris. The utility of mixing biometrics is demonstrated in two different applications. The first application deals with the issue of generating a joint digital identity. A joint identity inherits its uniqueness from two or more individuals and can be used in scenarios such as joint bank accounts or two-man rule systems. The second application deals with the issue of biometric privacy, where the concept of mixing is used for de-identifying or obscuring biometric images and for generating cancelable biometrics. Extensive experimental analysis suggests that the concept of biometric mixing has several benefits and can be easily incorporated into existing biometric systems

Fusing Multimedia Data Into Dynamic Virtual Environments

In spite of the dramatic growth of virtual and augmented reality (VR and AR) technology, content creation for immersive and dynamic virtual environments remains a significant challenge. In this dissertation, we present our research in fusing multimedia data, including text, photos, panoramas, and multi-view videos, to create rich and compelling virtual environments. First, we present Social Street View, which renders geo-tagged social media in its natural geo-spatial context provided by 360° panoramas. Our system takes into account visual saliency and uses maximal Poisson-disc placement with spatiotemporal filters to render social multimedia in an immersive setting. We also present a novel GPU-driven pipeline for saliency computation in 360° panoramas using spherical harmonics (SH). Our spherical residual model can be applied to virtual cinematography in 360° videos. We further present Geollery, a mixed-reality platform to render an interactive mirrored world in real time with three-dimensional (3D) buildings, user-generated content, and geo-tagged social media. Our user study has identified several use cases for these systems, including immersive social storytelling, experiencing the culture, and crowd-sourced tourism. We next present Video Fields, a web-based interactive system to create, calibrate, and render dynamic videos overlaid on 3D scenes. Our system renders dynamic entities from multiple videos, using early and deferred texture sampling. Video Fields can be used for immersive surveillance in virtual environments. Furthermore, we present VRSurus and ARCrypt projects to explore the applications of gestures recognition, haptic feedback, and visual cryptography for virtual and augmented reality. Finally, we present our work on Montage4D, a real-time system for seamlessly fusing multi-view video textures with dynamic meshes. We use geodesics on meshes with view-dependent rendering to mitigate spatial occlusion seams while maintaining temporal consistency. Our experiments show significant enhancement in rendering quality, especially for salient regions such as faces. We believe that Social Street View, Geollery, Video Fields, and Montage4D will greatly facilitate several applications such as virtual tourism, immersive telepresence, and remote education

Designing content-based adversarial perturbations and distributed one-class learning for images.

PhD Theses.This thesis covers two privacy-related problems for images: designing adversarial perturbations that can be added to the input images to protect the private content of images that a user shares with other users from the undesirable automatic inference of classifiers, and training privacy-preserving classifiers on images that are distributed among their owners (image holders) and contain their private information. Adversarial images can be easily detected using denoising algorithms when high-frequency spatial perturbations are used, or can be noticed by humans when perturbations are large and irrelevant to the content of images. In addition to this, adversarial images are not transferable to unseen classifiers as perturbations are small (in terms of the lp norm). In the first part of the thesis, we propose content-based adversarial perturbations that account for the content of the images (objects, colour, structure and details), human perception and the semantics of the class labels to address the above-mentioned limitations of perturbations. Our adversarial colour perturbations selectively modify the colours of objects within chosen ranges that are perceived as natural by humans. In addition to these natural-looking adversarial images, our structure-aware perturbations exploit traditional image processing filters, such as detail enhancement filter and Gamma correction filter, to generate enhanced adversarial images. We validate the proposed perturbations against three classifiers trained on ImageNet. Experiments show that the proposed perturbations are more robust and transferable and cause misclassification with a label that is semantically different from the label of the original image, when compared with seven state-ofthe- art perturbations. Classifiers are often trained by relying on centralised collection and aggregation of images that could lead to significant privacy concerns by disclosing the sensitive information of image holders. In the second part of the thesis, we propose a privacy-preserving technique, called distributed one-class learning, that enables training to take place on edge devices and therefore image holders do not need to centralise their images. Each image holder can independently use their images to locally train a reconstructive adversarial network as their one-class classifier. As sending the model parameters to the service provider would reveal sensitive information, we secret-share the parameters among two non-colluding service providers. Then, we provide cryptographically private prediction services through a mixture of multi-party computation protocols to achieve substantial gains in complexity and speed. A major advantage of the proposed technique is that none of the image holders and service providers can access the parameters and images of other image holders. We quantify the benefits of the proposed technique and compare its 3 4 performance with centralised training on three privacy-sensitive image-based tasks. Experiments show that the proposed technique achieves similar classification performance as non-private centralised training, while not violating the privacy of the image holders

Stochastic Memory Devices for Security and Computing

With the widespread use of mobile computing and internet of things, secured communication and chip authentication have become extremely important. Hardware-based security concepts generally provide the best performance in terms of a good standard of security, low power consumption, and large-area density. In these concepts, the stochastic properties of nanoscale devices, such as the physical and geometrical variations of the process, are harnessed for true random number generators (TRNGs) and physical unclonable functions (PUFs). Emerging memory devices, such as resistive-switching memory (RRAM), phase-change memory (PCM), and spin-transfer torque magnetic memory (STT-MRAM), rely on a unique combination of physical mechanisms for transport and switching, thus appear to be an ideal source of entropy for TRNGs and PUFs. An overview of stochastic phenomena in memory devices and their use for developing security and computing primitives is provided. First, a broad classification of methods to generate true random numbers via the stochastic properties of nanoscale devices is presented. Then, practical implementations of stochastic TRNGs, such as hardware security and stochastic computing, are shown. Finally, future challenges to stochastic memory development are discussed

Democracy Enhancing Technologies: Toward deployable and incoercible E2E elections

End-to-end verifiable election systems (E2E systems) provide a provably correct tally while maintaining the secrecy of each voter's ballot, even if the voter is complicit in demonstrating how they voted. Providing voter incoercibility is one of the main challenges of designing E2E systems, particularly in the case of internet voting. A second challenge is building deployable, human-voteable E2E systems that conform to election laws and conventions. This dissertation examines deployability, coercion-resistance, and their intersection in election systems. In the course of this study, we introduce three new election systems, (Scantegrity, Eperio, and Selections), report on two real-world elections using E2E systems (Punchscan and Scantegrity), and study incoercibility issues in one deployed system (Punchscan). In addition, we propose and study new practical primitives for random beacons, secret printing, and panic passwords. These are tools that can be used in an election to, respectively, generate publicly verifiable random numbers, distribute the printing of secrets between non-colluding printers, and to covertly signal duress during authentication. While developed to solve specific problems in deployable and incoercible E2E systems, these techniques may be of independent interest

Advancing iris biometric technology

PhD ThesisThe iris biometric is a well-established technology which is already in use in several nation-scale applications and it is still an active research area with several unsolved problems. This work focuses on three key problems in iris biometrics namely: segmentation, protection and cross-matching. Three novel methods in each of these areas are proposed and analyzed thoroughly. In terms of iris segmentation, a novel iris segmentation method is designed based on a fusion of an expanding and a shrinking active contour by integrating a new pressure force within the Gradient Vector Flow (GVF) active contour model. In addition, a new method for closed eye detection is proposed. The experimental results on the CASIA V4, MMU2, UBIRIS V1 and UBIRIS V2 databases show that the proposed method achieves state-of-theart results in terms of segmentation accuracy and recognition performance while being computationally more efficient. In this context, improvements by 60.5%, 42% and 48.7% are achieved in segmentation accuracy for the CASIA V4, MMU2 and UBIRIS V1 databases, respectively. For the UBIRIS V2 database, a superior time reduction is reported (85.7%) while maintaining a similar accuracy. Similarly, considerable time improvements by 63.8%, 56.6% and 29.3% are achieved for the CASIA V4, MMU2 and UBIRIS V1 databases, respectively. With respect to iris biometric protection, a novel security architecture is designed to protect the integrity of iris images and templates using watermarking and Visual Cryptography (VC). Firstly, for protecting the iris image, text which carries personal information is embedded in the middle band frequency region of the iris image using a novel watermarking algorithm that randomly interchanges multiple middle band pairs of the Discrete Cosine Transform (DCT). Secondly, for iris template protection, VC is utilized to protect the iii iris template. In addition, the integrity of the stored template in the biometric smart card is guaranteed by using the hash signatures. The proposed method has a minimal effect on the iris recognition performance of only 3.6% and 4.9% for the CASIA V4 and UBIRIS V1 databases, respectively. In addition, the VC scheme is designed to be readily applied to protect any biometric binary template without any degradation to the recognition performance with a complexity of only O(N). As for cross-spectral matching, a framework is designed which is capable of matching iris images in different lighting conditions. The first method is designed to work with registered iris images where the key idea is to synthesize the corresponding Near Infra-Red (NIR) images from the Visible Light (VL) images using an Artificial Neural Network (ANN) while the second method is capable of working with unregistered iris images based on integrating the Gabor filter with different photometric normalization models and descriptors along with decision level fusion to achieve the cross-spectral matching. A significant improvement by 79.3% in cross-spectral matching performance is attained for the UTIRIS database. As for the PolyU database, the proposed verification method achieved an improvement by 83.9% in terms of NIR vs Red channel matching which confirms the efficiency of the proposed method. In summary, the most important open issues in exploiting the iris biometric are presented and novel methods to address these problems are proposed. Hence, this work will help to establish a more robust iris recognition system due to the development of an accurate segmentation method working for iris images taken under both the VL and NIR. In addition, the proposed protection scheme paves the way for a secure iris images and templates storage. Moreover, the proposed framework for cross-spectral matching will help to employ the iris biometric in several security applications such as surveillance at-a-distance and automated watch-list identification.Ministry of Higher Education and Scientific Research in Ira

Secret Sharing Approach for Securing Cloud-Based Image Processing

Ph.DDOCTOR OF PHILOSOPH