MIPGAN -- Generating Strong and High Quality Morphing Attacks Using Identity Prior Driven GAN

Face morphing attacks target to circumvent Face Recognition Systems (FRS) by employing face images derived from multiple data subjects (e.g., accomplices and malicious actors). Morphed images can be verified against contributing data subjects with a reasonable success rate, given they have a high degree of facial resemblance. The success of morphing attacks is directly dependent on the quality of the generated morph images. We present a new approach for generating strong attacks extending our earlier framework for generating face morphs. We present a new approach using an Identity Prior Driven Generative Adversarial Network, which we refer to as MIPGAN (Morphing through Identity Prior driven GAN). The proposed MIPGAN is derived from the StyleGAN with a newly formulated loss function exploiting perceptual quality and identity factor to generate a high quality morphed facial image with minimal artefacts and with high resolution. We demonstrate the proposed approach's applicability to generate strong morphing attacks by evaluating its vulnerability against both commercial and deep learning based Face Recognition System (FRS) and demonstrate the success rate of attacks. Extensive experiments are carried out to assess the FRS's vulnerability against the proposed morphed face generation technique on three types of data such as digital images, re-digitized (printed and scanned) images, and compressed images after re-digitization from newly generated MIPGAN Face Morph Dataset. The obtained results demonstrate that the proposed approach of morph generation poses a high threat to FRS.Comment: Revised version. Submitted to IEEE T-BIOM 202

Face analysis and deepfake detection

This thesis concerns deep-learning-based face-related research topics. We explore how to improve the performance of several face systems when confronting challenging variations. In Chapter 1, we provide an introduction and background information on the theme, and we list the main research questions of this dissertation. In Chapter 2, we provide a synthetic face data generator with fully controlled variations and proposed a detailed experimental comparison of main characteristics that influence face detection performance. The result shows that our synthetic dataset could complement face detectors to become more robust against specific features in the real world. Our analysis also reveals that a variety of data augmentation is necessary to address differences in performance. In Chapter 3, we propose an age estimation method for handling large pose variations for unconstrained face images. A Wasserstein-based GAN model is used to complete the full uv texture presentation. The proposed AgeGAN method simultaneously learns to capture the facial uv texture map and age characteristics.In Chapter 4, we propose a maximum mean discrepancy (MMD) based cross-domain face forgery detection. The center and triplet losses are also incorporated to ensure that the learned features are shared by multiple domains and provide better generalization abilities to unseen deep fake samples. In Chapter 5, we introduce an end-to-end framework to predict ages from face videos. Clustering based transfer learning is used to provide proper prediction for imbalanced datasets

Natural Image Statistics for Digital Image Forensics

We describe a set of natural image statistics that are built upon two multi-scale image decompositions, the quadrature mirror filter pyramid decomposition and the local angular harmonic decomposition. These image statistics consist of first- and higher-order statistics that capture certain statistical regularities of natural images. We propose to apply these image statistics, together with classification techniques, to three problems in digital image forensics: (1) differentiating photographic images from computer-generated photorealistic images, (2) generic steganalysis; (3) rebroadcast image detection. We also apply these image statistics to the traditional art authentication for forgery detection and identification of artists in an art work. For each application we show the effectiveness of these image statistics and analyze their sensitivity and robustness

Improving Utility of GPU in Accelerating Industrial Applications with User-centred Automatic Code Translation

SMEs (Small and medium-sized enterprises), particularly those whose business is focused on developing innovative produces, are limited by a major bottleneck on the speed of computation in many applications. The recent developments in GPUs have been the marked increase in their versatility in many computational areas. But due to the lack of specialist GPU (Graphics processing units) programming skills, the explosion of GPU power has not been fully utilized in general SME applications by inexperienced users. Also, existing automatic CPU-to-GPU code translators are mainly designed for research purposes with poor user interface design and hard-to-use. Little attentions have been paid to the applicability, usability and learnability of these tools for normal users. In this paper, we present an online automated CPU-to-GPU source translation system, (GPSME) for inexperienced users to utilize GPU capability in accelerating general SME applications. This system designs and implements a directive programming model with new kernel generation scheme and memory management hierarchy to optimize its performance. A web-service based interface is designed for inexperienced users to easily and flexibly invoke the automatic resource translator. Our experiments with non-expert GPU users in 4 SMEs reflect that GPSME system can efficiently accelerate real-world applications with at least 4x and have a better applicability, usability and learnability than existing automatic CPU-to-GPU source translators

Partial Face Detection and Illumination Estimation

Face Analysis has long been a crucial component of many security applications. In this work, we shall propose and explore some face analysis algorithms which are applicable to two different security problems, namely Active Authentication and Image Tampering Detection. In the first section, we propose two algorithms, “Deep Feature based Face Detection for Mobile Devices” and “DeepSegFace” that are useful in detecting partial faces such as those seem in typical Active Authentication scenarios. In the second section, we propose an algorithm to detect discrepancies in illumination conditions given two face images, and use that as an indication to decide if an image has been tampered by transplanting faces. We also extend the illumination detection algorithm by proposing an adversarial data augmentation scheme. We show the efficacy of the proposed algorithms by evaluating them on multiple datasets

HYPERSPECTRAL IMAGING AND PATTERN RECOGNITION TECHNOLOGIES FOR REAL TIME FRUIT SAFETY AND QUALITY INSPECTION

Hyperspectral band selection and band combination has become a powerful tool and have gained enormous interest among researchers. An important task in hyperspectral data processing is to reduce the redundancy of the spectral and spatial information without losing any valuable details that are needed for the subsequent detection, discrimination and classification processes. An integrated principal component analysis (PCA) and Fisher linear discriminant (FLD) method has been developed for feature band selection, and other pattern recognition technologies have been applied and compared with the developed method. The results on different types of defects from cucumber and apple samples show that the integrated PCA-FLD method outperforms PCA, FLD and canonical discriminant methods when they are used separately for classification. The integrated method adds a new tool for the multivariate analysis of hyperspectral images and can be extended to other hyperspectral imaging applications. Dimensionality reduction not only serves as the first step of data processing that leads to a significant decrease in computational complexity in the successive procedures, but also a research tool for determining optimal spectra requirement for online automatic inspection of fruit. In this study, the hyperspectral research shows that the near infrared spectrum at 753nm is best for detecting apple defect. When applied for online apple defect inspection, over 98% of good apple detection rate is achieved. However, commercially available apple sorting and inspection machines cannot effectively solve the stem-calyx problems involved in automatic apple defects detection. In this study, a dual-spectrum NIR/MIR sensing method is applied. This technique can effectively distinguish true defects from stems and calyxes, which leads to a potential solution of the problem. The results of this study will advance the technology in fruit safety and quality inspection and improve the cost-effectiveness of fruit packing processes

The Analysis of Ballpoint Inks with APCI-MS after Fading with Light, Hydrogen Peroxide and Sodium Hypochlorite Bleach

The ability to discriminate between different inks and to determine the length of time an ink has been on a substrate can provide important scientific evidence, especially in cases involving document fraud. Many techniques have been used to analyse inks for ink dating including chromatography and spectroscopy, but the results are unreliable as a result of factors affecting the aging process such as light. This study utilises established techniques in Forensic Document Examination, including filtered light examination but also novel techniques for ink analysis; Atmospheric Pressure Chemical Ionisation (APCI) to analyse inks and dyes with the aim of discriminating between samples based on their degradation products. APCI-MS was used for the first time to study nineteen ballpoint pens from a range of manufacturers by investigating the chemical processes that occur and the products that are formed following the deposition of ink onto a substrate and in solution. Monitoring the degradation process as an ink ages and fades enables the identification of components present in the inks. Using molecular mass data, accurate ink component identifications could be made over a period of two years on samples subjected to a range of external influences. Light, hydrogen peroxide and sodium hypochlorite bleach were used to simulate natural and deliberate fading of inks and dye solutions. Benzophenone and phenol molecules were identified as degradation products but their presence differed for each of the different conditions tested such as no phenol products when bleach was used. This novel approach to ink analysis utilises existing equipment commonly used by document examiner to analyse inks that are old or faded in some way, in order to discriminate between the inks or determine method of alteratio