Fast Preprocessing for Robust Face Sketch Synthesis

Exemplar-based face sketch synthesis methods usually meet the challenging problem that input photos are captured in different lighting conditions from training photos. The critical step causing the failure is the search of similar patch candidates for an input photo patch. Conventional illumination invariant patch distances are adopted rather than directly relying on pixel intensity difference, but they will fail when local contrast within a patch changes. In this paper, we propose a fast preprocessing method named Bidirectional Luminance Remapping (BLR), which interactively adjust the lighting of training and input photos. Our method can be directly integrated into state-of-the-art exemplar-based methods to improve their robustness with ignorable computational cost.Comment: IJCAI 2017. Project page: http://www.cs.cityu.edu.hk/~yibisong/ijcai17_sketch/index.htm

Learning to Hallucinate Face Images via Component Generation and Enhancement

We propose a two-stage method for face hallucination. First, we generate facial components of the input image using CNNs. These components represent the basic facial structures. Second, we synthesize fine-grained facial structures from high resolution training images. The details of these structures are transferred into facial components for enhancement. Therefore, we generate facial components to approximate ground truth global appearance in the first stage and enhance them through recovering details in the second stage. The experiments demonstrate that our method performs favorably against state-of-the-art methodsComment: IJCAI 2017. Project page: http://www.cs.cityu.edu.hk/~yibisong/ijcai17_sr/index.htm

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Synthesizing face sketches from real photos and its inverse have many applications. However, photo/sketch synthesis remains a challenging problem due to the fact that photo and sketch have different characteristics. In this work, we consider this task as an image-to-image translation problem and explore the recently popular generative models (GANs) to generate high-quality realistic photos from sketches and sketches from photos. Recent GAN-based methods have shown promising results on image-to-image translation problems and photo-to-sketch synthesis in particular, however, they are known to have limited abilities in generating high-resolution realistic images. To this end, we propose a novel synthesis framework called Photo-Sketch Synthesis using Multi-Adversarial Networks, (PS2-MAN) that iteratively generates low resolution to high resolution images in an adversarial way. The hidden layers of the generator are supervised to first generate lower resolution images followed by implicit refinement in the network to generate higher resolution images. Furthermore, since photo-sketch synthesis is a coupled/paired translation problem, we leverage the pair information using CycleGAN framework. Both Image Quality Assessment (IQA) and Photo-Sketch Matching experiments are conducted to demonstrate the superior performance of our framework in comparison to existing state-of-the-art solutions. Code available at: https://github.com/lidan1/PhotoSketchMAN.Comment: Accepted by 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)(Oral

Context-Patch Face Hallucination Based on Thresholding Locality-Constrained Representation and Reproducing Learning

Face hallucination is a technique that reconstruct high-resolution (HR) faces from low-resolution (LR) faces, by using the prior knowledge learned from HR/LR face pairs. Most state-of-the-arts leverage position-patch prior knowledge of human face to estimate the optimal representation coefficients for each image patch. However, they focus only the position information and usually ignore the context information of image patch. In addition, when they are confronted with misalignment or the Small Sample Size (SSS) problem, the hallucination performance is very poor. To this end, this study incorporates the contextual information of image patch and proposes a powerful and efficient context-patch based face hallucination approach, namely Thresholding Locality-constrained Representation and Reproducing learning (TLcR-RL). Under the context-patch based framework, we advance a thresholding based representation method to enhance the reconstruction accuracy and reduce the computational complexity. To further improve the performance of the proposed algorithm, we propose a promotion strategy called reproducing learning. By adding the estimated HR face to the training set, which can simulates the case that the HR version of the input LR face is present in the training set, thus iteratively enhancing the final hallucination result. Experiments demonstrate that the proposed TLcR-RL method achieves a substantial increase in the hallucinated results, both subjectively and objectively. Additionally, the proposed framework is more robust to face misalignment and the SSS problem, and its hallucinated HR face is still very good when the LR test face is from the real-world. The MATLAB source code is available at https://github.com/junjun-jiang/TLcR-RL

An Experimental and Numerical Investigation of Nitrogen Dioxide Emissions Characteristics of Compression Ignition Dual Fuel Engines

Detailed experimental research was conducted to explore the impact of the addition of gaseous fuels, including H2 and natural gas (NG), and engine load on the emissions of NO2, NO, and NOx from dual fuel engines. The addition of less than 2% of H2 or NG was shown to dramatically increase the emissions of NO2 until a maximum level of NO2 emissions was reached. The increased NO 2 emissions were due to the conversion of NO to NO2. The maximum NO2/NOx ratio obtained with the addition of H2 was 3.2 to 5.0 times that of diesel operation. The maximum NO 2/NOx ratio obtained with the addition of NG was 3.4 to 4.3 times that of diesel operation. Further increasing the amount of gaseous fuel beyond the point of maximum NO2 emissions resulted in a reduction of NO2 emissions. Detailed examination of factors having the potential to affect the formation of NOx and NO2 in compression ignition engines reported a firm correlation between the emissions of NO 2 and emissions of unburned H2 and methane (CH4), and their relative emissions. The presence of unburned gaseous fuels that survived the main combustion process appears to be one of the main factors contributing to the enhanced conversion of NO to NO2. This was supported by the experimental data reported in the literature. The presence of fumigation fuels outside the diesel spray plume might be the main factor contributing to the increased emissions of NO2 from dual fuel engines. The spontaneous combustion of fumigation fuels that are entrained into the diesel spray plume may not contribute to the increased emissions of NO 2. In comparison, the correlations between the increased emissions of NO2 and the variation in bulk mixture temperature and heat release process including maximum heat release rate, and combustion duration were weak.;A single zone, zero-dimensional, constant volume numerical model with detailed chemistry was used to simulate the oxidization process of the gaseous fuel, as well as its effect on the conversion of NO to NO2 after the post-combustion mixing of the gaseous fuel surviving the main combustion process with the NOx-containing combustion products. The gaseous fuel examined included CH4, H2, and carbon monoxide (CO). The simulation results revealed the significant effects of the fuel mixed, its initial concentration in the mixture, and the initial temperature on the oxidization of gaseous fuel, the conversion of NO to NO2, and the destruction of NO2 to NO after the completion of the oxidation process.;The single zone zero-dimensional model was further modified to a variable volume model with the volume of the combustion chamber calculated using the geometry of the 1999 Cummins engine and engine speed. The modified variable volume model with detailed chemistry was used to improve the simulation of the effect on the conversion of NO to NO2 of the post-combustion mixing of surviving gaseous fuel with NOx-containing combustion products. The spatial variation of the local bulk mixture temperature with the progress of the combustion process and the variation of cylinder volume during the expansion process was taken into account by a pseudo temperature at the top dead center (TDC) noted as Tpseudo TDC defined in this research. The simulation identified the importance of the phasing of postcombustion mixing on the oxidation of gaseous fuel and its effect on the conversion of NO to NO2.;A preliminary sensitivity analysis was also conducted to identify the reactions having significant effect on the conversion of NO to NO2 and its destruction to NO. Among the four reactions associated with the formation and destruction of NO2, R186 was identified as the main reaction to the formation of NO2 during the oxidation process of H 2 and CO. This was due to the high concentration of HO2 formed during the oxidation process of H2 and CO in the combustion product. The destruction of NO2 to NO occurred through R187 and R189. (Abstract shortened by UMI.)

Face Image Modality Recognition and Photo-Sketch Matching

Face is an important physical characteristic of human body, and is widely used in many crucial applications, such as video surveillance, criminal investigation, and security access system. Based on realistic demand, such as useful face images in dark environment and criminal profile, different modalities of face images appeared, e.g. three-dimensional (3D), near infrared (NIR), and thermal infrared (TIR) face images. Thus, researches with various face image modalities become a hot area. Most of them are set on knowing the modality of face images in advance, which contains a few limitations. In this thesis, we present approaches for face image modality recognition to extend the possibility of cross-modality researches as well as handle new modality-mixed face images. Furthermore, a large facial image database is assembled with five commonly used modalities such as 3D, NIR, TIR, sketch, and visible light spectrum (VIS). Based on the analysis of results, a feature descriptor based on convolutional neural network with linear kernel SVM did an optimal performance.;As we mentioned above, face images are widely used in crucial applications, and one of them is using the sketch of suspect\u27s face, which based on the witness\u27 description, to assist law enforcement. Since it is difficult to capture face photos of the suspect during a criminal activity, automatic retrieving photos based on the suspect\u27s facial sketch is used for locating potential suspects. In this thesis, we perform photo-sketch matching by synthesizing the corresponding pseudo sketch from a given photo. There are three methods applied in this thesis, which are respectively based on style transfer, DualGAN, and cycle-consistent adversarial networks. Among the results of these methods, style transfer based method did a poor performance in photo-sketch matching, since it is an unsupervised one which is not purposeful in photo to sketch synthesis problem while the others need to train pointed models in synthesis stage

Sparse representation based stereoscopic image quality assessment accounting for perceptual cognitive process

In this paper, we propose a sparse representation based Reduced-Reference Image Quality Assessment (RR-IQA) index for stereoscopic images from the following two perspectives: 1) Human visual system (HVS) always tries to infer the meaningful information and reduces uncertainty from the visual stimuli, and the entropy of primitive (EoP) can well describe this visual cognitive progress when perceiving natural images. 2) Ocular dominance (also known as binocularity) which represents the interaction between two eyes is quantified by the sparse representation coefficients. Inspired by previous research, the perception and understanding of an image is considered as an active inference process determined by the level of “surprise”, which can be described by EoP. Therefore, the primitives learnt from natural images can be utilized to evaluate the visual information by computing entropy. Meanwhile, considering the binocularity in stereo image quality assessment, a feasible way is proposed to characterize this binocular process according to the sparse representation coefficients of each view. Experimental results on LIVE 3D image databases and MCL database further demonstrate that the proposed algorithm achieves high consistency with subjective evaluation