Gender and Ethnicity Classification based on Palmprint and Palmar Hand Images from Uncontrolled Environment

Soft biometric attributes such as gender, ethnicity or age may provide useful information for biometrics and forensics applications. Researchers used, e.g., face, gait, iris, and hand, etc. to classify such attributes. Even though hand has been widely studied for biometric recognition, relatively less attention has been given to soft biometrics from hand. Previous studies of soft biometrics based on hand images focused on gender and well-controlled imaging environment. In this paper, the gender and ethnicity classification in uncontrolled environment are considered. Gender and ethnicity labels are collected and provided for subjects in a publicly available database, which contains hand images from the Internet. Five deep learning models are fine-tuned and evaluated in gender and ethnicity classification scenarios based on palmar 1) full hand, 2) segmented hand and 3) palmprint images. The experimental results indicate that for gender and ethnicity classification in uncontrolled environment, full and segmented hand images are more suitable than palmprint images.Comment: Accepted in the International Joint Conference on Biometrics (IJCB 2020), scheduled for Sep 28-Oct 1, 202

Palmprint Recognition in Uncontrolled and Uncooperative Environment

Online palmprint recognition and latent palmprint identification are two branches of palmprint studies. The former uses middle-resolution images collected by a digital camera in a well-controlled or contact-based environment with user cooperation for commercial applications and the latter uses high-resolution latent palmprints collected in crime scenes for forensic investigation. However, these two branches do not cover some palmprint images which have the potential for forensic investigation. Due to the prevalence of smartphone and consumer camera, more evidence is in the form of digital images taken in uncontrolled and uncooperative environment, e.g., child pornographic images and terrorist images, where the criminals commonly hide or cover their face. However, their palms can be observable. To study palmprint identification on images collected in uncontrolled and uncooperative environment, a new palmprint database is established and an end-to-end deep learning algorithm is proposed. The new database named NTU Palmprints from the Internet (NTU-PI-v1) contains 7881 images from 2035 palms collected from the Internet. The proposed algorithm consists of an alignment network and a feature extraction network and is end-to-end trainable. The proposed algorithm is compared with the state-of-the-art online palmprint recognition methods and evaluated on three public contactless palmprint databases, IITD, CASIA, and PolyU and two new databases, NTU-PI-v1 and NTU contactless palmprint database. The experimental results showed that the proposed algorithm outperforms the existing palmprint recognition methods.Comment: Accepted in the IEEE Transactions on Information Forensics and Securit

Criminal and victim identification based on deep and large feature sets from hand biometrics

In forensics, criminal and victim identification based on digital evidence images is highly challenging because the face or other obvious characteristics such as tattoos are occluded, covered, or not visible at all. Existing recognition methods, which make use of biometric characteristics, such as vein, skin mark, height, skin color, weight, race, etc., are considered for solving this problem. The soft biometric traits, including skin color, gender, height, weight and race, provide useful information but not distinctive enough. Veins and skin marks are limited to high-resolution images and some body sites may neither have enough skin marks nor clear veins. Regardless of the availability of these characteristics, other ones, e.g., hands can be used to support the evidence or provide some useful clues to the investigator. Terrorists and rioters tend to expose their hands, including palms and wrists in a gesture of triumph, greeting or salute, while child sexual offenders usually show them when touching their victims. Wrists, in particular, were neglected by the biometric community for forensic applications. To study this problem, a wrist identification algorithm, which includes skin segmentation, key point localization, image to template alignment, large feature set extraction, classification and post-recognition score analysis has been proposed. The proposed algorithm has been evaluated on a new NTU-Wrist-Image-Database-v1, which consists of 3945 images from 731 different wrists, including 205 pairs of wrist images collected from the Internet, taken under uneven illuminations with different poses and resolutions. In the experiments, the proposed algorithm has been compared to palmprint recognition methods. The extracted large feature sets have been studied and compared with selected deep features and feature selection and reduction schemes. The experimental results indicated that wrist is a useful clue for criminal and victim identification. Online palmprint recognition and latent palmprint identification are two branches of palmprint studies. The former uses middle-resolution images collected by a digital camera in a well-controlled or contact-based environment with user cooperation for commercial applications and the latter uses high-resolution latent palmprints collected in crime scenes for forensic investigation. Nevertheless, the merit of contactless palmprint recognition in an uncontrolled and uncooperative environment for forensic investigation is not fully exposed yet. To study this problem, a new palmprint database is established and an end-to-end deep learning algorithm has been proposed. The new database named NTU Palmprints from the Internet (NTU-PI-v1) contains 7881 images from 2035 palms collected from the Internet. The proposed algorithm consists of an alignment network, feature extraction network, in-network data augmentation scheme, and is end-to-end trainable. The proposed algorithm has been compared with the state-of-the-art online palmprint recognition methods and evaluated on three publicly available contactless palmprint databases, IITD, CASIA, and PolyU and two new databases, NTU-PI-v1 and NTU contactless palmprint database. The experimental results showed that the proposed algorithm outperforms the existing palmprint recognition methods. In child sexual abuse images, the chest of the offender can be sometimes visible or even captured simultaneously with the victim's hand. Thus, the nipple-areola complex (NAC) is proposed for offender identification. Popular deep learning and hand-crafted methods are evaluated on a newly established NTU-Nipple-v1 database, which contains 2732 images from 428 different male's NAC. Experimental results indicate that the proposed NAC can be useful for offender identification.Doctor of Philosoph

A portrait photo-to-tattoo transform based on digital tattooing

Tattooing portraits of loved ones is a popular form of love expression and tribute. Tattooing portraits is complicated and challenging because of detailed facial expressions and unique characters of each person. Currently, it is hard for clients to give clear instructions on tattoo designs to tattooists, because there is no effective way to see a portrait tattoo before putting it on the body. In this paper, an algorithm which transforms a given portrait photo to a portrait tattoo is proposed. It takes a portrait photo, a reference portrait tattoo image, a skin image and a set of parameters as inputs. The portrait photo is the person’s face whom the client wants to put on his/her skin. The reference portrait tattoo image is used to control the color and style of the synthetic portrait tattoo. The skin image is taken from the skin region where the client wants to tattoo. By adjusting the parameters, portrait tattoos with different characteristics can be generated. The proposed algorithm uses a series of tailor-made image processing methods and a digital tattoo needle model to perform digital tattooing on the skin image. Comparing with the state-of-the-art style transfer methods, the proposed algorithm produces more realistic portrait tattoos.Ministry of Education (MOE)This work is partially supported by the Ministry of Education, Singapore through Academic Research Fund Tier 1, RG30/17 (S)

A study on giant panda recognition based on images of a large proportion of captive pandas

As a highly endangered species, the giant panda (panda) has attracted significant attention in the past decades. Considerable efforts have been put on panda conservation and reproduction, offering the promising outcome of maintaining the population size of pandas. To evaluate the effectiveness of conservation and management strategies, recognizing individual pandas is critical. However, it remains a challenging task because the existing methods, such as traditional tracking method, discrimination method based on footprint identification, and molecular biology method, are invasive, inaccurate, expensive, or challenging to perform. The advances of imaging technologies have led to the wide applications of digital images and videos in panda conservation and management, which makes it possible for individual panda recognition in a noninvasive manner by using image-based panda face recognition method. In recent years, deep learning has achieved great success in the field of computer vision and pattern recognition. For panda face recognition, a fully automatic deep learning algorithm which consists of a sequence of deep neural networks (DNNs) used for panda face detection, segmentation, alignment, and identity prediction is developed in this study. To develop and evaluate the algorithm, the largest panda image dataset containing 6,441 images from 218 different pandas, which is 39.78% of captive pandas in the world, is established. The algorithm achieved 96.27% accuracy in panda recognition and 100% accuracy in detection. This study shows that panda faces can be used for panda recognition. It enables the use of the cameras installed in their habitat for monitoring their population and behavior. This noninvasive approach is much more cost-effective than the approaches used in the previous panda surveys.Published versio