Automated Person Identification Framework Based on Fingernails and Dorsal Knuckle Patterns

Hand images are of paramount importance within critical domains like security and criminal investigation. They can sometimes be the only available evidence of an offender’s identity at a crime scene. Approaches to person identification that consider the human hand as a complex object composed of many components are rare. The approach proposed in this paper fills this gap, making use of knuckle creases and fingernail information. It introduces a framework for automatic person identification that includes localisation of the regions of interest within hand images, recognition of the detected components, segmentation of the region of interest using bounding boxes, and similarity matching between a query image and a library of available images. The following hand components are considered: i) the metacarpohalangeal, commonly known as base knuckle; ii) the proximal interphalangeal joint commonly known as major knuckle; iii) distal interphalangeal joint, commonly known as minor knuckle; iv) the interphalangeal joint, commonly known as thumb’s knuckle, and v) the fingernails. A key element of the proposed framework is the similarity matching and an important role for it is played by the feature extraction. In this paper, we exploit end-to-end deep convolutional neural networks to extract discriminative high-level abstract features. We further use BrayCurtis (BC) similarity for the matching process. We validated the proposed approach on well-known benchmarks, the ’11k Hands’ dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as ’PolyU HD’. We found that the results indicate that the knuckle patterns and fingernails play a significant role in the person identification. The results from the 11K dataset indicate that the results for the left hand are better than the results for the right hand. In both datasets, the fingernails produced consistently higher identification results than other hand components, with a rank-1 score of 93.65% on the ring finger of the left hand for the ’11k Hands’ dataset and rank-1 score of 93.81% for the thumb from the ’PolyU HD’ dataset

Machine Learning Methods for Human Identification from Dorsal Hand Images

Person identification is a process that uniquely identifies an individual based on physical or behavioural traits. This study investigates methods for the analysis of images of the human hand, focusing on their uniqueness and potential use for human identification. The human hand has significant and distinctive characteristics, and is highly complex and interesting, yet it has not been explored in much detail, particularly in the context of the contemporary high level of digitalisation and, more specifically, the advances in artificial intelligence (AI), machine learning (ML) and computer vision (CV). This research area is highly multi-disciplinary, involving anatomists, anthropologists, bioinformaticians, image analysts and, increasingly, computer scientists. A growing pool of advanced methods based on AI, ML and CV can benefit and relate directly to a better representation of the human hand in computer analysis. Historically, the research methods in this area relied on ‘handcrafted’ features such as the local binary pattern (LBP) and histogram of gradient (HOG) extraction, which necessitated human intervention. However, such approaches struggle to encode the human hand in variable conditions effectively, because of the change in camera viewpoint, hand pose, rotation, image quality, and self-occlusion. Thus, their performance is limited. Recently, there has been a surge of interest in deep learning neural network (DLNN) approaches, specifically convolutional neural networks (CNNs), due to the highly accurate results achieved in many applications and the wide availability of images. This work investigates advanced methods based on ML and DLNN for segmenting hand images with various rotation changes into different patches (e.g., knuckles and fingernails). The thesis focuses on developing ML methods like pre-trained CNN models on the 'ImageNet' dataset to learn the underlying structure of the human hand by extracting robust features from hand images with diverse conditions of rotation, and image quality. Also, this study investigates fine-tuning the pre-trained models of DLNN on subsets of hand images, as well as using various similarity metrics to find the best match of the individual’s hand. Furthermore, this work explores different types of ensemble learning or fusions, those of different region and similarity metrics to improve human identification results. This thesis also presents a study of a Siamese network on sub-images or segments of human dorsal hands for identification tasks. All presented approaches are compared with the state-of-the-art methods. This study advances the understanding of variations in and the uniqueness of humans using patches of their hands (e.g., different types of knuckles and fingernails). Lastly, it compares the matching performances of the left- and right-hand patches using various hand datasets and investigates whether the fingernail produces better identification results than the knuckles. This research shows that the proposed framework for person identification based on hand components led to better person identification results. The framework consists of vital feature extractions based on deep learning neural network (DLNN) and similarity metrics. These elements enhanced the performance. Also, the fingernails' shape performed better than other hand components, including the base, major, and minor knuckles. The left hand can be more distinguishable to individuals than the right hand. The fine-tuning of the hand components and ensemble learning improved the identification results

WearPut : Designing Dexterous Wearable Input based on the Characteristics of Human Finger Motions

Department of Biomedical Engineering (Human Factors Engineering)Powerful microchips for computing and networking allow a wide range of wearable devices to be miniaturized with high fidelity and availability. In particular, the commercially successful smartwatches placed on the wrist drive market growth by sharing the role of smartphones and health management. The emerging Head Mounted Displays (HMDs) for Augmented Reality (AR) and Virtual Reality (VR) also impact various application areas in video games, education, simulation, and productivity tools. However, these powerful wearables have challenges in interaction with the inevitably limited space for input and output due to the specialized form factors for fitting the body parts. To complement the constrained interaction experience, many wearable devices still rely on other large form factor devices (e.g., smartphones or hand-held controllers). Despite their usefulness, the additional devices for interaction can constrain the viability of wearable devices in many usage scenarios by tethering users' hands to the physical devices. This thesis argues that developing novel Human-Computer interaction techniques for the specialized wearable form factors is vital for wearables to be reliable standalone products. This thesis seeks to address the issue of constrained interaction experience with novel interaction techniques by exploring finger motions during input for the specialized form factors of wearable devices. The several characteristics of the finger input motions are promising to enable increases in the expressiveness of input on the physically limited input space of wearable devices. First, the input techniques with fingers are prevalent on many large form factor devices (e.g., touchscreen or physical keyboard) due to fast and accurate performance and high familiarity. Second, many commercial wearable products provide built-in sensors (e.g., touchscreen or hand tracking system) to detect finger motions. This enables the implementation of novel interaction systems without any additional sensors or devices. Third, the specialized form factors of wearable devices can create unique input contexts while the fingers approach their locations, shapes, and components. Finally, the dexterity of fingers with a distinctive appearance, high degrees of freedom, and high sensitivity of joint angle perception have the potential to widen the range of input available with various movement features on the surface and in the air. Accordingly, the general claim of this thesis is that understanding how users move their fingers during input will enable increases in the expressiveness of the interaction techniques we can create for resource-limited wearable devices. This thesis demonstrates the general claim by providing evidence in various wearable scenarios with smartwatches and HMDs. First, this thesis explored the comfort range of static and dynamic touch input with angles on the touchscreen of smartwatches. The results showed the specific comfort ranges on variations in fingers, finger regions, and poses due to the unique input context that the touching hand approaches a small and fixed touchscreen with a limited range of angles. Then, finger region-aware systems that recognize the flat and side of the finger were constructed based on the contact areas on the touchscreen to enhance the expressiveness of angle-based touch input. In the second scenario, this thesis revealed distinctive touch profiles of different fingers caused by the unique input context for the touchscreen of smartwatches. The results led to the implementation of finger identification systems for distinguishing two or three fingers. Two virtual keyboards with 12 and 16 keys showed the feasibility of touch-based finger identification that enables increases in the expressiveness of touch input techniques. In addition, this thesis supports the general claim with a range of wearable scenarios by exploring the finger input motions in the air. In the third scenario, this thesis investigated the motions of in-air finger stroking during unconstrained in-air typing for HMDs. The results of the observation study revealed details of in-air finger motions during fast sequential input, such as strategies, kinematics, correlated movements, inter-fingerstroke relationship, and individual in-air keys. The in-depth analysis led to a practical guideline for developing robust in-air typing systems with finger stroking. Lastly, this thesis examined the viable locations of in-air thumb touch input to the virtual targets above the palm. It was confirmed that fast and accurate sequential thumb touch can be achieved at a total of 8 key locations with the built-in hand tracking system in a commercial HMD. Final typing studies with a novel in-air thumb typing system verified increases in the expressiveness of virtual target selection on HMDs. This thesis argues that the objective and subjective results and novel interaction techniques in various wearable scenarios support the general claim that understanding how users move their fingers during input will enable increases in the expressiveness of the interaction techniques we can create for resource-limited wearable devices. Finally, this thesis concludes with thesis contributions, design considerations, and the scope of future research works, for future researchers and developers to implement robust finger-based interaction systems on various types of wearable devices.ope

Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level

An evaluation of the mechanisms of recovery of DNA and fingerprints from fire scenes

Incidents involving the intentional or deliberate setting of a fire within a compartment are frequently difficult to investigate both because of the damage to the property in question and the apparent lack of forensic evidence which could be used to potentially identify a suspect. The recovery of such evidence in the form of DNA and fingerprints from a fire scene would therefore be advantageous. During this project, replicate samples of DNA and fingerprints were deposited on both porous and non porous surfaces which were then exposed to laboratory controlled elevated temperatures for various time periods. In each case replicate DNA samples or replicate depleted series of fingerprint samples were used to produce robust data sets for subsequent statistical analysis. DNA and fingerprint samples were also exposed to a real fire environment using a fire training facility in order to simulate operational conditions. The results obtained suggest that the optimum recovery method for low template DNA was to use a wet followed by a dry cotton swabbing action of the surface before combining the two swabs for extraction. When the DNA was exposed to elevated temperatures in a controlled environment, there was a greater possibility of recovering a full SGM Plus profile if the DNA had been absorbed into a porous rather than non porous surface and the surface exposed up to a maximum of 100˚C only. All of the samples which were exposed to the uncontrollable fire environment produced partial DNA profiles. The survivability and chemical enhancement of fingerprints deposited on both porous and non porous surfaces was robustly investigated where 70 replicate fingerprints were examined in each case for each test condition. For porous surfaces the most efficient sequence of enhancement techniques was an initial visual examination, followed by a fluorescence examination prior to treatment with DFO, and finally PD. It was found that this sequence could be employed for both wet and dry articles. In the case of dry, non porous surfaces, visual examination followed by fluorescence examination should be utilised prior to undertaking superglue - BY40 treatment. Powder suspension should be substituted for superglue in the case of wet items.Incidents involving the intentional or deliberate setting of a fire within a compartment are frequently difficult to investigate both because of the damage to the property in question and the apparent lack of forensic evidence which could be used to potentially identify a suspect. The recovery of such evidence in the form of DNA and fingerprints from a fire scene would therefore be advantageous. During this project, replicate samples of DNA and fingerprints were deposited on both porous and non porous surfaces which were then exposed to laboratory controlled elevated temperatures for various time periods. In each case replicate DNA samples or replicate depleted series of fingerprint samples were used to produce robust data sets for subsequent statistical analysis. DNA and fingerprint samples were also exposed to a real fire environment using a fire training facility in order to simulate operational conditions. The results obtained suggest that the optimum recovery method for low template DNA was to use a wet followed by a dry cotton swabbing action of the surface before combining the two swabs for extraction. When the DNA was exposed to elevated temperatures in a controlled environment, there was a greater possibility of recovering a full SGM Plus profile if the DNA had been absorbed into a porous rather than non porous surface and the surface exposed up to a maximum of 100˚C only. All of the samples which were exposed to the uncontrollable fire environment produced partial DNA profiles. The survivability and chemical enhancement of fingerprints deposited on both porous and non porous surfaces was robustly investigated where 70 replicate fingerprints were examined in each case for each test condition. For porous surfaces the most efficient sequence of enhancement techniques was an initial visual examination, followed by a fluorescence examination prior to treatment with DFO, and finally PD. It was found that this sequence could be employed for both wet and dry articles. In the case of dry, non porous surfaces, visual examination followed by fluorescence examination should be utilised prior to undertaking superglue - BY40 treatment. Powder suspension should be substituted for superglue in the case of wet items

Osteoarthritis: From Molecular Pathways to Therapeutic Advances

In this book, we have reported the most recent discoveries and updates regarding molecular pathways in osteoarthritis. In particular, the advances regarding therapeutical options, from a molecular point of view, are largely discussed