A Survey on Artificial Intelligence Techniques for Biomedical Image Analysis in Skeleton-Based Forensic Human Identification

This paper represents the first survey on the application of AI techniques for the analysis of biomedical images with forensic human identification purposes. Human identification is of great relevance in today’s society and, in particular, in medico-legal contexts. As consequence, all technological advances that are introduced in this field can contribute to the increasing necessity for accurate and robust tools that allow for establishing and verifying human identity. We first describe the importance and applicability of forensic anthropology in many identification scenarios. Later, we present the main trends related to the application of computer vision, machine learning and soft computing techniques to the estimation of the biological profile, the identification through comparative radiography and craniofacial superimposition, traumatism and pathology analysis, as well as facial reconstruction. The potentialities and limitations of the employed approaches are described, and we conclude with a discussion about methodological issues and future research.Spanish Ministry of Science, Innovation and UniversitiesEuropean Union (EU) PGC2018-101216-B-I00Regional Government of Andalusia under grant EXAISFI P18-FR-4262Instituto de Salud Carlos IIIEuropean Union (EU) DTS18/00136European Commission H2020-MSCA-IF-2016 through the Skeleton-ID Marie Curie Individual Fellowship 746592Spanish Ministry of Science, Innovation and Universities-CDTI, Neotec program 2019 EXP-00122609/SNEO-20191236European Union (EU)Xunta de Galicia ED431G 2019/01European Union (EU) RTI2018-095894-B-I0

A Forensic Identification Utility to Create Facial Approximations using Cone-Beam Computed Tomography of 100 Hispanic Females: A Pilot Study

Introduction:Estimation of facial soft tissue appearance from human skeletal remains is often necessary in forensic identification. This process has been referred to as facial reconstruction or facial approximation and is a branch of forensic facial anthropology. Original methods for facial approximation originated in nineteenth century Europe and consisted of artists shaping clay over skull models using average soft tissue depths measured in cadavers. The last two decades have introduced numerous computerized techniques that have digitized this process while attempting to accurately and objectively define the relationship between a skull and its overlying soft tissue. This pilot study describes a method of facial approximation that combines cephalometric techniques for characterization of the craniofacial complex commonly used in the field of orthodontics with a database of cone-beam computed tomography (CBCT) skull images. Facial likenesses for an unknown skull are automatically located within the database by comparing cephalometric values recorded on the unknown skull with those within the database. A recently proposed method of sex determination based on the anatomy of the mastoid process, glabellar process, and frontal sinus area is also applied to the sample used in this study. Methods:A database consisting of one-hundred (100) cone-beam computed tomography (CBCT) skull images of Hispanic female patients of the University of Las Vegas, Nevada School of Dental Medicine Orthodontic Department [age range 8 to 23 years (mean 13.5 years)] was constructed. A cephalometric analysis consisting of twelve (12) landmarks and nineteen (19) skull measurements [sixteen (16) angular and three (3) proportional] was defined and applied to all database entries. Facial approximations were created for three skulls by sequentially removing three (3) random entries from the database and treating these as unknown (leave-one-out cross validation). A weighted least-sum-of-squares (WLSS) regression algorithm was applied to measure the cephalometric similarity between each entry in the database and the unknown skull data to find the three (3) most cephalometrically similar skulls in the database (three closest matches). Accuracy was assessed through expert face pool resemblance ranking. Soft tissue profiles associated with the three best matches were grouped with three random database entries to create a face pool array of size six (6) for each unknown. Fourteen (14) post-doctoral orthodontic graduate students were utilized as expert face pool evaluators. Sex determination accuracy was then assessed by comparing the values of eight (8) cephalometric measurements taken on this sample to those already described and proven efficacious on other samples in the literature. Results:Intraexaminer reliability was acceptable for all cephalometric measurements. Expert face pool resemblance rankings results implied that the described process was able to select database entries that approximated the unknown face better than random database entries. In Face Pools One, Two, and Three the three highest ranked faces contained two, two, and three algorithm-selected faces, respectively. Sex determination data recorded on this sample was comparable to data described in the literature. Conclusions:Contemporary methods of facial approximation have shown that estimation of soft tissues from skeletal data can be achieved by employing computationally and graphically complex techniques. It now also seems plausible to rapidly estimate the general shape of an unidentified skull\u27s facial profile by comparison of the unknown skull\u27s cephalometric data to those in a database of orthodontic patients. Further research involving the described method is warranted

Detecção automática de pontos cefalométricos em imagens faciais: uma abordagem aplicada na estimação de idade e sexo a partir da norma frontal

Tese (doutorado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Mecânica, 2019.Métodos forenses para estimação das informações de indivíduos são constantemente utilizados por peritos em cenários reais. O processo forense de estimação de idade e sexo necessita de um profissional capacitado e abordagens que normalmente exigem a presença física do indivíduo para a execução dos procedimentos periciais. A pornografia infantil é uma atividade ilícita que conta com a facilidade da internet para o compartilhamento de imagens e vídeos tornando fácil a disseminação deste tipo de conteúdo e dificultando a identificação e perícia do material. Pesquisas sobre técnicas de reconhecimento de padrões e aprendizado de máquina em visão computacional permitem o desenvolvimento de metodologias baseadas em fotoantropometria para a identificação de informações antropométricas apenas utilizando imagens faciais digitais. Com o avanço da tecnologia e com o aumento do volume de dados, os profissionais necessitam de alternativas para processar as informações antropométricas de indivíduos apenas analisando arquivos digitais. Este trabalho tem como objetivo desenvolvimento de metodologias de identificação automática de pontos cefalométricos em imagens faciais, gerar medidas fotoantropométricas e propor uma solução computacional para auxiliar os profissionais forenses para a estimação da idade e sexo em uma base de dados com 105 mil de imagens faciais. O trabalho proposto para identificação de pontos cefalométricos obteve precisão similar com as marcações realizadas por especialistas com resultado de erro médio da distância (em pixels) normalizada de 0:014 contra 0:009 de dispersão média dos testes entre especialistas. Os resultados obtidos demonstram significância no processo de estimação de dados antropométricos, utilizando imagens faciais com redes neurais convolucionais e medidas fotoantropométricas faciais. A proposta desenvolvida obteve resultado de 99; 2% de acerto para estimação de sexo. Para estimação de maior/menor de 18 anos o resultado F1 score foi de 0; 926 enquanto para maior/menor de 14 anos foi de 0; 957. Por último, estimação de idade, a proposta obteve um resultado MAE de 1; 42 utilizando uma amostra de indivíduos com idades entre 2 a 22 anos.Forensic methods of estimating information from individuals are constantly used by experts in real scenarios. The forensic process of age and sex estimation requires an expert and approaches that normally need the physical presence of the individual for the execution of the expert procedures. Child pornography is an illicit activity that relies on the ease of the internet to acess and disseminate this type of content, making it difficult to identify and exploit the material. Research on pattern recognition techniques and machine learning in computer vision allow the development of methodologies based on photo-anthropometry for identification of anthropometric information using facial images. The advancement of technology and the increasing process of the data volume, the experts need alternatives to be inferring the age and of individuals by analyzing digital files only. This work has the goal to develop methodologies for automatic identification of cephalometric points in facial images, generate photo-anthropometric measurements and propose a computational solution to assist forensic professionals to estimate age and sex in a database with thousands of images. The proposed work to identify cephalometric landmarks obtained similar accuracy with the manual points made by experts with result of normalized average error (in pixels) of 0:014 versus 0:009 of average dispersion by experts. The results present significance in the estimation process of anthropometric data using facial images with convolutional neural networks and facial photo-anthropometric measurements. The developed proposal obtained 99:2% positive results for sex estimation. For the estimation of over 18 years old the result of F1 was 0:926 while for over 14 years old it was 0:957. Finally, the age estimate, the proposal obtained a MAE result of 1:42 using a sample of individuals over ages from 2 to 22 years old

Contributions to the three-dimensional virtual treatment planning of orthognathic surgery

Orientadores: José Mario De Martino, Luis Augusto PasseriTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: A tecnologia mais recente à disposição da Cirurgia Ortognática possibilita que o diagnóstico e o planejamento do tratamento das deformidades dentofaciais sejam realizados sob uma representação virtual tridimensional (3D) da cabeça do paciente. Com o propósito de contribuir para o aperfeiçoamento desta tecnologia, o trabalho apresentado nesta tese identificou e tratou quatro problemas. A primeira contribuição consistiu na verificação da validade da hipótese de que a mudança de definição do plano horizontal de Frankfort não produz diferenças de medição clinicamente relevantes quando sob indivíduos cujos crânios são consideravelmente simétricos. Os resultados da análise realizada no contexto deste tese indicam que, ao contrário do que se presumia, a hipótese é falsa. A segunda contribuição consistiu na extensão do método de análise cefalométrica de McNamara para que ele pudesse produzir valores 3D. Ao contrário de outros métodos de análise cefalométrica 3D, a extensão criada produz valores verdadeiramente 3D, não perde as informações do método original e preserva as definições geométricas originais das linhas e planos cefalométricos. A terceira contribuição consistiu a) no estabelecimento de normas cefalométricas para brasileiros adultos de ascendência europeia, a partir de imagens de tomografia computadorizada de feixe cônico, que produz uma imagem craniofacial mais precisa e confiável do que a telerradiografia; e b) na avaliação de dimorfismo sexual, para a identificação de características anatômicas diferenciadas entre homens e mulheres desta população. A quarta e última contribuição consistiu na automatização da principal etapa da tecnologia em questão, na qual o cirurgião executa o reposicionamento dos segmentos ósseos maxilares no crânio. O método criado é capaz de corrigir automaticamente os problemas dentofaciais mais comuns tratados pela Cirurgia Ortognática, que envolvem maloclusão esquelética, assimetria facial e discrepância de maxilares. Todas as contribuições deste trabalho foram publicadas em periódicos internacionais do campo da Odontologia e afinsAbstract: The latest technology available for orthognathic surgery allows the diagnosis and treatment planning of dentofacial deformities based on a three-dimensional (3D) virtual representation of the patient's head. In order to contribute to the improvement of this technology, the work presented in this thesis identified and treated four problems. The first contribution consisted in testing the validity of the hypothesis that changing the definition of the Frankfort horizontal plane does not produce clinically relevant measurement differences for subjects whose skulls are considerably symmetrical. The results of the analysis performed in this thesis indicate that, contrary to what was presumed, the hypothesis is false. The second contribution is an extension of the McNamara's method of cephalometric analysis to produce 3D values. Unlike other methods of 3D cephalometric analysis, the extension produces true 3D values, does not lose information captured by the original method, and preserves the original geometric definitions of the cephalometric lines and planes. The third contribution consisted in a) establishing cephalometric norms for Brazilian adults of European descent, based on images from cone-beam computed tomography, which produce a more accurate and reliable craniofacial image than cephalometric radiography; and b) evaluating sexual dimorphism, for the identification of distinct anatomic features between males and females of this population. The fourth contribution consisted in automating the main stage of the technology in question, in which the surgeon performs the positioning of jaw bone segments in the skull. The created method is able to automatically correct the most common dentofacial problems treated by orthognathic surgery, which involves skeletal malocclusion, facial asymmetry, and jaw discrepancy. The contributions of this work were published in international journals of the field of Dentistry and relatedDoutoradoEngenharia de ComputaçãoDoutor em Engenharia ElétricaCAPE

Sexual dimorphism in white South African crania

Original published work submitted to the Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Doctor of Philosophy. Johannesburg, 2016Sexual dimorphism is one of four critical factors assessed by forensic anthropologists when compiling biological profiles. The current study used geometric morphometric methods to analyse various aspects of sexual dimorphism in white South African crania to significantly contribute to current forensic standards for this underrepresented population. As edentulous crania are a major contributing factor to the low number of publications on white South African populations, the question arose as to how tooth loss affects cranial structures and the accuracy of sex and ancestry estimation. Two hundred and twenty nine crania were digitised using landmarks and sliding semilandmarks, both globally and for a number of cranial subsets. Although a number of effects were identified when the skull was analysed globally, only the maxillary alveolar ridges were significantly affected when subsets were analysed individually. As both upper facial height and palate shape were significantly altered by tooth loss, the effects of tooth loss on cranial structures and sex and ancestry estimations were investigated. Next, to parse out the mechanisms by which sexual dimorphism causes morphological variation, overall sexual dimorphism, common allometry and nonallometric sexual dimorphism were individually assessed. Global and subset data were studied and the effects of sexual dimorphism and allometry were found to be universal, with significant differences being observed between the sexes both globally and regionally. A significant non-allometric component was, however, only found to contribute to the shape of the zygomatic bone. Finally, the accuracy of 17 widely used traditional cranial measurements was compared to all possible interlandmark distances (ILDs) attainable from 45 fixed landmarks. Discriminant functions derived using the ILDs compared well to those of previous work on white South Africans, thus demonstrating the similarity between traditional and 3-D methods. Finally, custom discriminant functions were created for a number of cranial subsets and for the cranium in its entirety. The subsets achieved sexing accuracies ranging between 71.8% and 83.7%, with the nasomaxilla proving most accurate. The overall cranial function attained a cross-validated sexing accuracy of 88.2%. These functions are critical for sex estimation not only for intact crania, but also for the innumerable fragmentary cranial remains recovered regularly in South Africa.MT201

Craniofacial identification standards: a review of reliability, reproducibility, and implementation

There are numerous anatomical and anthropometrical standards that can be utilised for craniofacial analysis and identification. These standards originate from a wide variety of sources, such as orthodontic, maxillofacial, surgical, anatomical, anthropological and forensic literature, and numerous media have been employed to collect data from living and deceased subjects. With the development of clinical imaging and the enhanced technology associated with this field, multiple methods of data collection have become accessible, including Computed Tomography, Cone-Beam Computed Tomography, Magnetic Resonance Imaging, Radiographs, Three-dimensional Scanning, Photogrammetry and Ultrasound, alongside the more traditional in vivo methods, such as palpation and direct measurement, and cadaveric human dissection. Practitioners often struggle to identify the most appropriate standards and research results are frequently inconsistent adding to the confusion. This paper aims to clarify how practitioners can choose optimal standards, which standards are the most reliable and when to apply these standards for craniofacial identification. This paper describes the advantages and disadvantages of each mode of data collection and collates published research to review standards across different populations for each facial feature. This paper does not aim to be a practical instruction paper; since this field encompasses a wide range of 2D and 3D approaches (e.g., clay sculpture, sketch, automated, computer-modelling), the implementation of these standards is left to the individual practitioner

Recent Advances in Forensic Anthropological Methods and Research

Forensic anthropology, while still relatively in its infancy compared to other forensic science disciplines, adopts a wide array of methods from many disciplines for human skeletal identification in medico-legal and humanitarian contexts. The human skeleton is a dynamic tissue that can withstand the ravages of time given the right environment and may be the only remaining evidence left in a forensic case whether a week or decades old. Improved understanding of the intrinsic and extrinsic factors that modulate skeletal tissues allows researchers and practitioners to improve the accuracy and precision of identification methods ranging from establishing a biological profile such as estimating age-at-death, and population affinity, estimating time-since-death, using isotopes for geolocation of unidentified decedents, radiology for personal identification, histology to assess a live birth, to assessing traumatic injuries and so much more

3D face morphology classification for medical applications

Classification of facial morphology traits is an important problem for many medical applications, especially with regard to determining associations between facial morphological traits or facial abnormalities and genetic variants. A modern approach to the classification of facial characteristics(traits) is to use three-dimensional facial images. In clinical practice, classification is usually performed manually, which makes the process very tedious, time-consuming and prone to operator error. Also using simple landmark-to-landmark facial measurements may not accurately represent the underlying complex three-dimensional facial shape. This thesis presents the first automatic approach for classification and categorisation of facial morphological traits with application to lips and nose traits. It also introduces new 3D geodesic curvature features obtained along the geodesic paths between 3D facial anthropometric landmarks. These geometric features were used for lips and nose traits classification and categorisation. Finally, the influence of the discovered categories on the facial physical appearance are analysed using a new visualisation method in order to gain insight into suitability of categories for description of the underlying facial traits. The proposed approach was tested on the ALSPAC (Avon Longitudinal Study of Parents and Children) dataset consisting of 4747 3D full face meshes. The classification accuracy obtained using expert manual categories was not very high, in the region of 72%-79%, indicating that the manual categories may be unreliable. In an attempt to improve these accuracies,an automatic categorisation method was applied. In general,the classification accuracies based on the automatic lip categories were higher than those obtained using the manual categories by at least 8% and the automatic categories were found to be statistically more significant in the lip area than the manual categories. The same approach was used to categorise the nose traits, the result indicating that the proposed categorisation approach was capable of categorising any face morphological trait without the ground truth about its traits categories. Additionally, to test the robustness of the proposed features, they were used in a popular problem of gender classification and analysis. The results demonstrated superior classification accuracy to that of comparable methods. Finally, a discovery phase of a genome wide association analysis(GWAS) was carried out for 11 automatic lip and nose traits categories. As a result, statistically significant associations were found between four traits and six single nucleotide polymorphisms (SNPs). This is a very good result considering that for the 27 manual lip traits categories provided by medical expert, the associations were found between two traits and two SNPs only. This result testifies that the method proposed in this thesis for automatic categorisation of 3D facial morphology has a considerable potential for application to GWAS