Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes

Bone age assessment is a task performed daily in hospitals worldwide. This involves a clinician estimating the age of a patient from a radiograph of the non-dominant hand. Our approach to automated bone age assessment is to modularise the algorithm into the following three stages: segment and verify hand outline; segment and verify bones; use the bone outlines to construct models of age. In this paper we address the final question: given outlines of bones, can we learn how to predict the bone age of the patient? We examine two alternative approaches. Firstly, we attempt to train classifiers on individual bones to predict the bone stage categories commonly used in bone ageing. Secondly, we construct regression models to directly predict patient age. We demonstrate that models built on summary features of the bone outline perform better than those built using the one dimensional representation of the outline, and also do at least as well as other automated systems. We show that models constructed on just three bones are as accurate at predicting age as expert human assessors using the standard technique. We also demonstrate the utility of the model by quantifying the importance of ethnicity and sex on age development. Our conclusion is that the feature based system of separating the image processing from the age modelling is the best approach for automated bone ageing, since it offers flexibility and transparency and produces accurate estimate

Automated Bone Age Assessment: Motivation, Taxonomies, and Challenges

Bone age assessment (BAA) of unknown people is one of the most important topics in clinical procedure for evaluation of biological maturity of children. BAA is performed usually by comparing an X-ray of left hand wrist with an atlas of known sample bones. Recently, BAA has gained remarkable ground from academia and medicine. Manual methods of BAA are time-consuming and prone to observer variability. This is a motivation for developing automated methods of BAA. However, there is considerable research on the automated assessment, much of which are still in the experimental stage. This survey provides taxonomy of automated BAA approaches and discusses the challenges. Finally, we present suggestions for future research

Segmentation of Carpal Bones Using Gradient Inverse Coefficient of Variation with Dynamic Programming Method

Segmentation of the carpal bones (CBs) especially for children above seven years old is a challenging task in computer vision mainly because of poor definitions of the bone contours and the occurrence of the partial overlapping of the bones. Although active contour methods are widely employed in image bone segmentation, they are sensitive to initialization and have limitation in segmenting overlapping objects.  Thus, there is a need for a robust segmentation method for bone segmentation. This paper presents an automatic active boundary-based segmentation method, gradient inverse coefficient of variation, based on dynamic programming (DP-GICOV) method to segment carpal bones on radiographic images of children age 5 to 8 years old. A mapping procedure is designed based on a priori knowledge about the natural growth and the arrangement of carpal bones in human body. The accuracy of the DP-GICOV is compared qualitatively and quantitatively with the de-regularized level set (DRLS) and multi-scale gradient vector flow (MGVF) on a dataset of 20 images of carpal bones from University of Southern California. The presented method is capable to detect the bone boundaries fast and accurate. Results show that the DP-GICOV is highly accurate especially for overlapping bones, which is more than 85% in many cases, and it requires minimal user’s intervention. This method has produced a promised result in overcoming both issues faced by active contours method; initialization and overlapping objects

Predictive Modelling of Bone Ageing

Bone age assessment (BAA) is a task performed daily by paediatricians in hospitalsworldwide. The main reasons for BAA to be performed are: fi�rstly, diagnosis of growth disorders through monitoring skeletal development; secondly, prediction of final adult height; and fi�nally, verifi�cation of age claims. Manually predicting bone age from radiographs is a di�fficult and time consuming task. This thesis investigates bone age assessment and why automating the process will help. A review of previous automated bone age assessment systems is undertaken and we investigate why none of these systems have gained widespread acceptance. We propose a new automated method for bone age assessment, ASMA (Automated Skeletal Maturity Assessment). The basic premise of the approach is to automatically extract descriptive shape features that capture the human expertise in forming bone age estimates. The algorithm consists of the following six modularised stages: hand segmentation; hand segmentation classifi�cation; bone segmentation; feature extraction; bone segmentation classifi�cation; bone age prediction. We demonstrate that ASMA performs at least as well as other automated systems and that models constructed on just three bones are as accurate at predicting age as expert human assessors using the standard technique. We also investigate the importance of ethnicity and gender in skeletal development. Our conclusion is that the feature based system of separating the image processing from the age modelling is the best approach, since it off�ers flexibility and transparency, and produces accurate estimates

Carpals and tarsals of mule deer, black bear and human: an osteology guide for the archaeologist

Existing osteological literature often lacks descriptions and illustrations of the smaller elements, such as hand and foot bones, of animals commonly found in the archaeological record. Black bear (Ursus americanus) and mule deer (Odocoileus hemionus) are both cosmopolitan species and important resources for indigenous peoples, resulting in their widespread presence in faunal assemblages. Additionally, the carpal and tarsal elements of these two mammalian taxa can be difficult to distinguish from human elements because of their similarities in size and shape. Proper identification of faunal and human remains is paramount to responsible cultural resource management (CRM). This thesis presents a textual and photographic osteological guide of black bear and mule deer carpals and tarsals and provides the means for distinguishing these elements from their human counterparts

A Survey on Artificial Intelligence Techniques for Biomedical Image Analysis in Skeleton-Based Forensic Human Identification

This paper represents the first survey on the application of AI techniques for the analysis of biomedical images with forensic human identification purposes. Human identification is of great relevance in today’s society and, in particular, in medico-legal contexts. As consequence, all technological advances that are introduced in this field can contribute to the increasing necessity for accurate and robust tools that allow for establishing and verifying human identity. We first describe the importance and applicability of forensic anthropology in many identification scenarios. Later, we present the main trends related to the application of computer vision, machine learning and soft computing techniques to the estimation of the biological profile, the identification through comparative radiography and craniofacial superimposition, traumatism and pathology analysis, as well as facial reconstruction. The potentialities and limitations of the employed approaches are described, and we conclude with a discussion about methodological issues and future research.Spanish Ministry of Science, Innovation and UniversitiesEuropean Union (EU) PGC2018-101216-B-I00Regional Government of Andalusia under grant EXAISFI P18-FR-4262Instituto de Salud Carlos IIIEuropean Union (EU) DTS18/00136European Commission H2020-MSCA-IF-2016 through the Skeleton-ID Marie Curie Individual Fellowship 746592Spanish Ministry of Science, Innovation and Universities-CDTI, Neotec program 2019 EXP-00122609/SNEO-20191236European Union (EU)Xunta de Galicia ED431G 2019/01European Union (EU) RTI2018-095894-B-I0

Carpal Bone Analysis using Geometric and Deep Learning Models

The recent trend for analyzing 3D shapes in medical application has arisen new challenges for a vast amount of research activities. Quantitative shape comparison is a fundamental problem in computer vision, geometry processing and medical imaging. This thesis is motivated by the availability of carpal bone shape dataset to develop efficient techniques for diagnosis of a variety of wrist diseases and examine human skeletal. This study is conducted in two sections. First, we propose a spectral graph wavelet approach for shape analysis of carpal bones of the human wrist. More precisely, we employ spectral graph wavelets to represent the cortical surface of a carpal bone via the spectral geometric analysis of the Laplace-Beltrami operator in the discrete domain. We then propose global spectral graph wavelet (GSGW) descriptor that is isometric invariant, efficient to compute and combines the advantages of both low-pass and band-pass filters. Subsequently, we perform experiments on shapes of the carpal bones of ten women and ten men from a publicly-available database of wrist bones. Using one-way multivariate analysis of variance (MANOVA) and permutation testing, we show through extensive experiments that the proposed GSGW framework gives a much better performance compared to the global point signature (GPS) embedding approach for comparing shapes of the carpal bones across populations. In the second section, we evaluate bone age to assess children’s biological maturity and to diagnose any growth disorders in children. Manual bone age assessment (BAA) methods are timeconsuming and prone to observer variability by even expert radiologists. These drawbacks motivate us for proposing an accurate computerized BAA method based on human wrist bones X-ray images. We also investigate automated BAA methods using state-of-the-art deep learning models that estimate the bone age more accurate than the manual methods by eliminating human observation variations. The presented approaches provide faster assessment process and cost reduction in the hospitals/clinics. The accuracy of our experiments is evaluated using mean absolute error (MAE), and the results demonstrate that exploiting InceptionResNet-V2 model in our architecture achieves higher performance compared to the other used pre-trained models

Functional adaptation of internal bone structure in the wrist of extant hominids and fossil hominins

The shape of wrist bones (carpals) in living hominids are thought to be adapted to the primary function of the hand, which in Homo sapiens is for manipulation, and in non-human hominids, locomotion. However, the hominid hand is inherently versatile in its use, and parsimony would suggest that the hominid last common ancestor was capable of manipulating and using simple tools. Therefore, key questions in palaeoanthropology ask when, why, and how tool use moved from facultative, as it is in other hominids, to obligate, as it is in H. sapiens. Inferring this transition within the fossil record is challenging as habitual behaviours are not always reflected in the external morphology of the skeleton. As the internal microstructure of bone is known to adapt to load dynamically, bone functional adaptation analyses provide an avenue to investigate how a joint has actually been loaded over an individual’s lifetime. The central question asked by this thesis was: ‘How and why does the internal structure of wrist bones differ among extant and extinct hominids?’. To achieve this aim, I investigated 1) whether functionally meaningful differences exist in the microarchitecture of extant hominid carpals; 2) how to detect signals of functional adaptation within the complex biomechanical environment of the wrist; 3) what can be inferred about hand use from the proximal capitate bone of fossil hominins? This thesis undertook three research projects, which all use ‘whole-bone’ methodologies for investigating functional signals of hand use. Using micro-computed tomography, I quantified and compared trabecular and cortical bone microarchitecture in 264 individual carpal bones across four extant hominids (Pongo, Gorilla, Pan, and H. sapiens) and four extinct hominins (Australopithecus sediba, Homo naledi, Homo floresiensis and Neanderthals). In the first project, I used inter- and intraspecific analyses to compare the trabecular and cortical microstructure of the proximal and distal capitate in extant hominids. Unique combinations of microarchitecture across the two segments of the bone differentiated the extant taxa. Notably, non-human hominids exhibited a distinctive pattern of extremely thick cortical bone in the distal capitate. This result suggested that highly localised functional adaptation responses were occurring across the capitate, and studying biomechanically distinct subregions of the carpus may be required to detect signals of functional adaptation. I then conducted intraspecific analyses on the scaphoid, lunate and triquetrum's trabecular and cortical bone microstructure across extant hominids. Results identified that microarchitectural differences across the three bones could be linked to the known or assumed biomechanics of the proximal row. Relative differences in the three bones differentiated locomotor mode between the genera: Gorilla and Pan expressed the same relative patterns of architecture, with Pongo and H. sapiens showing unique patterns. This project demonstrated that establishing relative patterns across a biomechanically distinct subregion of the wrist can differentiate hand use among extant hominids. Using a novel canonical holistic morphometric analysis, my final research project indicated that extant hominids have statistically distinct distributions of relative bone volume in the proximal capitate. Neanderthals and fossil H. sapiens exhibited the same pattern of relative bone distribution in the proximal capitate as modern H. sapiens suggesting a functional commitment to tool use leaves a distinct distribution of bone in the proximal capitate. Despite being the geologically oldest fossil, A. sediba was the only other species to exhibit a human-like distribution of bone, with evidence of a highly strained capitolunate and capitoscaphoid joint. Although H. naledi has human-like carpal morphology, it showed no evidence for human-like force transfer and loading at the midcarpal joint suggesting its hand use was not similar to a typical modern H. sapiens. The distribution of bone in H. floresiensis suggested that Oldowan-type tools were made and used with high ulnar-side loading of the hand and relatively lower loading of the thumb. This thesis demonstrated that a hand used primarily for manipulation has distinctive and statistically differentiated microarchitecture in the carpal bones. Unique microarchitectural features within the hominin species support a model of adaptive radiations of hand and tool behaviours among hominins. The similarity in microarchitecture at the midcarpal joint of H. sapiens and Neanderthals suggests it may be a strong signal of human-like commitment to tool use but is unlikely to capture variation in tool behaviour. Further analyses are needed to better understand how manipulation and arboreality are reflected in bone architecture. In particular, this thesis discussed how both climbing and transverse grips might be biomechanically compatible behaviours, as both emphasise high loading at the ulnar side of the hand and wrist and deemphasise the use of the thumb. Thus the use of transverse-type grips may have provided fossil hominins with an opportunity to improve the functional efficiency of tool behaviours without highly compromising climbing ability. Future analyses are likely to be most informative when numerous bones across biomechanically meaningful subregions of the wrist are analysed together. Analyses at the ulnar side of the wrist may be informative for identifying signals of climbing and grip preference differences in H. sapiens and Neanderthals

Kinematic Analysis of Multi-Fingered, Anthropomorphic Robotic Hands

The ability of stable grasping and fine manipulation with the multi-fingered robot hand with required precision and dexterity is playing an increasingly important role in the applications like service robots, rehabilitation, humanoid robots, entertainment robots, industries etc.. A number of multi-fingered robotic hands have been developed by various researchers in the past. The distinct advantages of a multi-fingered robot hand having structural similarity with human hand motivate the need for an anthropomorphic robot hand. Such a hand provides a promising base for supplanting human hand in execution of tedious, complicated and dangerous tasks, especially in situations such as manufacturing, space, undersea etc. These can also be used in orthopaedic rehabilitation of humans for improving the quality of the life of people having orthopedically and neurological disabilities. The developments so far are mostly driven by the application requirements. There are a number of bottlenecks with industrial grippers as regards to the stability of grasping objects of irregular geometries or complex manipulation operations. A multi-fingered robot hand can be made to mimic the movements of a human hand. The present piece of research work attempts to conceptualize and design a multi-fingered, anthropomorphic robot hand by structurally imitating the human hand. In the beginning, a brief idea about the history, types of robotic hands and application of multi-fingered hands in various fields are presented. A review of literature based on different aspects of the multi-fingered hand like structure, control, optimization, gasping etc. is made. Some of the important and more relevant literatures are elaborately discussed and a brief analysis is made on the outcomes and shortfalls with respect to multi-fingered hands. Based on the analysis of the review of literature, the research work aims at developing an improved anthropomorphic robot hand model in which apart from the four fingers and a thumb, the palm arch effect of human hand is also considered to increase its dexterity. A robotic hand with five anthropomorphic fingers including the thumb and palm arch effect having 25 degrees-of-freedom in all is investigated in the present work. Each individual finger is considered as an open loop kinematic chain and each finger segment is considered as a link of the manipulator. The wrist of the hand is considered as a fixed point. The kinematic analyses of the model for both forward kinematics and inverse kinematic are carried out. The trajectories of the tip positions of the thumb and the fingers with respect to local coordinate system are determined and plotted. This gives the extreme position of the fingertips which is obtained from the forward kinematic solution with the help of MATLAB. Similarly, varying all the joint iv angles of the thumb and fingers in their respective ranges, the reachable workspace of the hand model is obtained. Adaptive Neuro-Fuzzy Inference System (ANFIS) is used for solving the inverse kinematic problem of the fingers. Since the multi-fingered hand grasps the object mainly through its fingertips and the manipulation of the object is facilitated by the fingers due to their dexterity, the grasp is considered to be force-closure grasp. The grasping theory and different types of contacts between the fingertip and object are presented and the conditions for stable and equilibrium grasp are elaborately discussed. The proposed hand model is simulated to grasp five different shaped objects with equal base dimension and height. The forces applied on the fingertip during grasping are calculated. The hand model is also analysed using ANSYS to evaluate the stresses being developed at various points in the thumb and fingers. This analysis was made for the hand considering two different hand materials i.e. aluminium alloy and structural steel. The solution obtained from the forward kinematic analysis of the hand determines the maximum size for differently shaped objects while the solution to the inverse kinematic problem indicates the configurations of the thumb and the fingers inside the workspace of the hand. The solutions are predicted in which all joint angles are within their respective ranges. The results of the stress analysis of the hand model show that the structure of the fingers and the hand as a whole is capable of handling the selected objects. The robot hand under investigation can be realized and can be a very useful tool for many critical areas such as fine manipulation of objects, combating orthopaedic or neurological impediments, service robotics, entertainment robotics etc. The dissertation concludes with a summary of the contribution and the scope of further work

Faces and hands : modeling and animating anatomical and photorealistic models with regard to the communicative competence of virtual humans

In order to be believable, virtual human characters must be able to communicate in a human-like fashion realistically. This dissertation contributes to improving and automating several aspects of virtual conversations. We have proposed techniques to add non-verbal speech-related facial expressions to audiovisual speech, such as head nods for of emphasis. During conversation, humans experience shades of emotions much more frequently than the strong Ekmanian basic emotions. This prompted us to develop a method that interpolates between facial expressions of emotions to create new ones based on an emotion model. In the area of facial modeling, we have presented a system to generate plausible 3D face models from vague mental images. It makes use of a morphable model of faces and exploits correlations among facial features. The hands also play a major role in human communication. Since the basis for every realistic animation of gestures must be a convincing model of the hand, we devised a physics-based anatomical hand model, where a hybrid muscle model drives the animations. The model was used to visualize complex hand movement captured using multi-exposure photography.Um überzeugend zu wirken, müssen virtuelle Figuren auf dieselbe Art wie lebende Menschen kommunizieren können. Diese Dissertation hat das Ziel, verschiedene Aspekte virtueller Unterhaltungen zu verbessern und zu automatisieren. Wir führten eine Technik ein, die es erlaubt, audiovisuelle Sprache durch nichtverbale sprachbezogene Gesichtsausdrücke zu bereichern, wie z.B. Kopfnicken zur Betonung. Während einer Unterhaltung empfinden Menschen weitaus öfter Emotionsnuancen als die ausgeprägten Ekmanschen Basisemotionen. Dies bewog uns, eine Methode zu entwickeln, die Gesichtsausdrücke für neue Emotionen erzeugt, indem sie, ausgehend von einem Emotionsmodell, zwischen bereits bekannten Gesichtsausdrücken interpoliert. Auf dem Gebiet der Gesichtsmodellierung stellten wir ein System vor, um plausible 3D-Gesichtsmodelle aus vagen geistigen Bildern zu erzeugen. Dieses System basiert auf einem Morphable Model von Gesichtern und nutzt Korrelationen zwischen Gesichtszügen aus. Auch die Hände spielen ein große Rolle in der menschlichen Kommunikation. Da der Ausgangspunkt für jede realistische Animation von Gestik ein überzeugendes Handmodell sein muß, entwikkelten wir ein physikbasiertes anatomisches Handmodell, bei dem ein hybrides Muskelmodell die Animationen antreibt. Das Modell wurde verwendet, um komplexe Handbewegungen zu visualisieren, die aus mehrfach belichteten Photographien extrahiert worden waren