Preterm infants' limb-pose estimation from depth images using convolutional neural networks

Preterm infants' limb-pose estimation is a crucial but challenging task, which may improve patients' care and facilitate clinicians in infant's movements monitoring. Work in the literature either provides approaches to whole-body segmentation and tracking, which, however, has poor clinical value, or retrieve a posteriori limb pose from limb segmentation, increasing computational costs and introducing inaccuracy sources. In this paper, we address the problem of limb-pose estimation under a different point of view. We proposed a 2D fully-convolutional neural network for roughly detecting limb joints and joint connections, followed by a regression convolutional neural network for accurate joint and joint-connection position estimation. Joints from the same limb are then connected with a maximum bipartite matching approach. Our analysis does not require any prior modeling of infants' body structure, neither any manual interventions. For developing and testing the proposed approach, we built a dataset of four videos (video length = 90 s) recorded with a depth sensor in a neonatal intensive care unit (NICU) during the actual clinical practice, achieving median root mean square distance [pixels] of 10.790 (right arm), 10.542 (left arm), 8.294 (right leg), 11.270 (left leg) with respect to the ground-truth limb pose. The idea of estimating limb pose directly from depth images may represent a future paradigm for addressing the problem of preterm-infants' movement monitoring and offer all possible support to clinicians in NICUs

Markerless Human Motion Analysis

Measuring and understanding human motion is crucial in several domains, ranging from neuroscience, to rehabilitation and sports biomechanics. Quantitative information about human motion is fundamental to study how our Central Nervous System controls and organizes movements to functionally evaluate motor performance and deficits. In the last decades, the research in this field has made considerable progress. State-of-the-art technologies that provide useful and accurate quantitative measures rely on marker-based systems. Unfortunately, markers are intrusive and their number and location must be determined a priori. Also, marker-based systems require expensive laboratory settings with several infrared cameras. This could modify the naturalness of a subject\u2019s movements and induce discomfort. Last, but not less important, they are computationally expensive in time and space. Recent advances on markerless pose estimation based on computer vision and deep neural networks are opening the possibility of adopting efficient video-based methods for extracting movement information from RGB video data. In this contest, this thesis presents original contributions to the following objectives: (i) the implementation of a video-based markerless pipeline to quantitatively characterize human motion; (ii) the assessment of its accuracy if compared with a gold standard marker-based system; (iii) the application of the pipeline to different domains in order to verify its versatility, with a special focus on the characterization of the motion of preterm infants and on gait analysis. With the proposed approach we highlight that, starting only from RGB videos and leveraging computer vision and machine learning techniques, it is possible to extract reliable information characterizing human motion comparable to that obtained with gold standard marker-based systems

Shape analysis of the human brain.

Autism is a complex developmental disability that has dramatically increased in prevalence, having a decisive impact on the health and behavior of children. Methods used to detect and recommend therapies have been much debated in the medical community because of the subjective nature of diagnosing autism. In order to provide an alternative method for understanding autism, the current work has developed a 3-dimensional state-of-the-art shape based analysis of the human brain to aid in creating more accurate diagnostic assessments and guided risk analyses for individuals with neurological conditions, such as autism. Methods: The aim of this work was to assess whether the shape of the human brain can be used as a reliable source of information for determining whether an individual will be diagnosed with autism. The study was conducted using multi-center databases of magnetic resonance images of the human brain. The subjects in the databases were analyzed using a series of algorithms consisting of bias correction, skull stripping, multi-label brain segmentation, 3-dimensional mesh construction, spherical harmonic decomposition, registration, and classification. The software algorithms were developed as an original contribution of this dissertation in collaboration with the BioImaging Laboratory at the University of Louisville Speed School of Engineering. The classification of each subject was used to construct diagnoses and therapeutic risk assessments for each patient. Results: A reliable metric for making neurological diagnoses and constructing therapeutic risk assessment for individuals has been identified. The metric was explored in populations of individuals having autism spectrum disorders, dyslexia, Alzheimers disease, and lung cancer. Conclusion: Currently, the clinical applicability and benefits of the proposed software approach are being discussed by the broader community of doctors, therapists, and parents for use in improving current methods by which autism spectrum disorders are diagnosed and understood

Menetelmiä lasten näkötiedon käsittelyn arvioimiseksi katseenseurannan avulla

Cortical visual processing and mechanism under eye movements and visiospatial attention undergo prominent developmental changes during the first 12 months of infancy. At that time, these key functions of vision are tightly connected to the early brain development in general. Thus, they are favourable targets for new research methods that can be used in treatment, prediction, or detection of various adverse visual of neurocognitive conditions. This thesis presents two eye tracker assisted test paradigms that may be used to evaluate and quantify different functions of infants’ visual processing. The first study concentrates on the analysis of the gaze patterns in classic face-distractor competition paradigm known to tap mechanisms under infant’s attention disengagement and visuospatial orienting. A novel stimuli over a given period of time. In further evaluation, the metric is shown to be sensitive to developmental changes in infants’ face processing between 5 and 7 months of age. The second study focuses on the visual evoked potentials (VEPs) elicited by orientation reversal, global form, and clobal motion stimulation known to measure distinct aspects of visual processing at the cortical level. To improve the reality of such methods, an eye tracker is integrated to the recording setup, which can be used to control stimulus presentation to capture the attention of the infant, and in the analysis to exclude the electroencephalography (EEG) segments with disorientated gaze. With this setup, VEPs can be detected from the vast majority of the tested 3-month-old infants (N=39) using circular variant of Hotelling’s T2 test statistic and two developed power spectrum based metrics. After further development already in progress, the presented methods are ready to be used clinically in assessments of neurocognitive development, preferably alongside other similar biomarker tests of infancy.Näkötiedon käsittely aivokuorella sekä silmänliikkeiden ja visuospatiaalisen tarkkaavaisuuden mekanismit kehittyvät valtavasti lapsen ensimmäisen 12 elinkuukauden kuluessa. Nämä näön avaintoiminnot ovat tiukasti sidoksissa aivojen yleiseen varhaiskehitykseen, jonka vuoksi ne ovat suotuisia kohteita uusille tutkimusmenetelmille käytettäväksi visuaalisen tai neurologisten ongelmien hoidossa, ennustuksessa ja löytämisessä. Tämä työ esittelee kaksi katseenseurantaa hyödyntävää koeasetelmaa, joita voidaan käyttää lasten kortikaalisen näkötiedon käsittelyn arvioinnissa ja kvantifionnissa. Ensimmäisessä tutkimuksessa kehitettiin mitattujen katsekuvioiden analyysiä klassisessa kasvokuva-distraktori-koeasetelmassa, jonka tiedetään koskettavan lasten tarkkavaisuuden vapauttamiseen ja katseen siirtoon liittyviä mekanismeja. Työssä kehitetyllä laskennallisella mittarilla pystytään määrittämään tarkkavaisuuden jakautuminen ruudun keskellä ja raunalla esitettyjen ärsykkeiden välillä haluttuna aikana. Jatkotarkastelu osoittaa mittarin olevan herkkä kasvokuvien käsittelyn kehityksen muutoksille 5 ja 7 kuukauden ikäisten lasten välillä. Toinen osatyö keskittyy näkötiedon kortikaalista käsittelyä heijastavien, suunnan kääntämisen, globaalin muodon tai liikkeen tuottamien näköherätepotentiaalien mittaamiseen ja analyysiin. Parantaakseen menetelmien luotettavuutta laitteistoon liitetään silmänliikekamera, joka mahdollistaa sekä ärsyketoiston ohjaamisen lapsen tarkkaavaisuuden mukaisesti että kerätyn aivosähkökäyrän karsimisen niiltä osin, jolloin lapsen katse oli harhautunut esityksestä. Käyttäen muunnelmaa Hotellingin T2 statistiikasta ja kahta työssä kehitettyä, tehospektriin pohjautuvaa analyysimenetelmää herätevasteet pystytään löytämään valtaosalta 3 kuukauden ikäisistä lapsista (N=39). Meneillään olevan jatkokehityksen jälkeen esitetyt menetelmät ovat valmiita kliiniseen käyttöön neurokognitiivisen kehityksen arvioinnissa muiden vastaavien biomarkkeritutkimuksen rinnalla

The effect of unsupportive and supportive footwear on children’s multi-segment foot dynamics during gait

Footwear is necessary for children’s foot comfort and protection. Despite the popularity of flip-flop (thongs) footwear among children, strong clinical opinion endures of the potential deleterious effect this footwear may have on developing feet. On the contrary, thongs may be beneficial for children’s developing feet due to the footwear’s flexible and unrestrictive nature, as children who mature within habitually barefoot communities are observed to develop stronger and healthier feet. This thesis considers the developing nature of human ambulation and the physiological basis for children’s foot maturation. It then explores the effect of thong footwear on childrens barefoot dynamics with comparisons to traditionally advocated supportive footwear. Foot compensations were observed when thongs were worn while walking and to a lesser extent while jogging. Greater ankle dorsiflexion and reduced hallux dorsiflexion suggests a mechanism to retain the thong. Greater midfoot plantarflexion indicates a gripping action to sustain the thong. Barefoot motions were unaffected by thongs during the sidestep. The midfoot splinting effect of supportive shoes was reinforced while walking, jogging and sidestepping. Thongs had a minimal effect on barefoot dynamics, while supportive shoes limited midfoot power generation with a corresponding increase in ankle power generation. Overall findings suggest that foot motion when wearing thongs may be more replicable of barefoot motion than originally believed. In terms of foot arch development, thongs may be more beneficial than supportive shoes, due to the minimal alterations to barefoot motions when they are worn. The reported midfoot plantarflexion required to grip the thong may be beneficial to children’s foot arch strengthening and overall foot development. While supportive shoes have the necessary protective features, they have been shown to inhibit midfoot and hallux motions with a compensatory increase in ankle motions

XR, music and neurodiversity: design and application of new mixed reality technologies that facilitate musical intervention for children with autism spectrum conditions

This thesis, accompanied by the practice outputs,investigates sensory integration, social interaction and creativity through a newly developed VR-musical interface designed exclusively for children with a high-functioning autism spectrum condition (ASC).The results aim to contribute to the limited expanse of literature and research surrounding Virtual Reality (VR) musical interventions and Immersive Virtual Environments (IVEs) designed to support individuals with neurodevelopmental conditions. The author has developed bespoke hardware, software and a new methodology to conduct field investigations. These outputs include a Virtual Immersive Musical Reality Intervention (ViMRI) protocol, a Supplemental Personalised, immersive Musical Experience(SPiME) programme, the Assisted Real-time Three-dimensional Immersive Musical Intervention System’ (ARTIMIS) and a bespoke (and fully configurable) ‘Creative immersive interactive Musical Software’ application (CiiMS). The outputs are each implemented within a series of institutional investigations of 18 autistic child participants. Four groups are evaluated using newly developed virtual assessment and scoring mechanisms devised exclusively from long-established rating scales. Key quantitative indicators from the datasets demonstrate consistent findings and significant improvements for individual preferences (likes), fear reduction efficacy, and social interaction. Six individual case studies present positive qualitative results demonstrating improved decision-making and sensorimotor processing. The preliminary research trials further indicate that using this virtual-reality music technology system and newly developed protocols produces notable improvements for participants with an ASC. More significantly, there is evidence that the supplemental technology facilitates a reduction in psychological anxiety and improvements in dexterity. The virtual music composition and improvisation system presented here require further extensive testing in different spheres for proof of concept

Entropy in Image Analysis II

Image analysis is a fundamental task for any application where extracting information from images is required. The analysis requires highly sophisticated numerical and analytical methods, particularly for those applications in medicine, security, and other fields where the results of the processing consist of data of vital importance. This fact is evident from all the articles composing the Special Issue "Entropy in Image Analysis II", in which the authors used widely tested methods to verify their results. In the process of reading the present volume, the reader will appreciate the richness of their methods and applications, in particular for medical imaging and image security, and a remarkable cross-fertilization among the proposed research areas