Segmentation of skin lesions in 2D and 3D ultrasound images using a spatially coherent generalized Rayleigh mixture model

This paper addresses the problem of jointly estimating the statistical distribution and segmenting lesions in multiple-tissue high-frequency skin ultrasound images. The distribution of multiple-tissue images is modeled as a spatially coherent finite mixture of heavy-tailed Rayleigh distributions. Spatial coherence inherent to biological tissues is modeled by enforcing local dependence between the mixture components. An original Bayesian algorithm combined with a Markov chain Monte Carlo method is then proposed to jointly estimate the mixture parameters and a label-vector associating each voxel to a tissue. More precisely, a hybrid Metropolis-within-Gibbs sampler is used to draw samples that are asymptotically distributed according to the posterior distribution of the Bayesian model. The Bayesian estimators of the model parameters are then computed from the generated samples. Simulation results are conducted on synthetic data to illustrate the performance of the proposed estimation strategy. The method is then successfully applied to the segmentation of in vivo skin tumors in high-frequency 2-D and 3-D ultrasound images

Computer vision tools for the non-invasive assessment of autism-related behavioral markers

The early detection of developmental disorders is key to child outcome, allowing interventions to be initiated that promote development and improve prognosis. Research on autism spectrum disorder (ASD) suggests behavioral markers can be observed late in the first year of life. Many of these studies involved extensive frame-by-frame video observation and analysis of a child's natural behavior. Although non-intrusive, these methods are extremely time-intensive and require a high level of observer training; thus, they are impractical for clinical and large population research purposes. Diagnostic measures for ASD are available for infants but are only accurate when used by specialists experienced in early diagnosis. This work is a first milestone in a long-term multidisciplinary project that aims at helping clinicians and general practitioners accomplish this early detection/measurement task automatically. We focus on providing computer vision tools to measure and identify ASD behavioral markers based on components of the Autism Observation Scale for Infants (AOSI). In particular, we develop algorithms to measure three critical AOSI activities that assess visual attention. We augment these AOSI activities with an additional test that analyzes asymmetrical patterns in unsupported gait. The first set of algorithms involves assessing head motion by tracking facial features, while the gait analysis relies on joint foreground segmentation and 2D body pose estimation in video. We show results that provide insightful knowledge to augment the clinician's behavioral observations obtained from real in-clinic assessments

Single camera pose estimation using Bayesian filtering and Kinect motion priors

Traditional approaches to upper body pose estimation using monocular vision rely on complex body models and a large variety of geometric constraints. We argue that this is not ideal and somewhat inelegant as it results in large processing burdens, and instead attempt to incorporate these constraints through priors obtained directly from training data. A prior distribution covering the probability of a human pose occurring is used to incorporate likely human poses. This distribution is obtained offline, by fitting a Gaussian mixture model to a large dataset of recorded human body poses, tracked using a Kinect sensor. We combine this prior information with a random walk transition model to obtain an upper body model, suitable for use within a recursive Bayesian filtering framework. Our model can be viewed as a mixture of discrete Ornstein-Uhlenbeck processes, in that states behave as random walks, but drift towards a set of typically observed poses. This model is combined with measurements of the human head and hand positions, using recursive Bayesian estimation to incorporate temporal information. Measurements are obtained using face detection and a simple skin colour hand detector, trained using the detected face. The suggested model is designed with analytical tractability in mind and we show that the pose tracking can be Rao-Blackwellised using the mixture Kalman filter, allowing for computational efficiency while still incorporating bio-mechanical properties of the upper body. In addition, the use of the proposed upper body model allows reliable three-dimensional pose estimates to be obtained indirectly for a number of joints that are often difficult to detect using traditional object recognition strategies. Comparisons with Kinect sensor results and the state of the art in 2D pose estimation highlight the efficacy of the proposed approach.Comment: 25 pages, Technical report, related to Burke and Lasenby, AMDO 2014 conference paper. Code sample: https://github.com/mgb45/SignerBodyPose Video: https://www.youtube.com/watch?v=dJMTSo7-uF

Graph-based Object Understanding

Computer Vision algorithms become increasingly prevalent in our everyday lives. Especially recognition systems are often employed to automatize certain tasks (i.e. quality control). In State-of-the-Art approaches global shape char acteristics are leveraged, discarding nuanced shape varieties in the individual parts of the object. Thus, these systems fall short on both learning and utilizing the inherent underlying part structures of objects. By recognizing common substructures between known and queried objects, part-based systems may identify objects more robustly in lieu of occlusion or redundant parts. As we observe these traits, there are theories that such part-based approaches are indeed present in humans. Leveraging abstracted representations of decomposed objects may additionally offer better generalization on less training data. Enabling computer systems to reason about objects on the basis of their parts is the focus of this dissertation. Any part-based method first requires a segmentation approach to assign object regions to individual parts. Therefore, a 2D multi-view segmentation approach for 3D mesh segmentation is extended. The approach uses the normal and depth information of the objects to reliably extract part boundary contours. This method significantly reduces training time of the segmentation model compared to other segmentation approaches while still providing good segmentation results on the test data. To explore the benefits of part-based systems, a symbolic object classification dataset is created that inherently adheres to underlying rules made of spatial relations between part entities. This abstract data is also transformed into 3D point clouds. This enables us to benchmark conventional 3D point cloud classification models against the newly developed model that utilizes ground truth symbol segmentations for the classification task. With the new model, improved classification performance can be observed. This offers empirical evidence that part segmentation may boost classification accuracy if the data obey part-based rules. Additionally, prediction results of the model on segmented 3D data are compared against a modified variant of the model that directly uses the underlying symbols. The perception gap, representing issues with extracting the symbols from the segmented point clouds, is quantified. Furthermore, a framework for 3D object classification on real world objects is developed. The designed pipeline automatically segments an object into its parts, creates the according part graph and predicts the object class based on the similarity to graphs in the training dataset. The advantage of subgraph similarity is utilized in a second experiment, where out-of-distribution samples ofobjects are created, which contain redundant parts. Whereas traditional classification methods working on the global shape may misinterpret extracted feature vectors, the model creates robust predictions. Lastly, the task of object repairment is considered, in which a single part of the given object is compromised by a certain manipulation. As human-made objects follow an underlying part structure, a system to exploit this part structure in order to mend the object is developed. Given the global 3D point cloud of a compromised object, the object is automatically segmented, the shape features are extracted from the individual part clouds and are fed into a Graph Neural Network that predicts a manipulation action for each part. In conclusion, the opportunities of part-graph based methods for object understanding to improve 3D classification and regression tasks are explored. These approaches may enhance robotic computer vision pipelines in the future.2021-06-2

CNN Based Touch Interaction Detection for Infant Speech Development

In this paper, we investigate the detection of interaction in videos between two people, namely, a caregiver and an infant. We are interested in a particular type of human interaction known as touch, as touch is a key social and emotional signal used by caregivers when interacting with their children. We propose an automatic touch event recognition method to determine the potential time interval when the caregiver touches the infant. In addition to label the touch events, we also classify them into six touch types based on which body part of infant has been touched. CNN based human pose estimation and person segmentation are used to analyze the spatial relationship between the caregivers hands and the infants. We demonstrate promising results for touch detection and show great potential of reducing human effort in manually generating precise touch annotations

Structure analysis and lesion detection from retinal fundus images

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Ocular pathology is one of the main health problems worldwide. The number of people with retinopathy symptoms has increased considerably in recent years. Early adequate treatment has demonstrated to be effective to avoid the loss of the vision. The analysis of fundus images is a non intrusive option for periodical retinal screening. Different models designed for the analysis of retinal images are based on supervised methods, which require of hand labelled images and processing time as part of the training stage. On the other hand most of the methods have been designed under the basis of specific characteristics of the retinal images (e.g. field of view, resolution). This compromises its performance to a reduce group of retinal image with similar features. For these reasons an unsupervised model for the analysis of retinal image is required, a model that can work without human supervision or interaction. And that is able to perform on retinal images with different characteristics. In this research, we have worked on the development of this type of model. The system locates the eye structures (e.g. optic disc and blood vessels) as first step. Later, these structures are masked out from the retinal image in order to create a clear field to perform the lesion detection. We have selected the Graph Cut technique as a base to design the retinal structures segmentation methods. This selection allows incorporating prior knowledge to constraint the searching for the optimal segmentation. Different link weight assignments were formulated in order to attend the specific needs of the retinal structures (e.g. shape). This research project has put to work together the fields of image processing and ophthalmology to create a novel system that contribute significantly to the state of the art in medical image analysis. This new knowledge provides a new alternative to address the analysis of medical images and opens a new panorama for researchers exploring this research area.Mexican National Council of Science and Technolog

Digitalization of the IOM: A comprehensive cadaveric study for obtaining three-dimensional models and morphological properties of the forearm's interosseous membrane

State-of-the-art of preoperative planning for forearm orthopaedic surgeries is currently limited to simple bone procedures. The increasing interest of clinicians for more comprehensive analysis of complex pathologies often requires dynamic models, able to include the soft tissue influence into the preoperative process. Previous studies have shown that the interosseous membrane (IOM) influences forearm motion and stability, but due to the lack of morphological and biomechanical data, existing simulation models of the IOM are either too simple or clinically unreliable. This work aims to address this problematic by generating 3D morphological and tensile properties of the individual IOM structures. First, micro- and standard-CT acquisitions were performed on five fresh-frozen annotated cadaveric forearms for the generation of 3D models of the radius, ulna and each of the individual ligaments of the IOM. Afterwards, novel 3D methods were developed for the measurement of common morphological features, which were validated against established optical ex-vivo measurements. Finally, we investigated the individual tensile properties of each IOM ligament. The generated 3D morphological features can provide the basis for the future development of functional planning simulation of the forearm

Co-skeletons:Consistent curve skeletons for shape families

We present co-skeletons, a new method that computes consistent curve skeletons for 3D shapes from a given family. We compute co-skeletons in terms of sampling density and semantic relevance, while preserving the desired characteristics of traditional, per-shape curve skeletonization approaches. We take the curve skeletons extracted by traditional approaches for all shapes from a family as input, and compute semantic correlation information of individual skeleton branches to guide an edge-pruning process via skeleton-based descriptors, clustering, and a voting algorithm. Our approach achieves more concise and family-consistent skeletons when compared to traditional per-shape methods. We show the utility of our method by using co-skeletons for shape segmentation and shape blending on real-world data