Unsupervised Discovery of Parts, Structure, and Dynamics

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose a novel formulation that simultaneously learns a hierarchical, disentangled object representation and a dynamics model for object parts from unlabeled videos. Our Parts, Structure, and Dynamics (PSD) model learns to, first, recognize the object parts via a layered image representation; second, predict hierarchy via a structural descriptor that composes low-level concepts into a hierarchical structure; and third, model the system dynamics by predicting the future. Experiments on multiple real and synthetic datasets demonstrate that our PSD model works well on all three tasks: segmenting object parts, building their hierarchical structure, and capturing their motion distributions.Comment: ICLR 2019. The first two authors contributed equally to this wor

Development and Validation of a Three-Dimensional Optical Imaging System for Chest Wall Deformity Measurement

Congenital chest wall deformities (CWD) are malformations of the thoracic cage that become more pronounced during early adolescence. Pectus excavatum (PE) is the most common CWD, characterized by an inward depression of the sternum and adjacent costal cartilage. A cross-sectional computed tomography (CT) image is mainly used to calculate the chest thoracic indices. Physicians use the indices to quantify PE deformity, prescribe surgical or non-surgical therapies, and evaluate treatment outcomes. However, the use of CT is increasingly causing physicians to be concerned about the radiation doses administered to young patients. Furthermore, radiographic indices are an unsafe and expensive method of evaluating non-surgical treatments involving gradual chest wall changes. Flexible tape or a dowel-shaped ruler can be used to measure changes on the anterior side of the thorax; however, these methods are subjective, prone to human error, and cannot accurately measure small changes. This study aims to fill this gap by exploring three-dimensional optical imaging techniques to capture patients’ chest surfaces. The dissertation describes the development and validation of a cost-effective and safe method for objectively evaluating treatment progress in children with chest deformities. First, a study was conducted to evaluate the performance of low-cost 3D scanning technologies in measuring the severity of CWD. Second, a multitemporal surface mesh registration pipeline was developed for aligning 3D torso scans taken at different clinical appointments. Surface deviations were assessed between closely aligned scans. Optical indices were calculated without exposing patients to ionizing radiation, and changes in chest shape were visualized on a color-coded heat map. Additionally, a statistical model of chest shape built from healthy subjects was proposed to assess progress toward normal chest and aesthetic outcomes. The system was validated with 3D and CT datasets from a multi-institutional cohort. The findings indicate that optical scans can detect differences on a millimeter scale, and optical indices can be applied to approximate radiographic indices. In addition to improving patient awareness, visual representations of changes during nonsurgical treatment can enhance patient compliance

Biomechanical Morphing for Personalized Fitting of Scoliotic Torso Skeleton Models

The use of patient-specific biomechanical models offers many opportunities in the treatment of adolescent idiopathic scoliosis, such as the design of personalized braces. The first step in the development of these patient-specific models is to fit the geometry of the torso skeleton to the patient’s anatomy. However, existing methods rely on high-quality imaging data. The exposure to radiation of these methods limits their applicability for regular monitoring of patients. We present a method to fit personalized models of the torso skeleton that takes as input biplanar low-dose radiographs. The method morphs a template to fit annotated points on visible portions of the spine, and it relies on a default biomechanical model of the torso for regularization and robust fitting of hardly visible parts of the torso skeleton, such as the rib cage. The proposed method provides an accurate and robust solution to obtain personalized models of the torso skeleton, which can be adopted as part of regular management of scoliosis patients. We have evaluated the method on ten young patients who participated in our study. We have analyzed and compared clinical metrics on the spine and the full torso skeleton, and we have found that the accuracy of the method is at least comparable to other methods that require more demanding imaging methods, while it offers superior robustness to artifacts such as interpenetration of ribs. Normal-dose X-rays were available for one of the patients, and for the other nine we acquired low-dose X-rays, allowing us to validate that the accuracy of the method persisted under less invasive imaging modalities

3D digital breast cancer models with multimodal fusion algorithms

Breast cancer image fusion consists of registering and visualizing different sets of a patient synchronized torso and radiological images into a 3D model. Breast spatial interpretation and visualization by the treating physician can be augmented with a patient-specific digital breast model that integrates radiological images. But the absence of a ground truth for a good correlation between surface and radiological information has impaired the development of potential clinical applications. A new image acquisition protocol was designed to acquire breast Magnetic Resonance Imaging (MRI) and 3D surface scan data with surface markers on the patient's breasts and torso. A patient-specific digital breast model integrating the real breast torso and the tumor location was created and validated with a MRI/3D surface scan fusion algorithm in 16 breast cancer patients. This protocol was used to quantify breast shape differences between different modalities, and to measure the target registration error of several variants of the MRI/3D scan fusion algorithm. The fusion of single breasts without the biomechanical model of pose transformation had acceptable registration errors and accurate tumor locations. The performance of the fusion algorithm was not affected by breast volume. Further research and virtual clinical interfaces could lead to fast integration of this fusion technology into clinical practice.publishersversionpublishe

Point Cloud Segmentation based on Hypergraph Spectral Clustering

Hypergraph spectral analysis has emerged as an effective tool processing complex data structures in data analysis. The surface of a three-dimensional (3D) point cloud and the multilateral relationship among their points can be naturally captured by the high-dimensional hyperedges. This work investigates the power of hypergraph spectral analysis in unsupervised segmentation of 3D point clouds. We estimate and order the hypergraph spectrum from observed point cloud coordinates. By trimming the redundancy from the estimated hypergraph spectral space based on spectral component strengths, we develop a clustering-based segmentation method. We apply the proposed method to various point clouds, and analyze their respective spectral properties. Our experimental results demonstrate the effectiveness and efficiency of the proposed segmentation method

Human Pose Detection for Robotic-Assisted and Rehabilitation Environments

Assistance and rehabilitation robotic platforms must have precise sensory systems for human–robot interaction. Therefore, human pose estimation is a current topic of research, especially for the safety of human–robot collaboration and the evaluation of human biomarkers. Within this field of research, the evaluation of the low-cost marker-less human pose estimators of OpenPose and Detectron 2 has received much attention for their diversity of applications, such as surveillance, sports, videogames, and assessment in human motor rehabilitation. This work aimed to evaluate and compare the angles in the elbow and shoulder joints estimated by OpenPose and Detectron 2 during four typical upper-limb rehabilitation exercises: elbow side flexion, elbow flexion, shoulder extension, and shoulder abduction. A setup of two Kinect 2 RGBD cameras was used to obtain the ground truth of the joint and skeleton estimations during the different exercises. Finally, we provided a numerical comparison (RMSE and MAE) among the angle measurements obtained with OpenPose, Detectron 2, and the ground truth. The results showed how OpenPose outperforms Detectron 2 in these types of applications.Óscar G. Hernández holds a grant from the Spanish Fundación Carolina, the University of Alicante, and the National Autonomous University of Honduras