Learning Disentangled Representations with Reference-Based Variational Autoencoders

Learning disentangled representations from visual data, where different high-level generative factors are independently encoded, is of importance for many computer vision tasks. Solving this problem, however, typically requires to explicitly label all the factors of interest in training images. To alleviate the annotation cost, we introduce a learning setting which we refer to as "reference-based disentangling". Given a pool of unlabeled images, the goal is to learn a representation where a set of target factors are disentangled from others. The only supervision comes from an auxiliary "reference set" containing images where the factors of interest are constant. In order to address this problem, we propose reference-based variational autoencoders, a novel deep generative model designed to exploit the weak-supervision provided by the reference set. By addressing tasks such as feature learning, conditional image generation or attribute transfer, we validate the ability of the proposed model to learn disentangled representations from this minimal form of supervision

CT based three-dimensional measurement of adult orbit in Hospital Universiti Sains Malaysia

Purpose: To correlate the orbital parameters with age, race and gender of the HUSM population for forensic radiology future reference. Materials and methods: In an institutional review board-approved study, the authors obtained 126 samples referred to HUSM Radiology Department for CT scan of the head. Measurement of the orbital parameters is done on reconstructed 3D images using electronic calipers on workstation screen with the supervision of the supervisor, a senior radiologist with more than 30 years of experience in the Head and Neck imaging. Correlation and regression was used to assess the association between the orbital parameters with patient’s age, race and gender. Results: We were able to obtain 126 patients in which 113 were Malays, 13 were non-Malays, 68 were male and 58 were female. There was a significant correlation between the left and right orbital height (regression line 0.877), orbital width (regression line 0.759) and orbital perimeter (regression line 0.850). There was no significant statistical difference with race correlation and the orbital anthropometry. In terms of correlation with gender, independent t-test showed there was significant statistical difference in the left orbital height (mean difference 0.05), right orbital height (mean difference 0.06) and right orbital perimeter (mean difference 0.16). One way ANOVA showed that in terms of age, there was a significant statistical difference between the left orbital width (mean difference 0.16), right orbital width (mean difference 0.15) and left orbital perimeter (mean difference 0.43). Conclusions: This study provided useful baseline anthropometric data that will be of clinical and surgical interest in ophthalmology, oral and maxillofacial surgery in Kelantan. We recommended that anthropologists, clinicians and forensic experts to obtain this data and use them in any way deemed necessary for the quest of research and knowledge

Facial soft tissue segmentation

The importance of the face for socio-ecological interaction is the cause for a high demand on any surgical intervention on the facial musculo-skeletal system. Bones and soft-tissues are of major importance for any facial surgical treatment to guarantee an optimal, functional and aesthetical result. For this reason, surgeons want to pre-operatively plan, simulate and predict the outcome of the surgery allowing for shorter operation times and improved quality. Accurate simulation requires exact segmentation knowledge of the facial tissues. Thus semi-automatic segmentation techniques are required. This thesis proposes semi-automatic methods for segmentation of the facial soft-tissues, such as muscles, skin and fat, from CT and MRI datasets, using a Markov Random Fields (MRF) framework. Due to image noise, artifacts, weak edges and multiple objects of similar appearance in close proximity, it is difficult to segment the object of interest by using image information alone. Segmentations would leak at weak edges into neighboring structures that have a similar intensity profile. To overcome this problem, additional shape knowledge is incorporated in the energy function which can then be minimized using Graph-Cuts (GC). Incremental approaches by incorporating additional prior shape knowledge are presented. The proposed approaches are not object specific and can be applied to segment any class of objects be that anatomical or non-anatomical from medical or non-medical image datasets, whenever a statistical model is present. In the first approach a 3D mean shape template is used as shape prior, which is integrated into the MRF based energy function. Here, the shape knowledge is encoded into the data and the smoothness terms of the energy function that constrains the segmented parts to a reasonable shape. In the second approach, to improve handling of shape variations naturally found in the population, the fixed shape template is replaced by a more robust 3D statistical shape model based on Probabilistic Principal Component Analysis (PPCA). The advantages of using the Probabilistic PCA are that it allows reconstructing the optimal shape and computing the remaining variance of the statistical model from partial information. By using an iterative method, the statistical shape model is then refined using image based cues to get a better fitting of the statistical model to the patient's muscle anatomy. These image cues are based on the segmented muscle, edge information and intensity likelihood of the muscle. Here, a linear shape update mechanism is used to fit the statistical model to the image based cues. In the third approach, the shape refinement step is further improved by using a non-linear shape update mechanism where vertices of the 3D mesh of the statistical model incur the non-linear penalty depending on the remaining variability of the vertex. The non-linear shape update mechanism provides a more accurate shape update and helps in a finer shape fitting of the statistical model to the image based cues in areas where the shape variability is high. Finally, a unified approach is presented to segment the relevant facial muscles and the remaining facial soft-tissues (skin and fat). One soft-tissue layer is removed at a time such as the head and non-head regions followed by the skin. In the next step, bones are removed from the dataset, followed by the separation of the brain and non-brain regions as well as the removal of air cavities. Afterwards, facial fat is segmented using the standard Graph-Cuts approach. After separating the important anatomical structures, finally, a 3D fixed shape template mesh of the facial muscles is used to segment the relevant facial muscles. The proposed methods are tested on the challenging example of segmenting the masseter muscle. The datasets were noisy with almost all possessing mild to severe imaging artifacts such as high-density artifacts caused by e.g. dental fillings and dental implants. Qualitative and quantitative experimental results show that by incorporating prior shape knowledge leaking can be effectively constrained to obtain better segmentation results