単眼画像入力型ビューベース3Dモデル検索に関する研究

Tohoku University北村喜文課

Active Boundary Component Models for Robotic Dressing Assistance

Twardon L, Ritter H. Active Boundary Component Models for Robotic Dressing Assistance. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ: IEEE; 2016: 2811-2818

3D shape matching and registration : a probabilistic perspective

Dense correspondence is a key area in computer vision and medical image analysis. It has applications in registration and shape analysis. In this thesis, we develop a technique to recover dense correspondences between the surfaces of neuroanatomical objects over heterogeneous populations of individuals. We recover dense correspondences based on 3D shape matching. In this thesis, the 3D shape matching problem is formulated under the framework of Markov Random Fields (MRFs). We represent the surfaces of neuroanatomical objects as genus zero voxel-based meshes. The surface meshes are projected into a Markov random field space. The projection carries both geometric and topological information in terms of Gaussian curvature and mesh neighbourhood from the original space to the random field space. Gaussian curvature is projected to the nodes of the MRF, and the mesh neighbourhood structure is projected to the edges. 3D shape matching between two surface meshes is then performed by solving an energy function minimisation problem formulated with MRFs. The outcome of the 3D shape matching is dense point-to-point correspondences. However, the minimisation of the energy function is NP hard. In this thesis, we use belief propagation to perform the probabilistic inference for 3D shape matching. A sparse update loopy belief propagation algorithm adapted to the 3D shape matching is proposed to obtain an approximate global solution for the 3D shape matching problem. The sparse update loopy belief propagation algorithm demonstrates significant efficiency gain compared to standard belief propagation. The computational complexity and convergence property analysis for the sparse update loopy belief propagation algorithm are also conducted in the thesis. We also investigate randomised algorithms to minimise the energy function. In order to enhance the shape matching rate and increase the inlier support set, we propose a novel clamping technique. The clamping technique is realized by combining the loopy belief propagation message updating rule with the feedback from 3D rigid body registration. By using this clamping technique, the correct shape matching rate is increased significantly. Finally, we investigate 3D shape registration techniques based on the 3D shape matching result. Based on the point-to-point dense correspondences obtained from the 3D shape matching, a three-point based transformation estimation technique is combined with the RANdom SAmple Consensus (RANSAC) algorithm to obtain the inlier support set. The global registration approach is purely dependent on point-wise correspondences between two meshed surfaces. It has the advantage that the need for orientation initialisation is eliminated and that all shapes of spherical topology. The comparison of our MRF based 3D registration approach with a state-of-the-art registration algorithm, the first order ellipsoid template, is conducted in the experiments. These show dense correspondence for pairs of hippocampi from two different data sets, each of around 20 60+ year old healthy individuals

Medial Axis Approximation and Regularization

Medial axis is a classical shape descriptor. Among many good properties, medial axis is thin, centered in the shape, and topology preserving. Therefore, it is constantly sought after by researchers and practitioners in their respective domains. However, two barriers remain that hinder wide adoption of medial axis. First, exact computation of medial axis is very difficult. Hence, in practice medial axis is approximated discretely. Though abundant approximation methods exist, they are either limited in scalability, insufficient in theoretical soundness, or susceptible to numerical issues. Second, medial axis is easily disturbed by small noises on its defining shape. A majority of current works define a significance measure to prune noises on medial axis. Among them, local measures are widely available due to their efficiency, but can be either too aggressive or conservative. While global measures outperform local ones in differentiating noises from features, they are rarely well-defined or efficient to compute. In this dissertation, we attempt to address these issues with sound, robust and efficient solutions. In Chapter 2, we propose a novel medial axis approximation called voxel core. We show voxel core is topologically and geometrically convergent to the true medial axis. We then describe a straightforward implementation as a result of our simple definition. In a variety of experiments, our method is shown to be efficient and robust in delivering topological promises on a wide range of shapes. In Chapter 3, we present Erosion Thickness (ET) to regularize instability. ET is the first global measure in 3D that is well-defined and efficient to compute. To demonstrate its usefulness, we utilize ET to generate a family of shape revealing and topology preserving skeletons. Finally, we point out future directions, and potential applications of our works in real world problems

Multi-Surface Simplex Spine Segmentation for Spine Surgery Simulation and Planning

This research proposes to develop a knowledge-based multi-surface simplex deformable model for segmentation of healthy as well as pathological lumbar spine data. It aims to provide a more accurate and robust segmentation scheme for identification of intervertebral disc pathologies to assist with spine surgery planning. A robust technique that combines multi-surface and shape statistics-aware variants of the deformable simplex model is presented. Statistical shape variation within the dataset has been captured by application of principal component analysis and incorporated during the segmentation process to refine results. In the case where shape statistics hinder detection of the pathological region, user-assistance is allowed to disable the prior shape influence during deformation. Results have been validated against user-assisted expert segmentation

Capturing Hand-Object Interaction and Reconstruction of Manipulated Objects

Hand motion capture with an RGB-D sensor gained recently a lot of research attention, however, even most recent approaches focus on the case of a single isolated hand. We focus instead on hands that interact with other hands or with a rigid or articulated object. Our framework successfully captures motion in such scenarios by combining a generative model with discriminatively trained salient points, collision detection and physics simulation to achieve a low tracking error with physically plausible poses. All components are unified in a single objective function that can be optimized with standard optimization techniques. We initially assume a-priori knowledge of the object’s shape and skeleton. In case of unknown object shape there are existing 3d reconstruction methods that capitalize on distinctive geometric or texture features. These methods though fail for textureless and highly symmetric objects like household articles, mechanical parts or toys. We show that extracting 3d hand motion for in-hand scanning e↵ectively facilitates the reconstruction of such objects and we fuse the rich additional information of hands into a 3d reconstruction pipeline. Finally, although shape reconstruction is enough for rigid objects, there is a lack of tools that build rigged models of articulated objects that deform realistically using RGB-D data. We propose a method that creates a fully rigged model consisting of a watertight mesh, embedded skeleton and skinning weights by employing a combination of deformable mesh tracking, motion segmentation based on spectral clustering and skeletonization based on mean curvature flow

Towards development of automatic path planning system in image-guided neurosurgery

With the advent of advanced computer technology, many computer-aided systems have evolved to assist in medical related work including treatment, diagnosis, and even surgery. In modern neurosurgery, Magnetic Resonance Image guided stereotactic surgery exactly complies with this trend. It is a minimally invasive operation being much safer than the traditional open-skull surgery, and offers higher precision and more effective operating procedures compared to conventional craniotomy. However, such operations still face significant challenges of planning the optimal neurosurgical path in order to reach the ideal position without damage to important internal structures. This research aims to address this major challenge. The work begins with an investigation of the problem of distortion induced by MR images. It then goes on to build a template of the Circle of Wills brain vessels, realized from a collection of Magnetic Resonance Angiography images, which is needed to maintain operating standards when, as in many cases, Magnetic Resonance Angiography images are not available for patients. Demographic data of brain tumours are also studied to obtain further understanding of diseased human brains through the development of an effect classifier. The developed system allows the internal brain structure to be ‘seen’ clearly before the surgery, giving surgeons a clear picture and thereby makes a significant contribution to the eventual development of a fully automatic path planning system

Efficient sketch-based 3D character modelling.

Sketch-based modelling (SBM) has undergone substantial research over the past two decades. In the early days, researchers aimed at developing techniques useful for modelling of architectural and mechanical models through sketching. With the advancement of technology used in designing visual effects for film, TV and games, the demand for highly realistic 3D character models has skyrocketed. To allow artists to create 3D character models quickly, researchers have proposed several techniques for efficient character modelling from sketched feature curves. Moreover several research groups have developed 3D shape databases to retrieve 3D models from sketched inputs. Unfortunately, the current state of the art in sketch-based organic modelling (3D character modelling) contains a lot of gaps and limitations. To bridge the gaps and improve the current sketch-based modelling techniques, this research aims to develop an approach allowing direct and interactive modelling of 3D characters from sketched feature curves, and also make use of 3D shape databases to guide the artist to create his / her desired models. The research involved finding a fusion between 3D shape retrieval, shape manipulation, and shape reconstruction / generation techniques backed by an extensive literature review, experimentation and results. The outcome of this research involved devising a novel and improved technique for sketch-based modelling, the creation of a software interface that allows the artist to quickly and easily create realistic 3D character models with comparatively less effort and learning. The proposed research work provides the tools to draw 3D shape primitives and manipulate them using simple gestures which leads to a better modelling experience than the existing state of the art SBM systems