Video Interpolation using Optical Flow and Laplacian Smoothness

Non-rigid video interpolation is a common computer vision task. In this paper we present an optical flow approach which adopts a Laplacian Cotangent Mesh constraint to enhance the local smoothness. Similar to Li et al., our approach adopts a mesh to the image with a resolution up to one vertex per pixel and uses angle constraints to ensure sensible local deformations between image pairs. The Laplacian Mesh constraints are expressed wholly inside the optical flow optimization, and can be applied in a straightforward manner to a wide range of image tracking and registration problems. We evaluate our approach by testing on several benchmark datasets, including the Middlebury and Garg et al. datasets. In addition, we show application of our method for constructing 3D Morphable Facial Models from dynamic 3D data

From 4D medical images (CT, MRI, and Ultrasound) to 4D structured mesh models of the left ventricular endocardium for patient-specific simulations

With cardiovascular disease (CVD) remaining the primary cause of death worldwide, early detection of CVDs becomes essential. The intracardiac flow is an important component of ventricular function, motion kinetics, wash-out of ventricular chambers, and ventricular energetics. Coupling between Computational Fluid Dynamics (CFD) simulations and medical images can play a fundamental role in terms of patient-specific diagnostic tools. From a technical perspective, CFD simulations with moving boundaries could easily lead to negative volumes errors and the sudden failure of the simulation. The generation of high-quality 4D meshes (3D in space + time) with 1-to-l vertex becomes essential to perform a CFD simulation with moving boundaries. In this context, we developed a semiautomatic morphing tool able to create 4D high-quality structured meshes starting from a segmented 4D dataset. To prove the versatility and efficiency, the method was tested on three different 4D datasets (Ultrasound, MRI, and CT) by evaluating the quality and accuracy of the resulting 4D meshes. Furthermore, an estimation of some physiological quantities is accomplished for the 4D CT reconstruction. Future research will aim at extending the region of interest, further automation of the meshing algorithm, and generating structured hexahedral mesh models both for the blood and myocardial volume

PialNN: A fast deep learning framework for cortical pial surface reconstruction

Traditional cortical surface reconstruction is time consuming and limited by the resolution of brain Magnetic Resonance Imaging (MRI). In this work, we introduce Pial Neural Network (PialNN), a 3D deep learning framework for pial surface reconstruction. PialNN is trained end-to-end to deform an initial white matter surface to a target pial surface by a sequence of learned deformation blocks. A local convolutional operation is incorporated in each block to capture the multi-scale MRI information of each vertex and its neighborhood. This is fast and memory-efficient, which allows reconstructing a pial surface mesh with 150k vertices in one second. The performance is evaluated on the Human Connectome Project (HCP) dataset including T1-weighted MRI scans of 300 subjects. The experimental results demonstrate that PialNN reduces the geometric error of the predicted pial surface by 30% compared to state-of-the-art deep learning approaches. The codes are publicly available at https://github.com/m-qiang/PialNN

Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed

Wire mesh design

We present a computational approach for designing wire meshes, i.e., freeform surfaces composed of woven wires arranged in a regular grid. To facilitate shape exploration, we map material properties of wire meshes to the geometric model of Chebyshev nets. This abstraction is exploited to build an efficient optimization scheme. While the theory of Chebyshev nets suggests a highly constrained design space, we show that allowing controlled deviations from the underlying surface provides a rich shape space for design exploration. Our algorithm balances globally coupled material constraints with aesthetic and geometric design objectives that can be specified by the user in an interactive design session. In addition to sculptural art, wire meshes represent an innovative medium for industrial applications including composite materials and architectural façades. We demonstrate the effectiveness of our approach using a variety of digital and physical prototypes with a level of shape complexity unobtainable using previous methods

Numerical simulation of fracture pattern development and implications for fuid flow

Simulations are instrumental to understanding flow through discrete fracture geometric representations that capture the large-scale permeability structure of fractured porous media. The contribution of this thesis is threefold: an efficient finite-element finite-volume discretisation of the advection/diffusion flow equations, a geomechanical fracture propagation algorithm to create fractured rock analogues, and a study of the effect of growth on hydraulic conductivity. We describe an iterative geomechanics-based finite-element model to simulate quasi-static crack propagation in a linear elastic matrix from an initial set of random flaws. The cornerstones are a failure and propagation criterion as well as a geometric kernel for dynamic shape housekeeping and automatic remeshing. Two-dimensional patterns exhibit connectivity, spacing, and density distributions reproducing en echelon crack linkage, tip hooking, and polygonal shrinkage forms. Differential stresses at the boundaries yield fracture curving. A stress field study shows that curvature can be suppressed by layer interaction effects. Our method is appropriate to model layered media where interaction with neighbouring layers does not dominate deformation. Geomechanically generated fracture patterns are the input to single-phase flow simulations through fractures and matrix. Thus, results are applicable to fractured porous media in addition to crystalline rocks. Stress state and deformation history control emergent local fracture apertures. Results depend on the number of initial flaws, their initial random distribution, and the permeability of the matrix. Straightpath fracture pattern simplifications yield a lower effective permeability in comparison to their curved counterparts. Fixed apertures overestimate the conductivity of the rock by up to six orders of magnitude. Local sample percolation effects are representative of the entire model flow behaviour for geomechanical apertures. Effective permeability in fracture dataset subregions are higher than the overall conductivity of the system. The presented methodology captures emerging patterns due to evolving geometric and flow properties essential to the realistic simulation of subsurface processes

Patch-based Progressive 3D Point Set Upsampling

We present a detail-driven deep neural network for point set upsampling. A high-resolution point set is essential for point-based rendering and surface reconstruction. Inspired by the recent success of neural image super-resolution techniques, we progressively train a cascade of patch-based upsampling networks on different levels of detail end-to-end. We propose a series of architectural design contributions that lead to a substantial performance boost. The effect of each technical contribution is demonstrated in an ablation study. Qualitative and quantitative experiments show that our method significantly outperforms the state-of-the-art learning-based and optimazation-based approaches, both in terms of handling low-resolution inputs and revealing high-fidelity details.Comment: accepted to cvpr2019, code available at https://github.com/yifita/P3