Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.Comment: Accepted to MICCAI 201

Vector offset operators for deformable organic objects.

Many natural materials and most of living tissues exhibit complex deformable behaviours that may be characteriseda s organic. In computer animation, deformable organic material behaviour is needed for the development of characters and scenes based on living creatures and natural phenomena. This study addresses the problem of deformable organic material behaviour in computer animated objects. The focus of this study is concentrated on problems inherent in geometry based deformation techniques, such as non-intuitive interaction and difficulty in achieving realism. Further, the focus is concentrated on problems inherent in physically based deformation techniques, such as inefficiency and difficulty in enforcing spatial and temporal constraints. The main objective in this study is to find a general and efficient solution to interaction and animation of deformable 3D objects with natural organic material properties and constrainable behaviour. The solution must provide an interaction and animation framework suitable for the creation of animated deformable characters. An implementation of physical organic material properties such as plasticity, elasticity and iscoelasticity can provide the basis for an organic deformation model. An efficient approach to stress and strain control is introduced with a deformation tool named Vector Offset Operator. Stress / strain graphs control the elastoplastic behaviour of the model. Strain creep, stress relaxation and hysteresis graphs control the viscoelastic behaviour of the model. External forces may be applied using motion paths equipped with momentum / time graphs. Finally, spatial and temporal constraints are applied directly on vector operators. The suggested generic deformation tool introduces an intermediate layer between user interaction, deformation, elastoplastic and viscoelastic material behaviour and spatial and temporal constraints. This results in an efficient approach to deformation, frees object representation from deformation, facilitates the application of constraints and enables further development

Fast character modeling with sketch-based PDE surfaces

© 2020, The Author(s). Virtual characters are 3D geometric models of characters. They have a lot of applications in multimedia. In this paper, we propose a new physics-based deformation method and efficient character modelling framework for creation of detailed 3D virtual character models. Our proposed physics-based deformation method uses PDE surfaces. Here PDE is the abbreviation of Partial Differential Equation, and PDE surfaces are defined as sculpting force-driven shape representations of interpolation surfaces. Interpolation surfaces are obtained by interpolating key cross-section profile curves and the sculpting force-driven shape representation uses an analytical solution to a vector-valued partial differential equation involving sculpting forces to quickly obtain deformed shapes. Our proposed character modelling framework consists of global modeling and local modeling. The global modeling is also called model building, which is a process of creating a whole character model quickly with sketch-guided and template-based modeling techniques. The local modeling produces local details efficiently to improve the realism of the created character model with four shape manipulation techniques. The sketch-guided global modeling generates a character model from three different levels of sketched profile curves called primary, secondary and key cross-section curves in three orthographic views. The template-based global modeling obtains a new character model by deforming a template model to match the three different levels of profile curves. Four shape manipulation techniques for local modeling are investigated and integrated into the new modelling framework. They include: partial differential equation-based shape manipulation, generalized elliptic curve-driven shape manipulation, sketch assisted shape manipulation, and template-based shape manipulation. These new local modeling techniques have both global and local shape control functions and are efficient in local shape manipulation. The final character models are represented with a collection of surfaces, which are modeled with two types of geometric entities: generalized elliptic curves (GECs) and partial differential equation-based surfaces. Our experiments indicate that the proposed modeling approach can build detailed and realistic character models easily and quickly

Advancing Creative Visual Thinking with Constructive Function-based Modelling.

Modern education technologies are destined to reflect the realities of a modern digital age. The juxtaposition of real and synthetic (computer-generated) worlds as well as a greater emphasis on visual dimension are especially important characteristics that have to be taken into account in learning and teaching. We describe the ways in which an approach to constructive shape modelling can be used to advancing creative visual thinking in artistic and technical education. This approach assumes the use of a simple programming language or interactive software tools for creating a shape model, generating its images, and finally fabricating a real object of that model. It can be considered an educational technology suitable not only for children and students but also for researchers, artists, and designers. The corresponding modelling language and software tools are being developed within an international HyperFun Project. These tools are easy to use by students of different age, specialization and abilities, and can easily be extended and adapted for various educational purposes in different areas

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Modelling the adhesion enhancement induced by sand particle breakage at the wheel-rail interface

The adhesion at the wheel-rail contact is critical in train operation. Low adhesion leads to a longer distance for a train to accelerate and brake, and this may cause serious accidents. Sand particles are applied onboard trains at the wheel-rail contact to enhance the adhesion level. In this study, a finite element model is developed to investigate the mechanical behaviour of sand particles in a wheel-rail contact and how they affect the adhesion level. The acceleration and braking events using rolling/slipping and sliding contacts are simulated. Morphological properties of sand particles such as size and aspect ratio are considered. The adhesion enhancement is quantified from each simulation for comparison. The results indicate that the adhesion enhancement during the first contact between the wheel and sand particles is negligible and starts to increase when the wheel is rolling on the fragments. Its magnitude is controlled by the new third-body layers generated during the particle breakage under both rolling and sliding contacts. However, under sliding contact, when a similar amount of fragments is considered, the coarser particles with a larger aspect ratio tend to produce a higher adhesion enhancement