Meta-Registration: Learning Test-Time Optimization for Single-Pair Image Registration

Neural networks have been proposed for medical image registration by learning, with a substantial amount of training data, the optimal transformations between image pairs. These trained networks can further be optimized on a single pair of test images - known as test-time optimization. This work formulates image registration as a meta-learning algorithm. Such networks can be trained by aligning the training image pairs while simultaneously improving test-time optimization efficacy; tasks which were previously considered two independent training and optimization processes. The proposed meta-registration is hypothesized to maximize the efficiency and effectiveness of the test-time optimization in the "outer" meta-optimization of the networks. For image guidance applications that often are time-critical yet limited in training data, the potentially gained speed and accuracy are compared with classical registration algorithms, registration networks without meta-learning, and single-pair optimization without test-time optimization data. Experiments are presented in this paper using clinical transrectal ultrasound image data from 108 prostate cancer patients. These experiments demonstrate the effectiveness of a meta-registration protocol, which yields significantly improved performance relative to existing learning-based methods. Furthermore, the meta-registration achieves comparable results to classical iterative methods in a fraction of the time, owing to its rapid test-time optimization process.Comment: Accepted to ASMUS 2022 Workshop at MICCA

Meta-Learning Initializations for Interactive Medical Image Registration

We present a meta-learning framework for interactive medical image registration. Our proposed framework comprises three components: a learning-based medical image registration algorithm, a form of user interaction that refines registration at inference, and a meta-learning protocol that learns a rapidly adaptable network initialization. This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application: registration of magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled transrectal ultrasound (TRUS) images. Our approach obtains comparable registration error (4.26 mm) to the best-performing non-interactive learning-based 3D-to-3D method (3.97 mm) while requiring only a fraction of the data, and occurring in real-time during acquisition. Applying sparsely sampled data to non-interactive methods yields higher registration errors (6.26 mm), demonstrating the effectiveness of interactive MR-TRUS registration, which may be applied intraoperatively given the real-time nature of the adaptation process.Comment: 11 pages, 10 figures. Paper accepted to IEEE Transactions on Medical Imaging (October 26 2022

Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks

In this paper, we describe how a patient-specific, ultrasound-probe-induced prostate motion model can be directly generated from a single preoperative MR image. Our motion model allows for sampling from the conditional distribution of dense displacement fields, is encoded by a generative neural network conditioned on a medical image, and accepts random noise as additional input. The generative network is trained by a minimax optimisation with a second discriminative neural network, tasked to distinguish generated samples from training motion data. In this work, we propose that 1) jointly optimising a third conditioning neural network that pre-processes the input image, can effectively extract patient-specific features for conditioning; and 2) combining multiple generative models trained separately with heuristically pre-disjointed training data sets can adequately mitigate the problem of mode collapse. Trained with diagnostic T2-weighted MR images from 143 real patients and 73,216 3D dense displacement fields from finite element simulations of intraoperative prostate motion due to transrectal ultrasound probe pressure, the proposed models produced physically-plausible patient-specific motion of prostate glands. The ability to capture biomechanically simulated motion was evaluated using two errors representing generalisability and specificity of the model. The median values, calculated from a 10-fold cross-validation, were 2.8+/-0.3 mm and 1.7+/-0.1 mm, respectively. We conclude that the introduced approach demonstrates the feasibility of applying state-of-the-art machine learning algorithms to generate organ motion models from patient images, and shows significant promise for future research.Comment: Accepted to MICCAI 201

Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.Comment: Accepted to MICCAI 201

Multimodality Biomedical Image Registration using Free Point Transformer Networks

We describe a point-set registration algorithm based on a novel free point transformer (FPT) network, designed for points extracted from multimodal biomedical images for registration tasks, such as those frequently encountered in ultrasound-guided interventional procedures. FPT is constructed with a global feature extractor which accepts unordered source and target point-sets of variable size. The extracted features are conditioned by a shared multilayer perceptron point transformer module to predict a displacement vector for each source point, transforming it into the target space. The point transformer module assumes no vicinity or smoothness in predicting spatial transformation and, together with the global feature extractor, is trained in a data-driven fashion with an unsupervised loss function. In a multimodal registration task using prostate MR and sparsely acquired ultrasound images, FPT yields comparable or improved results over other rigid and non-rigid registration methods. This demonstrates the versatility of FPT to learn registration directly from real, clinical training data and to generalize to a challenging task, such as the interventional application presented.Comment: 10 pages, 4 figures. Accepted for publication at International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) workshop on Advances in Simplifying Medical UltraSound (ASMUS) 202

Lysyl-tRNA Synthetase from Pseudomonas aeruginosa: Characterization and Identification of Inhibitory Compounds

Pseudomonas aeruginosa is an opportunistic pathogen that causes nosocomial infections and has highly developed systems for acquiring resistance against numerous antibiotics. The gene (lysS) encoding P. aeruginosa lysyl-tRNA synthetase (LysRS) was cloned and overexpressed, and the resulting protein was purified to 98% homogeneity. LysRS was kinetically evaluated, and the Km values for the interaction with lysine, adenosine triphosphate (ATP), and tRNALys were determined to be 45.5, 627, and 3.3 µM, respectively. The kcatobs values were calculated to be 13, 22.8, and 0.35 s−1, resulting in kcatobs/KM values of 0.29, 0.036, and 0.11 s−1µM−1, respectively. Using scintillation proximity assay technology, natural product and synthetic compound libraries were screened to identify inhibitors of function of the enzyme. Three compounds (BM01D09, BT06F11, and BT08F04) were identified with inhibitory activity against LysRS. The IC50 values were 17, 30, and 27 µM for each compound, respectively. The minimum inhibitory concentrations were determined against a panel of clinically important pathogens. All three compounds were observed to inhibit the growth of gram-positive organisms with a bacteriostatic mode of action. However, two compounds (BT06F11 and BT08F04) were bactericidal against cultures of gram-negative bacteria. When tested against human cell cultures, BT06F11 was not toxic at any concentration tested, and BM01D09 was toxic only at elevated levels. However, BT08F04 displayed a CC50 of 61 µg/mL. In studies of the mechanism of inhibition, BM01D09 inhibited LysRS activity by competing with ATP for binding, and BT08F04 was competitive with ATP and uncompetitive with the amino acid. BT06F11 inhibited LysRS activity by a mechanism other than substrate competition

An SPH multi-fluid model based on quasi-buoyancy for interface stabilization up to high density ratios and realistic wave speed ratios

We introduce a Smoothed Particle Hydrodynamics (SPH) concept for the stabilization of the interface between two fluids. It is demonstrated that the change in the pressure gradient across the interface leads to a force imbalance. This force imbalance is attributed to the particle approximation implicit to SPH. To stabilize the interface a pressure gradient correction is proposed. In this approach the multi-fluid pressure gradients are related to the (gravitational and fluid) accelerations. This leads to a quasi-buoyancy correction for hydrostatic (stratified) flows, which is extended to non-hydrostatic flows. The result is a simple density correction which involves no parameters or coefficients. This correction is included as an extra term in the SPH momentum equation. The new concept for the stabilization of the interface is explored in five case studies and compared with other multi-fluid models. The first case is the stagnant flow in a tank: the interface remains stable up to density ratios of 1:1000 (typical for water and air) in combination with artificial wave speed ratios up to 1:4. The second and third cases are the Rayleigh-Taylor instability and the rising bubble, where a reasonable agreement between SPH and level-set models is achieved. The fourth case is an air flow across a water surface up to density ratios of 1:100, artificial wave speeds for water higher than that of air, and high air velocities. The fifth case is about the propagation of internal gravity waves up to density ratios of 1:100 and artificial wave speed ratios of 1:2. It is demonstrated that the quasi-buoyancy model may be used to stabilize the interface between two fluids up to high density ratios, with real (low) viscosities and more realistic wave speed ratios than achieved by other WCSPH multi-fluid models. Real wave speed ratios can be achieved, as long as the fluid velocities are not very high. Although the wave speeds may be artificial in many cases, correct and realistic wave speed ratios are essential in the modelling of heat transfer between two fluids (e.g. in engineering applications such as gas turbines)

Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a cross-entropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fully-automatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global- and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201

Multi-phase synthetic contrast enhancement in interventional computed tomography for guiding renal cryotherapy

PURPOSE: Minimally invasive treatments for renal carcinoma offer a low rate of complications and quick recovery. One drawback of the use of computed tomography (CT) for needle guidance is the use of iodinated contrast agents, which require an increased X-ray dose and can potentially cause adverse reactions. The purpose of this work is to generalise the problem of synthetic contrast enhancement to allow the generation of multiple phases on non-contrast CT data from a real-world, clinical dataset without training multiple convolutional neural networks. METHODS: A framework for switching between contrast phases by conditioning the network on the phase information is proposed and compared with separately trained networks. We then examine how the degree of supervision affects the generated contrast by evaluating three established architectures: U-Net (fully supervised), Pix2Pix (adversarial with supervision), and CycleGAN (fully adversarial). RESULTS: We demonstrate that there is no performance loss when testing the proposed method against separately trained networks. Of the training paradigms investigated, the fully adversarial CycleGAN performs the worst, while the fully supervised U-Net generates more realistic voxel intensities and performed better than Pix2Pix in generating contrast images for use in a downstream segmentation task. Lastly, two models are shown to generalise to intra-procedural data not seen during the training process, also enhancing features such as needles and ice balls relevant to interventional radiological procedures. CONCLUSION: The proposed contrast switching framework is a feasible option for generating multiple contrast phases without the overhead of training multiple neural networks, while also being robust towards unseen data and enhancing contrast in features relevant to clinical practice

Identification and Characterization of a Chemical Compound that Inhibits Methionyl-tRNA Synthetase from Pseudomonas aeruginosa

Background: Pseudomonas aeruginosa is an opportunistic pathogen problematic in causing nosocomial infections and is highly susceptible to development of resistance to multiple antibiotics. The gene encoding methionyl-tRNA synthetase (MetRS) from P. aeruginosa was cloned and the resulting protein characterized. Methods: MetRS was kinetically evaluated and the KM for its three substrates, methionine, ATP and tRNAMet were determined to be 35, 515, and 29 μM, respectively. P. aeruginosaMetRS was used to screen two chemical compound libraries containing 1690 individual compounds. Results: A natural product compound (BM01C11) was identified that inhibited the aminoacylation function. The compound inhibited P. aeruginosa MetRS with an IC50 of 70 μM. The minimum inhibitory concentration (MIC) of BM01C11 was determined against nine clinically relevant bacterial strains, including efflux pump mutants and hypersensitive strains of P. aeruginosa and E. coli. The MIC against the hypersensitive strain of P. aeruginosa was 16 μg/ml. However, the compound was not effective against the wild-type and efflux pump mutant strains, indicating that efflux may not be responsible for the lack of activity against the wild-type strains. When tested in human cell cultures, the cytotoxicity concentration (CC50) was observed to be 30 μg/ml. The compound did not compete with methionine or ATP for binding MetRS, indicating that the mechanism of action of the compound likely occurs outside the active site of aminoacylation. Conclusion: An inhibitor of P. aeruginosa MetRS, BM01C11, was identified as a flavonoid compound named isopomiferin. Isopomiferin inhibited the enzymatic activity of MetRS and displayed broad spectrum antibacterial activity. These studies indicate that isopomiferin may be amenable to development as a therapeutic for bacterial infections