Fast Compressive 3D Single-pixel Imaging

In this work, we demonstrate a modified photometric stereo system with perfect pixel registration, capable of reconstructing continuous real-time 3D video at ~8 Hz for 64 x 64 image resolution by employing evolutionary compressed sensing

COMPARISON OF 3D VOLUME REGISTRATION TECHNIQUES APPLIED TO NEUROSURGERY

poster abstractIntroduction: Image guided surgery requires that the pre-operative da-ta used for planning the surgery should be aligned with the patient during surgery. For this surgical application a fast, effective volume registration al-gorithm is needed. In addition, such an algorithm can also be used to devel-op surgical training presentations. This research tests existing methods of image and volume registration with synthetic 3D models and with 3D skull data. The aim of this research is to find the most promising algorithms in ac-curacy and execution time that best fit the neurosurgery application. Methods: Medical image volumes acquired from MRI or CT medical im-aging scans provided by the Indiana University School of Medicine were used as Test image cases. Additional synthetic data with ground truth was devel-oped by the Informatics students. Each test image was processed through image registration algorithms found in four common medical imaging tools: MATLAB, 3D Slicer, VolView, and VTK/ITK. The resulting registration is com-pared against the ground truth evaluated with mean squared error metrics. Algorithm execution time is measured on standard personal computer (PC) hardware. Results: Data from this extensive set of tests reveal that the current state of the art algorithms all have strengths and weaknesses. These will be categorized and presented both in a poster form and in a 3D video presenta-tion produced by Informatics students in an auto stereoscopic 3D video. Conclusions: Preliminary results show that execution of image registra-tion in real-time is a challenging task for real time neurosurgery applica-tions. Final results will be available at paper presentation. Future research will focus on optimizing registration and also implementing deformable regis-tration in real-time

Real-time volumetric image reconstruction and 3D tumor localization based on a single x-ray projection image for lung cancer radiotherapy

Purpose: To develop an algorithm for real-time volumetric image reconstruction and 3D tumor localization based on a single x-ray projection image for lung cancer radiotherapy. Methods: Given a set of volumetric images of a patient at N breathing phases as the training data, we perform deformable image registration between a reference phase and the other N-1 phases, resulting in N-1 deformation vector fields (DVFs). These DVFs can be represented efficiently by a few eigenvectors and coefficients obtained from principal component analysis (PCA). By varying the PCA coefficients, we can generate new DVFs, which, when applied on the reference image, lead to new volumetric images. We then can reconstruct a volumetric image from a single projection image by optimizing the PCA coefficients such that its computed projection matches the measured one. The 3D location of the tumor can be derived by applying the inverted DVF on its position in the reference image. Our algorithm was implemented on graphics processing units (GPUs) to achieve real-time efficiency. We generated the training data using a realistic and dynamic mathematical phantom with 10 breathing phases. The testing data were 360 cone beam projections corresponding to one gantry rotation, simulated using the same phantom with a 50% increase in breathing amplitude. Results: The average relative image intensity error of the reconstructed volumetric images is 6.9% +/- 2.4%. The average 3D tumor localization error is 0.8 mm +/- 0.5 mm. On an NVIDIA Tesla C1060 GPU card, the average computation time for reconstructing a volumetric image from each projection is 0.24 seconds (range: 0.17 and 0.35 seconds). Conclusions: We have shown the feasibility of reconstructing volumetric images and localizing tumor positions in 3D in near real-time from a single x-ray image.Comment: 8 pages, 3 figures, submitted to Medical Physics Lette

Meta-Learning Initializations for Interactive Medical Image Registration

We present a meta-learning framework for interactive medical image registration. Our proposed framework comprises three components: a learning-based medical image registration algorithm, a form of user interaction that refines registration at inference, and a meta-learning protocol that learns a rapidly adaptable network initialization. This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application: registration of magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled transrectal ultrasound (TRUS) images. Our approach obtains comparable registration error (4.26 mm) to the best-performing non-interactive learning-based 3D-to-3D method (3.97 mm) while requiring only a fraction of the data, and occurring in real-time during acquisition. Applying sparsely sampled data to non-interactive methods yields higher registration errors (6.26 mm), demonstrating the effectiveness of interactive MR-TRUS registration, which may be applied intraoperatively given the real-time nature of the adaptation process.Comment: 11 pages, 10 figures. Paper accepted to IEEE Transactions on Medical Imaging (October 26 2022