27 research outputs found
Neural Gradient Regularizer
Owing to its significant success, the prior imposed on gradient maps has
consistently been a subject of great interest in the field of image processing.
Total variation (TV), one of the most representative regularizers, is known for
its ability to capture the sparsity of gradient maps. Nonetheless, TV and its
variants often underestimate the gradient maps, leading to the weakening of
edges and details whose gradients should not be zero in the original image.
Recently, total deep variation (TDV) has been introduced, assuming the sparsity
of feature maps, which provides a flexible regularization learned from
large-scale datasets for a specific task. However, TDV requires retraining when
the image or task changes, limiting its versatility. In this paper, we propose
a neural gradient regularizer (NGR) that expresses the gradient map as the
output of a neural network. Unlike existing methods, NGR does not rely on the
sparsity assumption, thereby avoiding the underestimation of gradient maps. NGR
is applicable to various image types and different image processing tasks,
functioning in a zero-shot learning fashion, making it a versatile and
plug-and-play regularizer. Extensive experimental results demonstrate the
superior performance of NGR over state-of-the-art counterparts for a range of
different tasks, further validating its effectiveness and versatility
Graph- and finite element-based total variation models for the inverse problem in diffuse optical tomography
Total variation (TV) is a powerful regularization method that has been widely
applied in different imaging applications, but is difficult to apply to diffuse
optical tomography (DOT) image reconstruction (inverse problem) due to complex
and unstructured geometries, non-linearity of the data fitting and
regularization terms, and non-differentiability of the regularization term. We
develop several approaches to overcome these difficulties by: i) defining
discrete differential operators for unstructured geometries using both finite
element and graph representations; ii) developing an optimization algorithm
based on the alternating direction method of multipliers (ADMM) for the
non-differentiable and non-linear minimization problem; iii) investigating
isotropic and anisotropic variants of TV regularization, and comparing their
finite element- and graph-based implementations. These approaches are evaluated
on experiments on simulated data and real data acquired from a tissue phantom.
Our results show that both FEM and graph-based TV regularization is able to
accurately reconstruct both sparse and non-sparse distributions without the
over-smoothing effect of Tikhonov regularization and the over-sparsifying
effect of L regularization. The graph representation was found to
out-perform the FEM method for low-resolution meshes, and the FEM method was
found to be more accurate for high-resolution meshes.Comment: 24 pages, 11 figures. Reviced version includes revised figures and
improved clarit
Optical System Identification for Passive Electro-Optical Imaging
A statistical inverse-problem approach is presented for jointly estimating camera blur from aliased data of a known calibration target. Specifically, a parametric Maximum Likelihood (ML) PSF estimate is derived for characterizing a camera's optical imperfections through the use of a calibration target in an otherwise loosely controlled environment. The unknown parameters are jointly estimated from data described by a physical forward-imaging model, and this inverse-problem approach allows one to accommodate all of the available sources of information jointly. These sources include knowledge of the forward imaging process, the types and sources of statistical uncertainty, available prior information, and the data itself. The forward model describes a broad class of imaging systems based on a parameterization with a direct mapping between its parameters and physical imaging phenomena. The imaging perspective, ambient light-levels, target-reflectance, detector gain and offset, quantum-efficiency, and read-noise levels are all treated as nuisance parameters. The Cram'{e}r-Rao Bound (CRB) is derived under this joint model, and simulations demonstrate that the proposed estimator achieves near-optimal MSE performance. Finally, the proposed method is applied to experimental data to validate both the fidelity of the forward-models, as well as to establish the utility of the resulting ML estimates for both system identification and subsequent image restoration.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/153395/1/jwleblan_1.pd
Robust contour propagation using deep learning and image registration for online adaptive proton therapy of prostate cancer
Purpose
To develop and validate a robust and accurate registration pipeline for automatic contour propagation for online adaptive IntensityâModulated Proton Therapy (IMPT) of prostate cancer using elastix software and deep learning.
Methods
A threeâdimensional (3D) Convolutional Neural Network was trained for automatic bladder segmentation of the computed tomography (CT) scans. The automatic bladder segmentation alongside the computed tomography (CT) scan is jointly optimized to add explicit knowledge about the underlying anatomy to the registration algorithm. We included three datasets from different institutes and CT manufacturers. The first was used for training and testing the ConvNet, where the second and the third were used for evaluation of the proposed pipeline. The system performance was quantified geometrically using the dice similarity coefficient (DSC), the mean surface distance (MSD), and the 95% Hausdorff distance (HD). The propagated contours were validated clinically through generating the associated IMPT plans and compare it with the IMPT plans based on the manual delineations. Propagated contours were considered clinically acceptable if their treatment plans met the dosimetric coverage constraints on the manual contours.
Results
The bladder segmentation network achieved a DSC of 88% and 82% on the test datasets. The proposed registration pipeline achieved a MSD of 1.29 ± 0.39, 1.48 ± 1.16, and 1.49 ± 0.44 mm for the prostate, seminal vesicles, and lymph nodes, respectively, on the second dataset and a MSD of 2.31 ± 1.92 and 1.76 ± 1.39 mm for the prostate and seminal vesicles on the third dataset. The automatically propagated contours met the dose coverage constraints in 86%, 91%, and 99% of the cases for the prostate, seminal vesicles, and lymph nodes, respectively. A Conservative Success Rate (CSR) of 80% was obtained, compared to 65% when only using intensityâbased registration.
Conclusion
The proposed registration pipeline obtained highly promising results for generating treatment plans adapted to the daily anatomy. With 80% of the automatically generated treatment plans directly usable without manual correction, a substantial improvement in system robustness was reached compared to a previous approach. The proposed method therefore facilitates more precise proton therapy of prostate cancer, potentially leading to fewer treatmentârelated adverse side effects
Dynamic 3D Avatar Creation from Hand-held Video Input
We present a complete pipeline for creating fully rigged, personalized 3D facial avatars from hand-held video. Our system faithfully recovers facial expression dynamics of the user by adapting a blendshape template to an image sequence of recorded expressions using an optimization that integrates feature tracking, optical flow, and shape from shading. Fine-scale details such as wrinkles are captured separately in normal maps and ambient occlusion maps. From this user- and expression-specific data, we learn a regressor for on-the-fly detail synthesis during animation to enhance the perceptual realism of the avatars. Our system demonstrates that the use of appropriate reconstruction priors yields compelling face rigs even with a minimalistic acquisition system and limited user assistance. This facilitates a range of new applications in computer animation and consumer-level online communication based on personalized avatars. We present realtime application demos to validate our method
Automated Online-Adaptive Intensity-Modulated Proton Therapy
In this thesis I developed algorithms to adapt a proton therapy irradiation plan automatically and rapidly to the patient anatomy of the day. This saves healthy tissue better, which is expected to lead to fewer side effects