90 research outputs found
OSNet & MNetO: Two Types of General Reconstruction Architectures for Linear Computed Tomography in Multi-Scenarios
Recently, linear computed tomography (LCT) systems have actively attracted
attention. To weaken projection truncation and image the region of interest
(ROI) for LCT, the backprojection filtration (BPF) algorithm is an effective
solution. However, in BPF for LCT, it is difficult to achieve stable interior
reconstruction, and for differentiated backprojection (DBP) images of LCT,
multiple rotation-finite inversion of Hilbert transform (Hilbert
filtering)-inverse rotation operations will blur the image. To satisfy multiple
reconstruction scenarios for LCT, including interior ROI, complete object, and
exterior region beyond field-of-view (FOV), and avoid the rotation operations
of Hilbert filtering, we propose two types of reconstruction architectures. The
first overlays multiple DBP images to obtain a complete DBP image, then uses a
network to learn the overlying Hilbert filtering function, referred to as the
Overlay-Single Network (OSNet). The second uses multiple networks to train
different directional Hilbert filtering models for DBP images of multiple
linear scannings, respectively, and then overlays the reconstructed results,
i.e., Multiple Networks Overlaying (MNetO). In two architectures, we introduce
a Swin Transformer (ST) block to the generator of pix2pixGAN to extract both
local and global features from DBP images at the same time. We investigate two
architectures from different networks, FOV sizes, pixel sizes, number of
projections, geometric magnification, and processing time. Experimental results
show that two architectures can both recover images. OSNet outperforms BPF in
various scenarios. For the different networks, ST-pix2pixGAN is superior to
pix2pixGAN and CycleGAN. MNetO exhibits a few artifacts due to the differences
among the multiple models, but any one of its models is suitable for imaging
the exterior edge in a certain direction.Comment: 13 pages, 13 figure
Deep Learning for Cancer Prognosis Prediction Using Portrait Photos by StyleGAN Embedding
Survival prediction for cancer patients is critical for optimal treatment
selection and patient management. Current patient survival prediction methods
typically extract survival information from patients' clinical record data or
biological and imaging data. In practice, experienced clinicians can have a
preliminary assessment of patients' health status based on patients' observable
physical appearances, which are mainly facial features. However, such
assessment is highly subjective. In this work, the efficacy of objectively
capturing and using prognostic information contained in conventional portrait
photographs using deep learning for survival predication purposes is
investigated for the first time. A pre-trained StyleGAN2 model is fine-tuned on
a custom dataset of our cancer patients' photos to empower its generator with
generative ability suitable for patients' photos. The StyleGAN2 is then used to
embed the photographs to its highly expressive latent space. Utilizing the
state-of-the-art survival analysis models and based on StyleGAN's latent space
photo embeddings, this approach achieved a C-index of 0.677, which is notably
higher than chance and evidencing the prognostic value embedded in simple 2D
facial images. In addition, thanks to StyleGAN's interpretable latent space,
our survival prediction model can be validated for relying on essential facial
features, eliminating any biases from extraneous information like clothing or
background. Moreover, a health attribute is obtained from regression
coefficients, which has important potential value for patient care
- …