12 research outputs found
Generative Adversarial Networks for Visible to Infrared Video Conversion
Deep learning models are data driven. For example, the most popular convolutional neural network (CNN) model used for image classification or object detection requires large labeled databases for training to achieve competitive performances. This requirement is not difficult to be satisfied in the visible domain since there are lots of labeled video and image databases available nowadays. However, given the less popularity of infrared (IR) camera, the availability of labeled infrared videos or image databases is limited. Therefore, training deep learning models in infrared domain is still challenging. In this chapter, we applied the pix2pix generative adversarial network (Pix2Pix GAN) and cycle-consistent GAN (Cycle GAN) models to convert visible videos to infrared videos. The Pix2Pix GAN model requires visible-infrared image pairs for training while the Cycle GAN relaxes this constraint and requires only unpaired images from both domains. We applied the two models to an open-source database where visible and infrared videos provided by the signal multimedia and telecommunications laboratory at the Federal University of Rio de Janeiro. We evaluated conversion results by performance metrics including Inception Score (IS), Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Our experiments suggest that cycle-consistent GAN is more effective than pix2pix GAN for generating IR images from optical images
μ¬μΈ΅ μ κ²½λ§μ νμ©ν μλ μμ μ‘μ μ κ±° κΈ°λ²
νμλ
Όλ¬Έ (λ°μ¬) -- μμΈλνκ΅ λνμ : μμ°κ³Όνλν νλκ³Όμ κ³μ°κ³Όνμ 곡, 2020. 8. κ°λͺ
μ£Ό.Noise removal in digital image data is a fundamental and important task in the field of image processing. The goal of the task is to remove noises from the given degraded images while maintaining essential details such as edges, curves, textures, etc. There have been various attempts on image denoising: mainly model-based methods such as filtering methods, total variation based methods, non-local mean based approaches. Deep learning have been attracting signiο¬cant research interest as they have shown better results than the classical methods in almost all fields. Deep learning-based methods use a large amount of data to train a network for its own objective; in the image denoising case, in order to map the corrupted image to a desired clean image.
In this thesis we proposed a new network architecture focusing on white Gaussian noise and real noise cancellation. Our model is a deep and wide network designed by constructing a basic block consisting of a mixture of various types of dilated convolutions and repeatedly stacking them. We did not use a batch normal layer to maintain the original own color information of each input data. Also skip connection was utilized so as not to lose the existing information. Through several experiments and comparisons, it was proved that the proposed network has better performance compared to the traditional and latest methods in image denoising.λμ§νΈ μμ λ°μ΄ν° λ΄μ μ‘μ μ κ±° λ° κ°μλ μ΄νλ μμμ λ
Έμ΄μ¦λ₯Ό μ κ±°νλ©΄μ λͺ¨μ리, 곑μ , μ§κ° λ±κ³Ό κ°μ νμ μΈλΆ μ 보λ₯Ό μ μ§νλ κ²μ΄ λͺ©μ μΈ μμ μ²λ¦¬ λΆμΌμ κΈ°λ³Έμ μ΄κ³ νμμ μΈ μμ
μ΄λ€. λ₯λ¬λ κΈ°λ°μ μμ μ‘μ μ κ±° λ°©λ²λ€μ μ΄νλ μμμ μνλ νμ§μ μμμΌλ‘ 맀ννλλ‘ λμ©λμ λ°μ΄ν°λ₯Ό μ΄μ©νμ¬ λ€νΈμν¬λ₯Ό μ§λνμ΅νλ©° κ³ μ μ μΈ λ°©λ²λ€λ³΄λ€ λ°μ΄λ κ²°κ³Όλ₯Ό 보μ¬μ£Όκ³ μλ€.
λ³Έ λ
Όλ¬Έμμλ μ΄λ―Έμ§ λλ
Έμ΄μ§μ λν μ¬λ¬ λ°©λ²λ€μ μ‘°μ¬νμ λΏλ§ μλλΌ, νΉν λ°±μ κ°μ°μμ μ‘μκ³Ό μ€μ μ‘μ μ κ±° λ¬Έμ μ μ§μ€νλ©΄μ λ€νΈμν¬ μν€ν
μ²λ₯Ό μ€κ³νκ³ μ€ννμλ€. μ¬λ¬ ννμ λλ μ΄ν°λ μ½λ³Όλ£¨μ
λ€μ νΌν©νμ¬ κΈ°λ³Έ λΈλ‘μ ꡬμ±νκ³ μ΄λ₯Ό λ°λ³΅νμ¬ μμμ μ€κ³ν λ€νΈμν¬λ₯Ό μ μνμκ³ , κ°κ° λ³Έμ°μ μμμ μ μ§ν μ μλλ‘ μ¬λ¬ μ
λ ₯ μμμ νλλ‘ λ¬Άμ΄ κ΅¬μ±νλ λ°°μΉλ₯Ό νμ€ννλ λ°°μΉλ
Έλ© λ μ΄μ΄λ μ¬μ©νμ§ μμλ€. κ·Έλ¦¬κ³ λΈλ‘μ΄ μ¬λ¬ μΈ΅ μ§νλλ λμ κΈ°μ‘΄μ μ 보λ₯Ό μμ€νμ§ μλλ‘ μ€ν΅ 컀λ₯μ
μ μ¬μ©νμλ€. μ¬λ¬ μ€νκ³Ό κΈ°μ‘΄μ μ μλ λ°©λ²κ³Ό μ΅μ λ²€μΉ λ§ν¬μμ λΉκ΅λ₯Ό ν΅νμ¬ μ μν λ€νΈμν¬κ° λ
Έμ΄μ¦ κ°μ λ° μ κ±° μμ
μμ κΈ°μ‘΄μ λ°©λ²λ€κ³Ό λΉκ΅νμ¬ μ°μν μ±λ₯μ κ°μ§κ³ μμμ μ
μ¦νμλ€. νμ§λ§ μ μν μν€ν
μ²λ°©λ²μ νκ³μ λ λͺ κ°μ§ μ‘΄μ¬νλ€. μ μν λ€νΈμν¬λ λ€μ΄μνλ§μ μ¬μ©νμ§ μμμΌλ‘μ¨ μ 보 μμ€μ μ΅μννμμ§λ§ μ΅μ λ²€μΉλ§ν¬μ λΉνμ¬ λ λ§μ μΆλ‘ μκ°μ΄ νμνμ¬ μ€μκ° μμ
μλ μ μ©νκΈ°κ° μ½μ§ μλ€. μ€μ μμμλ λ¨μν μ‘μ보λ€λ μμ νλ, μ μ₯ λ±κ³Ό κ°μ νλ‘μΈμ€λ₯Ό κ±°μΉλ©΄μ μ¬λ¬ μμΈλ€λ‘ μΈν λ€μν μ‘μ, λΈλ¬μ κ°μ μ΄νκ° νΌμ¬ λμ΄ μλ€. μ€μ μ‘μμ λν λ€μν κ°λμμμ λΆμκ³Ό μ¬λ¬ λͺ¨λΈλ§ μ€ν, κ·Έλ¦¬κ³ μμ μ‘μ λ° λΈλ¬, μμΆκ³Ό κ°μ λ³΅ν© λͺ¨λΈλ§μ΄ νμνλ€. ν₯νμλ μ΄λ¬ν μ λ€μ 보μν¨μΌλ‘μ¨ μ±λ₯μ ν₯μμν€κ³ λ€νΈμν¬μ μ‘°μ μ ν΅ν΄ μ€μκ°μΌλ‘ μ μ©λ μ μμμ κΈ°λνλ€.1 Introduction 1
2 Review on Image Denoising Methods 4
2.1 Image Noise Models 4
2.2 Traditional Denoising Methods 8
2.2.1 TV-based regularization 9
2.2.2 Non-local regularization 9
2.2.3 Sparse representation 10
2.2.4 Low-rank minimization 10
2.3 CNN-based Denoising Methods 11
2.3.1 DnCNN 11
2.3.2 FFDNet 12
2.3.3 WDnCNN 12
2.3.4 DHDN 13
3 Proposed models 15
3.1 Related Works 15
3.1.1 Residual learning 15
3.1.2 Dilated convolution 16
3.2 Proposed Network Architecture 17
4 Experiments 21
4.1 Training Details 21
4.2 Synthetic Noise Reduction 23
4.2.1 Set12 denoising 24
4.2.2 Kodak24 and BSD68 denoising 30
4.3 Real Noise Reduction 34
4.3.1 DnD test results 35
4.3.2 NTIRE 2020 real image denoising challenge 42
5 Conclusion and Future Works 46
Abstract (in Korean) 54Docto
Progressive Joint Low-light Enhancement and Noise Removal for Raw Images
Low-light imaging on mobile devices is typically challenging due to
insufficient incident light coming through the relatively small aperture,
resulting in a low signal-to-noise ratio. Most of the previous works on
low-light image processing focus either only on a single task such as
illumination adjustment, color enhancement, or noise removal; or on a joint
illumination adjustment and denoising task that heavily relies on short-long
exposure image pairs collected from specific camera models, and thus these
approaches are less practical and generalizable in real-world settings where
camera-specific joint enhancement and restoration is required. To tackle this
problem, in this paper, we propose a low-light image processing framework that
performs joint illumination adjustment, color enhancement, and denoising.
Considering the difficulty in model-specific data collection and the ultra-high
definition of the captured images, we design two branches: a coefficient
estimation branch as well as a joint enhancement and denoising branch. The
coefficient estimation branch works in a low-resolution space and predicts the
coefficients for enhancement via bilateral learning, whereas the joint
enhancement and denoising branch works in a full-resolution space and
progressively performs joint enhancement and denoising. In contrast to existing
methods, our framework does not need to recollect massive data when being
adapted to another camera model, which significantly reduces the efforts
required to fine-tune our approach for practical usage. Through extensive
experiments, we demonstrate its great potential in real-world low-light imaging
applications when compared with current state-of-the-art methods
Wavelet subband-specific learning for low-dose computed tomography denoising
Deep neural networks have shown great improvements in low-dose computed tomography (CT) denoising. Early algorithms were primarily optimized to obtain an accurate image with low distortion between the denoised image and reference full-dose image at the cost of yielding an overly smoothed unrealistic CT image. Recent research has sought to preserve the fine details of denoised images with high perceptual quality, which has been accompanied by a decrease in objective quality due to a trade-off between perceptual quality and distortion. We pursue a network that can generate accurate and realistic CT images with high objective and perceptual quality within one network, achieving a better perception-distortion trade-off. To achieve this goal, we propose a stationary wavelet transform-assisted network employing the characteristics of high- and low-frequency domains of the wavelet transform and frequency subband-specific losses defined in the wavelet domain. We first introduce a stationary wavelet transform for the network training procedure. Then, we train the network using objective loss functions defined for high- and low-frequency domains to enhance the objective quality of the denoised CT image. With this network design, we train the network again after replacing the objective loss functions with perceptual loss functions in high- and low-frequency domains. As a result, we acquired denoised CT images with high perceptual quality using this strategy while minimizing the objective quality loss. We evaluated our algorithms on the phantom and clinical images, and the quantitative and qualitative results indicate that ours outperform the existing state-of-the-art algorithms in terms of objective and perceptual quality.ope