49 research outputs found
Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank
For many applications the collection of labeled data is expensive laborious.
Exploitation of unlabeled data during training is thus a long pursued objective
of machine learning. Self-supervised learning addresses this by positing an
auxiliary task (different, but related to the supervised task) for which data
is abundantly available. In this paper, we show how ranking can be used as a
proxy task for some regression problems. As another contribution, we propose an
efficient backpropagation technique for Siamese networks which prevents the
redundant computation introduced by the multi-branch network architecture. We
apply our framework to two regression problems: Image Quality Assessment (IQA)
and Crowd Counting. For both we show how to automatically generate ranked image
sets from unlabeled data. Our results show that networks trained to regress to
the ground truth targets for labeled data and to simultaneously learn to rank
unlabeled data obtain significantly better, state-of-the-art results for both
IQA and crowd counting. In addition, we show that measuring network uncertainty
on the self-supervised proxy task is a good measure of informativeness of
unlabeled data. This can be used to drive an algorithm for active learning and
we show that this reduces labeling effort by up to 50%.Comment: Accepted at TPAMI. (Keywords: Learning from rankings, image quality
assessment, crowd counting, active learning). arXiv admin note: text overlap
with arXiv:1803.0309
Self2Self+: Single-Image Denoising with Self-Supervised Learning and Image Quality Assessment Loss
Recently, denoising methods based on supervised learning have exhibited
promising performance. However, their reliance on external datasets containing
noisy-clean image pairs restricts their applicability. To address this
limitation, researchers have focused on training denoising networks using
solely a set of noisy inputs. To improve the feasibility of denoising
procedures, in this study, we proposed a single-image self-supervised learning
method in which only the noisy input image is used for network training. Gated
convolution was used for feature extraction and no-reference image quality
assessment was used for guiding the training process. Moreover, the proposed
method sampled instances from the input image dataset using Bernoulli sampling
with a certain dropout rate for training. The corresponding result was produced
by averaging the generated predictions from various instances of the trained
network with dropouts. The experimental results indicated that the proposed
method achieved state-of-the-art denoising performance on both synthetic and
real-world datasets. This highlights the effectiveness and practicality of our
method as a potential solution for various noise removal tasks.Comment: Technical report and supplemantry materials are combined into one
paper. - Technical report: Page 1~7 - Supplemantry materials : Page 8~1
CrossScore: towards multi-view image evaluation and scoring
We introduce a novel cross-reference image quality assessment method that effectively fills the gap in the image assessment landscape, complementing the array of established evaluation schemes – ranging from full-reference metrics like SSIM, no-reference metrics such as NIQE, to general-reference metrics including FID, and Multi-modal-reference metrics, e.g., CLIPScore. Utilising a neural network with the cross-attention mechanism and a unique data collection pipeline from NVS optimisation, our method enables accurate image quality assessment without requiring ground truth references. By comparing a query image against multiple views of the same scene, our method addresses the limitations of existing metrics in novel view synthesis (NVS) and similar tasks where direct reference images are unavailable. Experimental results show that our method is closely correlated to the full-reference metric SSIM, while not requiring ground truth references
Learning based Image Quality Assessment
In this thesis, we present an abstract view of image quality assessment algorithms. Most of the research in the area of image quality assessment is focused on the
scenario where the end-user is a human observer and therefore commonly known as perceptual image quality assessment. However, we believe that we should extend
the field of image quality assessment to the task specific scenario where the end-user is not a human observer. The quality of image/video should be assessed based
on the end-system/user that we call task-based image quality assessment. In this thesis, we discuss both perceptual image quality assessment and task-based image quality assessment with respect to face recognition task
A Comprehensive Review of Deep Learning-based Single Image Super-resolution
Image super-resolution (SR) is one of the vital image processing methods that
improve the resolution of an image in the field of computer vision. In the last
two decades, significant progress has been made in the field of
super-resolution, especially by utilizing deep learning methods. This survey is
an effort to provide a detailed survey of recent progress in single-image
super-resolution in the perspective of deep learning while also informing about
the initial classical methods used for image super-resolution. The survey
classifies the image SR methods into four categories, i.e., classical methods,
supervised learning-based methods, unsupervised learning-based methods, and
domain-specific SR methods. We also introduce the problem of SR to provide
intuition about image quality metrics, available reference datasets, and SR
challenges. Deep learning-based approaches of SR are evaluated using a
reference dataset. Some of the reviewed state-of-the-art image SR methods
include the enhanced deep SR network (EDSR), cycle-in-cycle GAN (CinCGAN),
multiscale residual network (MSRN), meta residual dense network (Meta-RDN),
recurrent back-projection network (RBPN), second-order attention network (SAN),
SR feedback network (SRFBN) and the wavelet-based residual attention network
(WRAN). Finally, this survey is concluded with future directions and trends in
SR and open problems in SR to be addressed by the researchers.Comment: 56 Pages, 11 Figures, 5 Table