Astronomical image processing based on fractional calculus: the AstroFracTool

The implementation of fractional differential calculations can give new possibilities for image processing tools, in particular for those that are devoted to astronomical images analysis. As discussed in arxiv:0910.2381, the fractional differentiation is able to enhance the quality of images, with interesting effects in edge detection and image restoration. Here, we propose the AstroFracTool, developed to provide a simple yet powerful enhancement tool-set for astronomical images. This tool works evaluating the fractional gradient of an image map. It can help produce an output image useful for further research and scientific purposes, such as the detection of faint objects and galaxy structures, or, in the case of planetary studies, the enhancement of surface details.Comment: Keywords: Fractional calculation, image processing, astronom

UG^2: a Video Benchmark for Assessing the Impact of Image Restoration and Enhancement on Automatic Visual Recognition

Advances in image restoration and enhancement techniques have led to discussion about how such algorithmscan be applied as a pre-processing step to improve automatic visual recognition. In principle, techniques like deblurring and super-resolution should yield improvements by de-emphasizing noise and increasing signal in an input image. But the historically divergent goals of the computational photography and visual recognition communities have created a significant need for more work in this direction. To facilitate new research, we introduce a new benchmark dataset called UG^2, which contains three difficult real-world scenarios: uncontrolled videos taken by UAVs and manned gliders, as well as controlled videos taken on the ground. Over 160,000 annotated frames forhundreds of ImageNet classes are available, which are used for baseline experiments that assess the impact of known and unknown image artifacts and other conditions on common deep learning-based object classification approaches. Further, current image restoration and enhancement techniques are evaluated by determining whether or not theyimprove baseline classification performance. Results showthat there is plenty of room for algorithmic innovation, making this dataset a useful tool going forward.Comment: Supplemental material: https://goo.gl/vVM1xe, Dataset: https://goo.gl/AjA6En, CVPR 2018 Prize Challenge: ug2challenge.or

Techniques in image restoration and enhancement

Includes bibliographical references.Image processing in its broad sense pervades many areas but it is convenient to group it into three main sections, viz: image coding, usually for image transmission over telecommunication links; pattern recognition for detecting the presence of a particular distribution in an image which is generally corrupted to some extent by noise; and image restoration, which aims to recover a faithful reproduction of a perfect image which has been degraded, and image enhancement which attempts to present an image in a form which will convey most readily the desired information to the human brain and takes account of the characteristics of vision. It is the object of this thesis to investigate some of the techniques of image restoration and enhancement. There are many different media for implementing the various processes, but digital computation and coherent optics are prevalent

Burstormer: Burst Image Restoration and Enhancement Transformer

On a shutter press, modern handheld cameras capture multiple images in rapid succession and merge them to generate a single image. However, individual frames in a burst are misaligned due to inevitable motions and contain multiple degradations. The challenge is to properly align the successive image shots and merge their complimentary information to achieve high-quality outputs. Towards this direction, we propose Burstormer: a novel transformer-based architecture for burst image restoration and enhancement. In comparison to existing works, our approach exploits multi-scale local and non-local features to achieve improved alignment and feature fusion. Our key idea is to enable inter-frame communication in the burst neighborhoods for information aggregation and progressive fusion while modeling the burst-wide context. However, the input burst frames need to be properly aligned before fusing their information. Therefore, we propose an enhanced deformable alignment module for aligning burst features with regards to the reference frame. Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions. After multi-level alignment and enrichment, we re-emphasize on inter-frame communication within burst using a cyclic burst sampling module. Finally, the inter-frame information is aggregated using the proposed burst feature fusion module followed by progressive upsampling. Our Burstormer outperforms state-of-the-art methods on burst super-resolution, burst denoising and burst low-light enhancement. Our codes and pretrained models are available at https:// github.com/akshaydudhane16/BurstormerComment: Accepted at CVPR 202

Image Enhancement and Restoration for Colonoscopy Images

Colonoscopy images contain specular highlights that occur as a result of the tiny camera on the colonoscope being perpendicular to the image location. These specular highlights may prevent the Gastroenterologist from having a full picture of the patient’s condition and potentially giving an early diagnosis. The purpose of my honors research is to remove the specular highlights from these colonoscopy images. The process to achieve the above objective involves two steps. The first step would require locating the specular highlights in the image through image segmentation. For this purpose, information from nearby x and y pixels may be utilized. The second step consists of using image restoration to fill in the specular regions. The removal of these specular highlights refines the colonoscopy image and allows useful information to be deduced by the physician

Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond

An authentic face restoration system is becoming increasingly demanding in many computer vision applications, e.g., image enhancement, video communication, and taking portrait. Most of the advanced face restoration models can recover high-quality faces from low-quality ones but usually fail to faithfully generate realistic and high-frequency details that are favored by users. To achieve authentic restoration, we propose $\textbf{IDM}$, an $\textbf{I}$teratively learned face restoration system based on denoising $\textbf{D}$iffusion $\textbf{M}$odels (DDMs). We define the criterion of an authentic face restoration system, and argue that denoising diffusion models are naturally endowed with this property from two aspects: intrinsic iterative refinement and extrinsic iterative enhancement. Intrinsic learning can preserve the content well and gradually refine the high-quality details, while extrinsic enhancement helps clean the data and improve the restoration task one step further. We demonstrate superior performance on blind face restoration tasks. Beyond restoration, we find the authentically cleaned data by the proposed restoration system is also helpful to image generation tasks in terms of training stabilization and sample quality. Without modifying the models, we achieve better quality than state-of-the-art on FFHQ and ImageNet generation using either GANs or diffusion models.Comment: ICCV 202