Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy

In recent years, endomicroscopy has become increasingly used for diagnostic purposes and interventional guidance. It can provide intraoperative aids for real-time tissue characterization and can help to perform visual investigations aimed for example to discover epithelial cancers. Due to physical constraints on the acquisition process, endomicroscopy images, still today have a low number of informative pixels which hampers their quality. Post-processing techniques, such as Super-Resolution (SR), are a potential solution to increase the quality of these images. SR techniques are often supervised, requiring aligned pairs of low-resolution (LR) and high-resolution (HR) images patches to train a model. However, in our domain, the lack of HR images hinders the collection of such pairs and makes supervised training unsuitable. For this reason, we propose an unsupervised SR framework based on an adversarial deep neural network with a physically-inspired cycle consistency, designed to impose some acquisition properties on the super-resolved images. Our framework can exploit HR images, regardless of the domain where they are coming from, to transfer the quality of the HR images to the initial LR images. This property can be particularly useful in all situations where pairs of LR/HR are not available during the training. Our quantitative analysis, validated using a database of 238 endomicroscopy video sequences from 143 patients, shows the ability of the pipeline to produce convincing super-resolved images. A Mean Opinion Score (MOS) study also confirms this quantitative image quality assessment.Comment: Accepted for publication on Medical Image Analysis journa

Beyond the noise : high fidelity MR signal processing

This thesis describes a variety of methods developed to increase the sensitivity and resolution of liquid state nuclear magnetic resonance (NMR) experiments. NMR is known as one of the most versatile non-invasive analytical techniques yet often suffers from low sensitivity. The main contribution to this low sensitivity issue is a presence of noise and level of noise in the spectrum is expressed numerically as “signal-to-noise ratio”. NMR signal processing involves sensitivity and resolution enhancement achieved by noise reduction using mathematical algorithms. A singular value decomposition based reduced rank matrix method, composite property mapping, in particular is studied extensively in this thesis to present its advantages, limitations, and applications. In theory, when the sum of k noiseless sinusoidal decays is formatted into a specific matrix form (i.e., Toeplitz), the matrix is known to possess k linearly independent columns. This information becomes apparent only after a singular value decomposition of the matrix. Singular value decomposition factorises the large matrix into three smaller submatrices: right and left singular vector matrices, and one diagonal matrix containing singular values. Were k noiseless sinusoidal decays involved, there would be only k nonzero singular values appearing in the diagonal matrix in descending order providing the information of the amplitude of each sinusoidal decay. The number of non-zero singular values or the number of linearly independent columns is known as the rank of the matrix. With real NMR data none of the singular values equals zero and the matrix has full rank. The reduction of the rank of the matrix and thus the noise in the reconstructed NMR data can be achieved by replacing all the singular values except the first k values with zeroes. This noise reduction process becomes difficult when biomolecular NMR data is to be processed due to the number of resonances being unknown and the presence of a large solvent peak

Convolutional Dictionary Learning: Acceleration and Convergence

Convolutional dictionary learning (CDL or sparsifying CDL) has many applications in image processing and computer vision. There has been growing interest in developing efficient algorithms for CDL, mostly relying on the augmented Lagrangian (AL) method or the variant alternating direction method of multipliers (ADMM). When their parameters are properly tuned, AL methods have shown fast convergence in CDL. However, the parameter tuning process is not trivial due to its data dependence and, in practice, the convergence of AL methods depends on the AL parameters for nonconvex CDL problems. To moderate these problems, this paper proposes a new practically feasible and convergent Block Proximal Gradient method using a Majorizer (BPG-M) for CDL. The BPG-M-based CDL is investigated with different block updating schemes and majorization matrix designs, and further accelerated by incorporating some momentum coefficient formulas and restarting techniques. All of the methods investigated incorporate a boundary artifacts removal (or, more generally, sampling) operator in the learning model. Numerical experiments show that, without needing any parameter tuning process, the proposed BPG-M approach converges more stably to desirable solutions of lower objective values than the existing state-of-the-art ADMM algorithm and its memory-efficient variant do. Compared to the ADMM approaches, the BPG-M method using a multi-block updating scheme is particularly useful in single-threaded CDL algorithm handling large datasets, due to its lower memory requirement and no polynomial computational complexity. Image denoising experiments show that, for relatively strong additive white Gaussian noise, the filters learned by BPG-M-based CDL outperform those trained by the ADMM approach.Comment: 21 pages, 7 figures, submitted to IEEE Transactions on Image Processin

DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks

Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations - small sensor size, compact lenses and the lack of specific hardware, - impede them to achieve the quality results of DSLR cameras. In this work we present an end-to-end deep learning approach that bridges this gap by translating ordinary photos into DSLR-quality images. We propose learning the translation function using a residual convolutional neural network that improves both color rendition and image sharpness. Since the standard mean squared loss is not well suited for measuring perceptual image quality, we introduce a composite perceptual error function that combines content, color and texture losses. The first two losses are defined analytically, while the texture loss is learned in an adversarial fashion. We also present DPED, a large-scale dataset that consists of real photos captured from three different phones and one high-end reflex camera. Our quantitative and qualitative assessments reveal that the enhanced image quality is comparable to that of DSLR-taken photos, while the methodology is generalized to any type of digital camera

Computer-aided analysis for the Mechanics of Granular Materials (MGM) experiment, part 2

Computer vision based analysis for the MGM experiment is continued and expanded into new areas. Volumetric strains of granular material triaxial test specimens have been measured from digitized images. A computer-assisted procedure is used to identify the edges of the specimen, and the edges are used in a 3-D model to estimate specimen volume. The results of this technique compare favorably to conventional measurements. A simplified model of the magnification caused by diffraction of light within the water of the test apparatus was also developed. This model yields good results when the distance between the camera and the test specimen is large compared to the specimen height. An algorithm for a more accurate 3-D magnification correction is also presented. The use of composite and RGB (red-green-blue) color cameras is discussed and potentially significant benefits from using an RGB camera are presented

A cross-layer approach to enhance QoS for multimedia applications over satellite

The need for on-demand QoS support for communications over satellite is of primary importance for distributed multimedia applications. This is particularly true for the return link which is often a bottleneck due to the large set of end-users accessing a very limited uplink resource. Facing this need, Demand Assignment Multiple Access (DAMA) is a classical technique that allows satellite operators to offer various types of services, while managing the resources of the satellite system efficiently. Tackling the quality degradation and delay accumulation issues that can result from the use of these techniques, this paper proposes an instantiation of the Application Layer Framing (ALF) approach, using a cross-layer interpreter(xQoS-Interpreter). The information provided by this interpreter is used to manage the resource provided to a terminal by the satellite system in order to improve the quality of multimedia presentations from the end users point of view. Several experiments are carried out for different loads on the return link. Their impact on QoS is measured through different application as well as network level metrics