A deep learning framework for quality assessment and restoration in video endoscopy

Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, we contend that the robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution. We propose a fully automatic framework that can: 1) detect and classify six different primary artifacts, 2) provide a quality score for each frame and 3) restore mildly corrupted frames. To detect different artifacts our framework exploits fast multi-scale, single stage convolutional neural network detector. We introduce a quality metric to assess frame quality and predict image restoration success. Generative adversarial networks with carefully chosen regularization are finally used to restore corrupted frames. Our detector yields the highest mean average precision (mAP at 5% threshold) of 49.0 and the lowest computational time of 88 ms allowing for accurate real-time processing. Our restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos we show that our approach preserves an average of 68.7% which is 25% more frames than that retained from the raw videos.Comment: 14 page

Simultaneous use of Individual and Joint Regularization Terms in Compressive Sensing: Joint Reconstruction of Multi-Channel Multi-Contrast MRI Acquisitions

Purpose: A time-efficient strategy to acquire high-quality multi-contrast images is to reconstruct undersampled data with joint regularization terms that leverage common information across contrasts. However, these terms can cause leakage of uncommon features among contrasts, compromising diagnostic utility. The goal of this study is to develop a compressive sensing method for multi-channel multi-contrast magnetic resonance imaging (MRI) that optimally utilizes shared information while preventing feature leakage. Theory: Joint regularization terms group sparsity and colour total variation are used to exploit common features across images while individual sparsity and total variation are also used to prevent leakage of distinct features across contrasts. The multi-channel multi-contrast reconstruction problem is solved via a fast algorithm based on Alternating Direction Method of Multipliers. Methods: The proposed method is compared against using only individual and only joint regularization terms in reconstruction. Comparisons were performed on single-channel simulated and multi-channel in-vivo datasets in terms of reconstruction quality and neuroradiologist reader scores. Results: The proposed method demonstrates rapid convergence and improved image quality for both simulated and in-vivo datasets. Furthermore, while reconstructions that solely use joint regularization terms are prone to leakage-of-features, the proposed method reliably avoids leakage via simultaneous use of joint and individual terms. Conclusion: The proposed compressive sensing method performs fast reconstruction of multi-channel multi-contrast MRI data with improved image quality. It offers reliability against feature leakage in joint reconstructions, thereby holding great promise for clinical use.Comment: 13 pages, 13 figures. Submitted for possible publicatio

Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation

Many modern computer vision and machine learning applications rely on solving difficult optimization problems that involve non-differentiable objective functions and constraints. The alternating direction method of multipliers (ADMM) is a widely used approach to solve such problems. Relaxed ADMM is a generalization of ADMM that often achieves better performance, but its efficiency depends strongly on algorithm parameters that must be chosen by an expert user. We propose an adaptive method that automatically tunes the key algorithm parameters to achieve optimal performance without user oversight. Inspired by recent work on adaptivity, the proposed adaptive relaxed ADMM (ARADMM) is derived by assuming a Barzilai-Borwein style linear gradient. A detailed convergence analysis of ARADMM is provided, and numerical results on several applications demonstrate fast practical convergence.Comment: CVPR 201