Patch-based methods for variational image processing problems

Image Processing problems are notoriously difficult. To name a few of these difficulties, they are usually ill-posed, involve a huge number of unknowns (from one to several per pixel!), and images cannot be considered as the linear superposition of a few physical sources as they contain many different scales and non-linearities. However, if one considers instead of images as a whole small blocks (or patches) inside the pictures, many of these hurdles vanish and problems become much easier to solve, at the cost of increasing again the dimensionality of the data to process. Following the seminal NL-means algorithm in 2005-2006, methods that consider only the visual correlation between patches and ignore their spatial relationship are called non-local methods. While powerful, it is an arduous task to define non-local methods without using heuristic formulations or complex mathematical frameworks. On the other hand, another powerful property has brought global image processing algorithms one step further: it is the sparsity of images in well chosen representation basis. However, this property is difficult to embed naturally in non-local methods, yielding algorithms that are usually inefficient or circonvoluted. In this thesis, we explore alternative approaches to non-locality, with the goals of i) developing universal approaches that can handle local and non-local constraints and ii) leveraging the qualities of both non-locality and sparsity. For the first point, we will see that embedding the patches of an image into a graph-based framework can yield a simple algorithm that can switch from local to non-local diffusion, which we will apply to the problem of large area image inpainting. For the second point, we will first study a fast patch preselection process that is able to group patches according to their visual content. This preselection operator will then serve as input to a social sparsity enforcing operator that will create sparse groups of jointly sparse patches, thus exploiting all the redundancies present in the data, in a simple mathematical framework. Finally, we will study the problem of reconstructing plausible patches from a few binarized measurements. We will show that this task can be achieved in the case of popular binarized image keypoints descriptors, thus demonstrating a potential privacy issue in mobile visual recognition applications, but also opening a promising way to the design and the construction of a new generation of smart cameras

Towards unifying diffusion and exemplar-based inpainting

A novel framework for image inpainting is proposed, relying on graph-based diffusion processes. Depending on the construction of the graph, both flow-based and exemplar-based inpainting methods can be implemented by the same equations, hence providing a unique framework for geometry and texture-based approaches to inpainting. Furthermore, the use of a variational framework allows to overcome the usual sensitivity of exemplar-based methods to the heuristic issues by providing an evolution criterion. The use of graphs also makes our framework more flexible than former non-local variational formulations, allowing for example to mix spatial and non-local constraints and to use a data term to provide smoother blending between the initial image and the result

FULLY NON-LOCAL SUPER-RESOLUTION VIA SPECTRAL HASHING

Super-resolution is the task of creating an high resolution image from a low resolution input sequence. To overcome the difficul- ties of fine image registration, several methods have been proposed exploiting the non-local intuition, i.e. any datapoint can contribute to the final result if it is relevant. These algorithms however limit in practice the search region for relevant points in order to lower the corresponding computational cost. Furthermore, they define the non-local relations in the high resolution space, where the true im- ages are unknown. In this work, we introduce the use of spectral hashing to effi- ciently compute fully non-local neighbors. We also restate the super- resolution functional using fixed weights in the low resolution space, allowing us to use resolution schemes that avoid many artifacts

Beyond Bits: Reconstructing Images from Local Binary Descriptors

Local Binary Descriptors (LBDs) are good at matching image parts, but how much information is actually carried? Surprisingly, this question is usually ignored and replaced by a comparison of matching performances. In this paper, we directly address it by trying to reconstruct plausible images from different LBDs such as BRIEF and FREAK. Using an inverse problem framework, we show that this task is achievable with only the information in the descriptors, excluding the need of additional data. Hence, our results represent a novel justification for the performance of LBDs. Furthermore, since plausible images can be inferred using only these simple measurements, this emphasizes the concerns about privacy and secrecy of image keypoints raised by [Weinzaepfel 2011], that could have an important impact on public applications of image matching

Tracking and Structure from Motion

Dense three-dimensional reconstruction of a scene from images is a very challenging task. In the structure from motion approach one of the key points is to compute depth maps which contain the distance of objects in the scene to a moving camera. Usually, this is achieved by finding correspondences in successive images and computing the distance by means of epipolar geometry. In this Master's thesis, a variational framework to solve the depth from motion problem for planar image sequences is proposed. Camera ego-motion estimation equations are derived and combined with the depth from motion estimation in a single algorithm. The method is successfully tested on synthetic images for general camera translation. Since it does not depend on the correspondance problem and because it is highly parallelizable, it is well adapted for real-time implementation. Further work in this thesis include a review of general variational methods in image processing, and in particular TV-L1 optical flow as well as its real-time implementation on the graphics processing unit

FAST TV-L1 OPTICAL FLOW FOR INTERACTIVITY

Vision is a natural tool for human-computer interaction, since it pro- vides visual feedback to the user and mimics some human behaviors. It requires however the fast and robust computation of motion primi- tives, which remains a difficult problem. In this work, we propose to apply some recent mathematical results about convex optimization to the TV-L1 optical flow problem. At the cost of a small smoothing of the Total Variation (TV), the convergence speed of the numerical scheme is improved, leading to earlier termination. Furthermore, we successfully implement our algorithm on GPU for realtime perfor- mance using the OpenCL framework.We demonstrate the potential of our optical flow by using it as primary sensor in a remotely con- trolled image browsing software

Impact of safety-related dose reductions or discontinuations on sustained virologic response in HCV-infected patients: Results from the GUARD-C Cohort

BACKGROUND: Despite the introduction of direct-acting antiviral agents for chronic hepatitis C virus (HCV) infection, peginterferon alfa/ribavirin remains relevant in many resource-constrained settings. The non-randomized GUARD-C cohort investigated baseline predictors of safety-related dose reductions or discontinuations (sr-RD) and their impact on sustained virologic response (SVR) in patients receiving peginterferon alfa/ribavirin in routine practice. METHODS: A total of 3181 HCV-mono-infected treatment-naive patients were assigned to 24 or 48 weeks of peginterferon alfa/ribavirin by their physician. Patients were categorized by time-to-first sr-RD (Week 4/12). Detailed analyses of the impact of sr-RD on SVR24 (HCV RNA <50 IU/mL) were conducted in 951 Caucasian, noncirrhotic genotype (G)1 patients assigned to peginterferon alfa-2a/ribavirin for 48 weeks. The probability of SVR24 was identified by a baseline scoring system (range: 0-9 points) on which scores of 5 to 9 and <5 represent high and low probability of SVR24, respectively. RESULTS: SVR24 rates were 46.1% (754/1634), 77.1% (279/362), 68.0% (514/756), and 51.3% (203/396), respectively, in G1, 2, 3, and 4 patients. Overall, 16.9% and 21.8% patients experienced 651 sr-RD for peginterferon alfa and ribavirin, respectively. Among Caucasian noncirrhotic G1 patients: female sex, lower body mass index, pre-existing cardiovascular/pulmonary disease, and low hematological indices were prognostic factors of sr-RD; SVR24 was lower in patients with 651 vs. no sr-RD by Week 4 (37.9% vs. 54.4%; P = 0.0046) and Week 12 (41.7% vs. 55.3%; P = 0.0016); sr-RD by Week 4/12 significantly reduced SVR24 in patients with scores <5 but not 655. CONCLUSIONS: In conclusion, sr-RD to peginterferon alfa-2a/ribavirin significantly impacts on SVR24 rates in treatment-naive G1 noncirrhotic Caucasian patients. Baseline characteristics can help select patients with a high probability of SVR24 and a low probability of sr-RD with peginterferon alfa-2a/ribavirin

Elective cancer surgery in COVID-19-free surgical pathways during the SARS-CoV-2 pandemic: An international, multicenter, comparative cohort study

PURPOSE As cancer surgery restarts after the first COVID-19 wave, health care providers urgently require data to determine where elective surgery is best performed. This study aimed to determine whether COVID-19–free surgical pathways were associated with lower postoperative pulmonary complication rates compared with hospitals with no defined pathway. PATIENTS AND METHODS This international, multicenter cohort study included patients who underwent elective surgery for 10 solid cancer types without preoperative suspicion of SARS-CoV-2. Participating hospitals included patients from local emergence of SARS-CoV-2 until April 19, 2020. At the time of surgery, hospitals were defined as having a COVID-19–free surgical pathway (complete segregation of the operating theater, critical care, and inpatient ward areas) or no defined pathway (incomplete or no segregation, areas shared with patients with COVID-19). The primary outcome was 30-day postoperative pulmonary complications (pneumonia, acute respiratory distress syndrome, unexpected ventilation). RESULTS Of 9,171 patients from 447 hospitals in 55 countries, 2,481 were operated on in COVID-19–free surgical pathways. Patients who underwent surgery within COVID-19–free surgical pathways were younger with fewer comorbidities than those in hospitals with no defined pathway but with similar proportions of major surgery. After adjustment, pulmonary complication rates were lower with COVID-19–free surgical pathways (2.2% v 4.9%; adjusted odds ratio [aOR], 0.62; 95% CI, 0.44 to 0.86). This was consistent in sensitivity analyses for low-risk patients (American Society of Anesthesiologists grade 1/2), propensity score–matched models, and patients with negative SARS-CoV-2 preoperative tests. The postoperative SARS-CoV-2 infection rate was also lower in COVID-19–free surgical pathways (2.1% v 3.6%; aOR, 0.53; 95% CI, 0.36 to 0.76). CONCLUSION Within available resources, dedicated COVID-19–free surgical pathways should be established to provide safe elective cancer surgery during current and before future SARS-CoV-2 outbreaks