FastDVDnet: Towards Real-Time Video Denoising Without Explicit Motion Estimation

In this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. Until recently, video denoising with neural networks had been a largely under explored domain, and existing methods could not compete with the performance of the best patch-based methods. The approach we introduce in this paper, called FastDVDnet, shows similar or better performance than other state-of-the-art competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as fast run-times, and the ability to handle a wide range of noise levels with a single network model. The characteristics of its architecture make it possible to avoid using a costly motion compensation stage while achieving excellent performance. The combination between its denoising performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics

DVDnet: A Fast Network for Deep Video Denoising

In this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. Previous neural network based approaches to video denoising have been unsuccessful as their performance cannot compete with the performance of patch-based methods. However, our approach outperforms other patch-based competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as a small memory footprint, and the ability to handle a wide range of noise levels with a single network model. The combination between its denoising performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics. The experiments show that our algorithm compares favorably to other state-of-art methods. Video examples, code and models are publicly available at \url{https://github.com/m-tassano/dvdnet}

Deep Model-Based Super-Resolution with Non-uniform Blur

We propose a state-of-the-art method for super-resolution with non-uniform blur. Single-image super-resolution methods seek to restore a high-resolution image from blurred, subsampled, and noisy measurements. Despite their impressive performance, existing techniques usually assume a uniform blur kernel. Hence, these techniques do not generalize well to the more general case of non-uniform blur. Instead, in this paper, we address the more realistic and computationally challenging case of spatially-varying blur. To this end, we first propose a fast deep plug-and-play algorithm, based on linearized ADMM splitting techniques, which can solve the super-resolution problem with spatially-varying blur. Second, we unfold our iterative algorithm into a single network and train it end-to-end. In this way, we overcome the intricacy of manually tuning the parameters involved in the optimization scheme. Our algorithm presents remarkable performance and generalizes well after a single training to a large family of spatially-varying blur kernels, noise levels and scale factors

Approches rapides et embarquables pour le débruitage d’images et de vidéos

Cette thèse porte sur des problèmes de débruitage d'images et de vidéos. Elle mêle des approches traditionnelles en traitement d'images, dites ``par patchs'', et des méthodes plus récentes basées sur des architectures par réseaux de neurones. Les méthodes développées tiennent compte des contraintes liées aux systèmes embarqués, elles doivent donc être à la fois rapides et peu gourmandes. Dans une première partie, on explore plusieurs directions pour rendre les méthodes par patchs suffisamment efficaces et pertinentes pour satisfaire ces contraintes. La seconde partie de la thèse se concentre sur les progrès récents obtenus grâce aux approches par réseaux convolutionnels profonds, et mène à deux nouvelles approches de débruitage vidéo qui s'avèrent beaucoup plus rapides (parfois de plusieurs ordres de grandeur) que les meilleures méthodes de l'état de l'art, tout en les égalant du point de vue de la qualité des résultats. Enfin, une troisième partie présente un travail exploratoire sur la correction de flou. Comme dans les parties précédentes, nous essaierons de faire le lien entre les approches classiques de traitement d’image (transformée de Fourier) et les approches par réseaux de neurones. Nous nous efforcerons de comprendre et d’expliquer les mécanismes et les bénéfices de chaque méthodologie. En particulier, nous présenterons un algorithme intermédiaire (kernel prediction) qui fait le lien entre cesThis thesis approaches the problems of image and video denoising from the perspective of both “traditional” patch-based methods and CNN architectures, paying close attention to the constraints imposed by embedded systems. In its first part, several relevant and efficient techniques are explored to improve the performance of simple patch-based methods. The second part approaches these problems from a more recent optic: architectures based on convolutional neural networks. This work has led to two different innovative, state-of-the-art video denoising algorithms which compare favorably to and run orders of magnitude faster than other state-of-the-art methods. Lastly, the third final part presents an exploratory work on deblurring. As in the precedent parts, this work links traditional image processing algorithms (based on the Fourier transform) with neural architectures. In particular, this part proposes a deblurring approach based on kernel prediction networks and multi-image deblurring methods

DVDnet : Un réseau profond rapide pour le débruitage vidéo

International audienceIn this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. While neural network based approaches are nowadays state-of-the-art in image denoising, these methods have been unsuccessful for video denoising as their performance cannot compete with the performance of patch-based methods. However, our approach outperforms other patch-based competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as a small memory footprint, and the ability to handle a wide range of noise levels with a single network model. The combination between its denoising performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics. The experiments show that our algorithm compares favorably to other state-of-art methods. Video examples, code and models are publicly available at \url{https://github.com/m-tassano/dvdnet}.Dans cet article, nous proposons un algorithme de débruitage vidéo basé sur une architecture de réseau convolutif. Alors que ces réseaux ont déjà montré toute leur efficacité pour le débruitage d'image, ils ne permettaient pas encore de rivaliser avec les méthodes plus classiques de l'état de l'art (comme celles utilisant des patchs) pour le débruitage vidéo. Notre approche permet de dépasser ces limites tout en assurant des temps de calcul nettement plus courts que les méthodes de débruitage vidéo classiques. Contrairement à d'autres approches par réseaux de neurones existants, notre algorithme présente plusieurs propriétés souhaitables telles qu'une faible empreinte mémoire, et la capacité à gérer une large gamme de niveaux de bruit avec un seul réseau, ce qui le rend très attractif pour les applications pratiques. Nous comparons notre méthode avec différents algorithmes de l'état de l'art, à la fois visuellement et pour des mesures de qualité. Les expériences montrent que notre algorithme obtient des performances supérieures ou identiques à d'autres méthodes de l'état de l'art, pour un coût de calcul nettement plus faible. Différents exemples de vidéos, les codes et l'architecture utilisés sont disponibles à l'adresse https://github.com/m-tassano/ dvdnet. Abstract-In this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. While neural network based approaches are nowadays state-of-the-art in image denoising, these methods have been unsuccessful for video denoising as their performance cannot compete with the performance of patch-based methods. However, our approach outperforms other patch-based competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as a small memory footprint, and the ability to handle a wide range of noise levels with a single network model. The combination between its denoising performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics. The experiments show that our algorithm compares favorably to other state-of-art methods. Video examples, code and models are publicly available at https://github.com/m-tassano/dvdnet

A priori Plug-and-Play profond pour la restauration de vidéos

International audienceThis paper presents a method for restoring digital videos via a Plug-and-Play (PnP) approach. The method consists in using a deep convolutional denoising network in place of the proximal operator of the prior in an alternating optimization scheme. This way, a network trained once for denoising can be repurposed for other restoration tasks such as interpolation or deconvolution. Our experiments show the benefit of using a network specifically designed for video denoising, as it reaches better restoration performance and better temporal stability than a single image denoising network with similar denoising performance using the same PnP formulation.Cet article présente une méthode de restauration de vidéos numériques via une approche Plug-and-Play (PnP). La méthode consiste à utiliser un réseau convolutionnel profond de débruitage pour remplacer l'opérateur proximal de l'a priori dans un schéma d'optimisation alterné. Elle permet de réutiliser un réseau uniquement entrainé pour du débruitage pour d'autres tâches de restauration comme l'interpolation ou le déflouage. Nos expériences montrent l'intérêt d'utiliser un réseau spécifiquement conc ¸u pour le débruitage vidéo qui, avec la même formulation PnP, permet d'atteindre de meilleures performances de restauration et une meilleure stabilité temporelle qu'un réseau mono-image aux performances de débruitage similaires

An Analysis and Implementation of the FFDNet Image Denoising Method

International audienc

DVDNET: A FAST NETWORK FOR DEEP VIDEO DENOISING

International audienceIn this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. Previous neural network based approaches to video denois-ing have been unsuccessful as their performance cannot compete with the performance of patch-based methods. However, our approach outperforms other patch-based competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as a small memory footprint, and the ability to handle a wide range of noise levels with a single network model. The combination between its denois-ing performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics. The experiments show that our algorithm compares favorably to other state-of-art methods. Video examples, code and models are publicly available at https://github.com/ m-tassano/dvdnet

DVDnet : Un réseau profond rapide pour le débruitage vidéo

International audienceIn this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. While neural network based approaches are nowadays state-of-the-art in image denoising, these methods have been unsuccessful for video denoising as their performance cannot compete with the performance of patch-based methods. However, our approach outperforms other patch-based competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as a small memory footprint, and the ability to handle a wide range of noise levels with a single network model. The combination between its denoising performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics. The experiments show that our algorithm compares favorably to other state-of-art methods. Video examples, code and models are publicly available at \url{https://github.com/m-tassano/dvdnet}.Dans cet article, nous proposons un algorithme de débruitage vidéo basé sur une architecture de réseau convolutif. Alors que ces réseaux ont déjà montré toute leur efficacité pour le débruitage d'image, ils ne permettaient pas encore de rivaliser avec les méthodes plus classiques de l'état de l'art (comme celles utilisant des patchs) pour le débruitage vidéo. Notre approche permet de dépasser ces limites tout en assurant des temps de calcul nettement plus courts que les méthodes de débruitage vidéo classiques. Contrairement à d'autres approches par réseaux de neurones existants, notre algorithme présente plusieurs propriétés souhaitables telles qu'une faible empreinte mémoire, et la capacité à gérer une large gamme de niveaux de bruit avec un seul réseau, ce qui le rend très attractif pour les applications pratiques. Nous comparons notre méthode avec différents algorithmes de l'état de l'art, à la fois visuellement et pour des mesures de qualité. Les expériences montrent que notre algorithme obtient des performances supérieures ou identiques à d'autres méthodes de l'état de l'art, pour un coût de calcul nettement plus faible. Différents exemples de vidéos, les codes et l'architecture utilisés sont disponibles à l'adresse https://github.com/m-tassano/ dvdnet. Abstract-In this paper, we propose a state-of-the-art video denoising algorithm based on a convolutional neural network architecture. While neural network based approaches are nowadays state-of-the-art in image denoising, these methods have been unsuccessful for video denoising as their performance cannot compete with the performance of patch-based methods. However, our approach outperforms other patch-based competitors with significantly lower computing times. In contrast to other existing neural network denoisers, our algorithm exhibits several desirable properties such as a small memory footprint, and the ability to handle a wide range of noise levels with a single network model. The combination between its denoising performance and lower computational load makes this algorithm attractive for practical denoising applications. We compare our method with different state-of-art algorithms, both visually and with respect to objective quality metrics. The experiments show that our algorithm compares favorably to other state-of-art methods. Video examples, code and models are publicly available at https://github.com/m-tassano/dvdnet