Convergence Properties of a Randomized Primal-Dual Algorithm with Applications to Parallel MRI

The Stochastic Primal-Dual Hybrid Gradient (SPDHG) was proposed by Chambolle et al. (2018) and is an efficient algorithm to solve some nonsmooth large-scale optimization problems. In this paper we prove its almost sure convergence for convex but not necessarily strongly convex functionals. We also look into its application to parallel Magnetic Resonance Imaging reconstruction in order to test performance of SPDHG. Our numerical results show that for a range of settings SPDHG converges significantly faster than its deterministic counterpart

On the convergence and sampling of randomized primal-dual algorithms and their application to parallel MRI reconstruction

Stochastic Primal-Dual Hybrid Gradient (SPDHG) is an algorithm to efficiently solve a wide class of nonsmooth large-scale optimization problems. In this paper we contribute to its theoretical foundations and prove its almost sure convergence for convex but neither necessarily strongly convex nor smooth functionals. We also prove its convergence for any sampling. In addition, we study SPDHG for parallel Magnetic Resonance Imaging reconstruction, where data from different coils are randomly selected at each iteration. We apply SPDHG using a wide range of random sampling methods and compare its performance across a range of settings, including mini-batch size and step size parameters. We show that the sampling can significantly affect the convergence speed of SPDHG and for many cases an optimal sampling can be identified

On the convergence and sampling of randomized primal-dual algorithms and their application to parallel MRI reconstruction

Stochastic Primal-Dual Hybrid Gradient (SPDHG) is an algorithm to efficiently solve a wide class of nonsmooth large-scale optimization problems. In this paper we contribute to its theoretical foundations and prove its almost sure convergence for convex but neither necessarily strongly convex nor smooth functionals. We also prove its convergence for any sampling. In addition, we study SPDHG for parallel Magnetic Resonance Imaging reconstruction, where data from different coils are randomly selected at each iteration. We apply SPDHG using a wide range of random sampling methods and compare its performance across a range of settings, including mini-batch size and step size parameters. We show that the sampling can significantly affect the convergence speed of SPDHG and for many cases an optimal sampling can be identified

Core Imaging Library - Part II:multichannel reconstruction for dynamic and spectral tomography

The newly developed core imaging library (CIL) is a flexible plug and play library for tomographic imaging with a specific focus on iterative reconstruction. CIL provides building blocks for tailored regularized reconstruction algorithms and explicitly supports multichannel tomographic data. In the first part of this two-part publication, we introduced the fundamentals of CIL. This paper focuses on applications of CIL for multichannel data, e.g. dynamic and spectral. We formalize different optimization problems for colour processing, dynamic and hyperspectral tomography and demonstrate CIL’s capabilities for designing state-of-the-art reconstruction methods through case studies and code snapshots

Motion estimation and correction for simultaneous PET/MR using SIRF and CIL

SIRF is a powerful PET/MR image reconstruction research tool for processing data and developing new algorithms. In this research, new developments to SIRF are presented, with focus on motion estimation and correction. SIRF's recent inclusion of the adjoint of the resampling operator allows gradient propagation through resampling, enabling the MCIR technique. Another enhancement enabled registering and resampling of complex images, suitable for MRI. Furthermore, SIRF's integration with the optimization library CIL enables the use of novel algorithms. Finally, SPM is now supported, in addition to NiftyReg, for registration. Results of MR and PET MCIR reconstructions are presented, using FISTA and PDHG, respectively. These demonstrate the advantages of incorporating motion correction and variational and structural priors. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'

Quantitative methods for the asymptotic study of homogeneous and non-homogeneous Markov processes

L'objet de cette thèse est l'étude de certaines propriétés analytiques et asymptotiques des processus de Markov, et de leurs applications à la méthode de Stein. Le point de vue considéré consiste à déployer des inégalités fonctionnelles pour majorer la distance entre lois de probabilité. La première partie porte sur l'étude asymptotique de processus de Markov inhomogènes en temps via des inégalités de type Poincaré, établies par l'analyse spectrale fine de l'opérateur de transition. On se place d'abord dans le cadre du théorème central limite, qui affirme que la somme renormalisée de variables aléatoires converge vers la mesure gaussienne, et l'étude est consacrée à l'obtention d'une borne à la Berry-Esseen permettant de quantifier cette convergence. La distance choisie est une quantité naturelle et encore non étudiée dans ce cadre, la distance du chi-2, complétant ainsi la littérature relative à d'autres distances (Kolmogorov, variation totale, Wasserstein). Toujours dans le contexte non-homogène, on s'intéresse ensuite à un processus peu mélangeant relié à un algorithme stochastique de recherche de médiane. Ce processus évolue par sauts de deux types (droite ou gauche), dont la taille et l'intensité dépendent du temps. Une majoration de la distance de Wasserstein d'ordre 1 entre la loi du processus et la mesure gaussienne est établie dans le cas où celle-ci est invariante sous la dynamique considérée, et étendue à des exemples où seule la normalité asymptotique est vérifiée. La seconde partie s'attache à l'étude des entrelacements entre processus de Markov (homogènes) et gradients, qu'on peut interpréter comme un raffinement du critère de Bakry-Emery, et leur application à la méthode de Stein, qui est un ensemble de techniques permettant de majorer la distance entre deux mesures de probabilité. On prouve l'existence de relations d'entrelacement du second ordre pour les processus de naissance-mort, allant ainsi plus loin que les relations du premier ordre connues. Ces relations sont mises à profit pour construire une méthode originale et universelle d'évaluation des facteurs de Stein relatifs aux mesures de probabilité discrètes, qui forment une composante essentielle de la méthode de Stein-Chen.The object of this thesis is the study of some analytical and asymptotic properties of Markov processes, and their applications to Stein's method. The point of view consists in the development of functional inequalities in order to obtain upper-bounds on the distance between probability distributions. The first part is devoted to the asymptotic study of time-inhomogeneous Markov processes through Poincaré-like inequalities, established by precise estimates on the spectrum of the transition operator. The first investigation takes place within the framework of the Central Limit Theorem, which states the convergence of the renormalized sum of random variables towards the normal distribution. It results in the statement of a Berry-Esseen bound allowing to quantify this convergence with respect to the chi-2 distance, a natural quantity which had not been investigated in this setting. It therefore extends similar results relative to other distances (Kolmogorov, total variation, Wasserstein). Keeping with the non-homogeneous framework, we consider a weakly mixing process linked to a stochastic algorithm for median approximation. This process evolves by jumps of two sorts (to the right or to the left) with time-dependent size and intensity. An upper-bound on the Wasserstein distance of order 1 between the marginal distribution of the process and the normal distribution is provided when the latter is invariant under the dynamic, and extended to examples where only the asymptotic normality stands. The second part concerns intertwining relations between (homogeneous) Markov processes and gradients, which can be seen as refinment of the Bakry-Emery criterion, and their application to Stein's method, a collection of techniques to estimate the distance between two probability distributions. Second order intertwinings for birth-death processes are stated, going one step further than the existing first order relations. These relations are then exploited to construct an original and universal method of evaluation of discrete Stein's factors, a key component of Stein-Chen's method

Méthodes quantitatives pour l'étude asymptotique de processus de Markov homogènes et non-homogènes

The object of this thesis is the study of some analytical and asymptotic properties of Markov processes, and their applications to Stein's method. The point of view consists in the development of functional inequalities in order to obtain upper-bounds on the distance between probability distributions. The first part is devoted to the asymptotic study of time-inhomogeneous Markov processes through Poincaré-like inequalities, established by precise estimates on the spectrum of the transition operator. The first investigation takes place within the framework of the Central Limit Theorem, which states the convergence of the renormalized sum of random variables towards the normal distribution. It results in the statement of a Berry-Esseen bound allowing to quantify this convergence with respect to the chi-2 distance, a natural quantity which had not been investigated in this setting. It therefore extends similar results relative to other distances (Kolmogorov, total variation, Wasserstein). Keeping with the non-homogeneous framework, we consider a weakly mixing process linked to a stochastic algorithm for median approximation. This process evolves by jumps of two sorts (to the right or to the left) with time-dependent size and intensity. An upper-bound on the Wasserstein distance of order 1 between the marginal distribution of the process and the normal distribution is provided when the latter is invariant under the dynamic, and extended to examples where only the asymptotic normality stands. The second part concerns intertwining relations between (homogeneous) Markov processes and gradients, which can be seen as refinment of the Bakry-Emery criterion, and their application to Stein's method, a collection of techniques to estimate the distance between two probability distributions. Second order intertwinings for birth-death processes are stated, going one step further than the existing first order relations. These relations are then exploited to construct an original and universal method of evaluation of discrete Stein's factors, a key component of Stein-Chen's method.L'objet de cette thèse est l'étude de certaines propriétés analytiques et asymptotiques des processus de Markov, et de leurs applications à la méthode de Stein. Le point de vue considéré consiste à déployer des inégalités fonctionnelles pour majorer la distance entre lois de probabilité. La première partie porte sur l'étude asymptotique de processus de Markov inhomogènes en temps via des inégalités de type Poincaré, établies par l'analyse spectrale fine de l'opérateur de transition. On se place d'abord dans le cadre du théorème central limite, qui affirme que la somme renormalisée de variables aléatoires converge vers la mesure gaussienne, et l'étude est consacrée à l'obtention d'une borne à la Berry-Esseen permettant de quantifier cette convergence. La distance choisie est une quantité naturelle et encore non étudiée dans ce cadre, la distance du chi-2, complétant ainsi la littérature relative à d'autres distances (Kolmogorov, variation totale, Wasserstein). Toujours dans le contexte non-homogène, on s'intéresse ensuite à un processus peu mélangeant relié à un algorithme stochastique de recherche de médiane. Ce processus évolue par sauts de deux types (droite ou gauche), dont la taille et l'intensité dépendent du temps. Une majoration de la distance de Wasserstein d'ordre 1 entre la loi du processus et la mesure gaussienne est établie dans le cas où celle-ci est invariante sous la dynamique considérée, et étendue à des exemples où seule la normalité asymptotique est vérifiée. La seconde partie s'attache à l'étude des entrelacements entre processus de Markov (homogènes) et gradients, qu'on peut interpréter comme un raffinement du critère de Bakry-Emery, et leur application à la méthode de Stein, qui est un ensemble de techniques permettant de majorer la distance entre deux mesures de probabilité. On prouve l'existence de relations d'entrelacement du second ordre pour les processus de naissance-mort, allant ainsi plus loin que les relations du premier ordre connues. Ces relations sont mises à profit pour construire une méthode originale et universelle d'évaluation des facteurs de Stein relatifs aux mesures de probabilité discrètes, qui forment une composante essentielle de la méthode de Stein-Chen

Berry-Esseen bounds for the χ^2 -distance in the Central Limit Theorem: a Markovian approach

This article presents a new proof of the rate of convergence to the normal distribution of sums of independent, identically distributed random variables in chi-square distance, which was also recently studied by Bobkov, Chistyakov and Götze. Our method consists of taking advantage of the underlying time non-homogeneous Markovian structure and studying the spectral properties of the non-reversible transition operator, which allows to find the optimal rate in the convergence above under matching moments assumptions. Our main assumption is that the random variables involved in the sum are independent and have polynomial density; interestingly, our approach allows to relax the identical distribution hypothesis

Bornes à la Berry-Esseen pour la distance du chi-deux dans le théorème central limite : une approche markovienne

Comments welcome.\\New version: relevant reference added, introduction modified.This article presents a new proof of the rate of convergence to the normal distribution of sums of independent, identically distributed random variables in chi-square distance, which was also recently studied in \cite{BobkovRenyi}. Our method consists of taking advantage of the underlying time non-homogeneous Markovian structure and studying the spectral properties of the non-reversible transition operator, which allows to find the optimal rate in the convergence above under matching moments assumptions. Our main assumption is that the random variables involved in the sum are independent and have polynomial density; interestingly, our approach allows to relax the identical distribution hypothesis