Decentralized Algorithms for Wasserstein Barycenters

In dieser Arbeit beschäftigen wir uns mit dem Wasserstein Baryzentrumproblem diskreter Wahrscheinlichkeitsmaße sowie mit dem population Wasserstein Baryzentrumproblem gegeben von a Fréchet Mittelwerts von der rechnerischen und statistischen Seiten. Der statistische Fokus liegt auf der Schätzung der Stichprobengröße von Maßen zur Berechnung einer Annäherung des Fréchet Mittelwerts (Baryzentrum) der Wahrscheinlichkeitsmaße mit einer bestimmten Genauigkeit. Für empirische Risikominimierung (ERM) wird auch die Frage der Regularisierung untersucht zusammen mit dem Vorschlag einer neuen Regularisierung, die zu den besseren Komplexitätsgrenzen im Vergleich zur quadratischen Regularisierung beiträgt. Der Rechenfokus liegt auf der Entwicklung von dezentralen Algorithmen zurBerechnung von Wasserstein Baryzentrum: duale Algorithmen und Sattelpunktalgorithmen. Die Motivation für duale Optimierungsmethoden ist geschlossene Formen für die duale Formulierung von entropie-regulierten Wasserstein Distanz und ihren Derivaten, während, die primale Formulierung nur in einigen Fällen einen Ausdruck in geschlossener Form hat, z.B. für Gaußsches Maß. Außerdem kann das duale Orakel, das den Gradienten der dualen Darstellung für die entropie-regulierte Wasserstein Distanz zurückgibt, zu einem günstigeren Preis berechnet werden als das primale Orakel, das den Gradienten der (entropie-regulierten) Wasserstein Distanz zurückgibt. Die Anzahl der dualen Orakel rufe ist in diesem Fall ebenfalls weniger, nämlich die Quadratwurzel der Anzahl der primalen Orakelrufe. Im Gegensatz zum primalen Zielfunktion, hat das duale Zielfunktion Lipschitz-stetig Gradient aufgrund der starken Konvexität regulierter Wasserstein Distanz. Außerdem untersuchen wir die Sattelpunktformulierung des (nicht regulierten) Wasserstein Baryzentrum, die zum Bilinearsattelpunktproblem führt. Dieser Ansatz ermöglicht es uns auch, optimale Komplexitätsgrenzen zu erhalten, und kann einfach in einer dezentralen Weise präsentiert werden.In this thesis, we consider the Wasserstein barycenter problem of discrete probability measures as well as the population Wasserstein barycenter problem given by a Fréchet mean from computational and statistical sides. The statistical focus is estimating the sample size of measures needed to calculate an approximation of a Fréchet mean (barycenter) of probability distributions with a given precision. For empirical risk minimization approaches, the question of the regularization is also studied along with proposing a new regularization which contributes to the better complexity bounds in comparison with the quadratic regularization. The computational focus is developing decentralized algorithms for calculating Wasserstein barycenters: dual algorithms and saddle point algorithms. The motivation for dual approaches is closed-forms for the dual formulation of entropy-regularized Wasserstein distances and their derivatives, whereas the primal formulation has a closed-form expression only in some cases, e.g., for Gaussian measures.Moreover, the dual oracle returning the gradient of the dual representation forentropy-regularized Wasserstein distance can be computed for a cheaper price in comparison with the primal oracle returning the gradient of the (entropy-regularized) Wasserstein distance. The number of dual oracle calls in this case will be also less, i.e., the square root of the number of primal oracle calls. Furthermore, in contrast to the primal objective, the dual objective has Lipschitz continuous gradient due to the strong convexity of regularized Wasserstein distances. Moreover, we study saddle-point formulation of the non-regularized Wasserstein barycenter problem which leads to the bilinear saddle-point problem. This approach also allows us to get optimal complexity bounds and it can be easily presented in a decentralized setup

On Primal-Dual Approach for Distributed Stochastic Convex Optimization over Networks

We introduce a primal-dual stochastic gradient oracle method for distributed convex optimization problems over networks. We show that the proposed method is optimal in terms of communication steps. Additionally, we propose a new analysis method for the rate of convergence in terms of duality gap and probability of large deviations. This analysis is based on a new technique that allows to bound the distance between the iteration sequence and the optimal point. By the proper choice of batch size, we can guarantee that this distance equals (up to a constant) to the distance between the starting point and the solution

Remote refocusing light-sheet fluorescence microscopy for high-speed 2D and 3D imaging of calcium dynamics in cardiomyocytes

The high prevalence and poor prognosis of heart failure are two key drivers for research into cardiac electrophysiology and regeneration. Dyssynchrony in calcium release and loss of structural organization within individual cardiomyocytes (CM) has been linked to reduced contractile strength and arrhythmia. Correlating calcium dynamics and cell microstructure requires multidimensional imaging with high spatiotemporal resolution. In light-sheet fluorescence microscopy (LSFM), selective plane illumination enables fast optically sectioned imaging with lower phototoxicity, making it suitable for imaging subcellular dynamics. In this work, a custom remote refocusing LSFM system is applied to studying calcium dynamics in isolated CM, cardiac cell cultures and tissue slices. The spatial resolution of the LSFM system was modelled and experimentally characterized. Simulation of the illumination path in Zemax was used to estimate the light-sheet beam waist and confocal parameter. Automated MATLAB-based image analysis was used to quantify the optical sectioning and the 3D point spread function using Gaussian fitting of bead image intensity distributions. The results demonstrated improved and more uniform axial resolution and optical sectioning with the tighter focused beam used for axially swept light-sheet microscopy. High-speed dual-channel LSFM was used for 2D imaging of calcium dynamics in correlation with the t-tubule structure in left and right ventricle cardiomyocytes at 395 fps. The high spatio-temporal resolution enabled the characterization of calcium sparks. The use of para-nitro-blebbistatin (NBleb), a non-phototoxic, low fluorescence contraction uncoupler, allowed 2D-mapping of the spatial dyssynchrony of calcium transient development across the cell. Finally, aberration-free remote refocusing was used for high-speed volumetric imaging of calcium dynamics in human induced pluripotent stem-cell derived cardiomyocytes (hiPSC-CM) and their co-culture with adult-CM. 3D-imaging at up to 8 Hz demonstrated the synchronization of calcium transients in co-culture, with increased coupling with longer co-culture duration, uninhibited by motion uncoupling with NBleb.Open Acces

Statistical investigation of local variations of the horizontal component of a geomagnetic vector in quiet days /summer season/

Random processes in daily variations of geomagnetic vector during summertime quiet days by statistical investigation

Optimal decentralized distributed algorithms for stochastic convex optimization

We consider stochastic convex optimization problems with affine constraints and develop several methods using either primal or dual approach to solve it. In the primal case we use special penalization technique to make the initial problem more convenient for using optimization methods. We propose algorithms to solve it based on Similar Triangles Method with Inexact Proximal Step for the convex smooth and strongly convex smooth objective functions and methods based on Gradient Sliding algorithm to solve the same problems in the non-smooth case. We prove the convergence guarantees in smooth convex case with deterministic first-order oracle. We propose and analyze three novel methods to handle stochastic convex optimization problems with affine constraints: SPDSTM, R-RRMA-AC-SA and SSTM_sc. All methods use stochastic dual oracle. SPDSTM is the stochastic primal-dual modification of STM and it is applied for the dual problem when the primal functional is strongly convex and Lipschitz continuous on some ball. R-RRMA-AC-SA is an accelerated stochastic method based on the restarts of RRMA-AC-SA and SSTM_sc is just stochastic STM for strongly convex problems. Both methods are applied to the dual problem when the primal functional is strongly convex, smooth and Lipschitz continuous on some ball and use stochastic dual first-order oracle. We develop convergence analysis for these methods for the unbiased and biased oracles respectively. Finally, we apply all aforementioned results and approaches to solve decentralized distributed optimization problem and discuss optimality of the obtained results in terms of communication rounds and number of oracle calls per node

Suppressing magnetization exchange effects in stimulated-echo diffusion experiments

AbstractExchange of nuclear magnetization between spin pools, either by chemical exchange or by cross-relaxation or both, has a significant influence on the signal attenuation in stimulated-echo-type pulsed field gradient experiments. Hence, in such cases the obtained molecular self-diffusion coefficients can carry a large systematic error. We propose a modified stimulated echo pulse sequence that contains T2-filters during the z-magnetization store period. We demonstrate, using a common theoretical description for chemical exchange and cross-relaxation, that these filters suppress the effects of exchange on the diffusional decay in that frequent case where one of the participating spin pools is immobile and exhibits a short T2. We demonstrate the performance of this experiment in an agarose/water gel. We posit that this new experiment has advantages over other approaches hitherto used, such as that consisting of measuring separately the magnetization exchange rate, if suitable by Goldman–Shen type experiments, and then correcting for exchange effects within the framework of a two-site exchange model. We also propose experiments based on selective decoupling and applicable in systems with no large T2 difference between the different spin pools

High-speed 2D light-sheet fluorescence microscopy enables quantification of spatially varying calcium dynamics in ventricular cardiomyocytes

Introduction: Reduced synchrony of calcium release and t-tubule structure organization in individual cardiomyocytes has been linked to loss of contractile strength and arrhythmia. Compared to confocal scanning techniques widely used for imaging calcium dynamics in cardiac muscle cells, light-sheet fluorescence microscopy enables fast acquisition of a 2D plane in the sample with low phototoxicity. Methods: A custom light-sheet fluorescence microscope was used to achieve dual-channel 2D timelapse imaging of calcium and the sarcolemma, enabling calcium sparks and transients in left and right ventricle cardiomyocytes to be correlated with the cell microstructure. Imaging electrically stimulated dual-labelled cardiomyocytes immobilized with para-nitroblebbistatin, a non-phototoxic, low fluorescence contraction uncoupler, with sub-micron resolution at 395 fps over a 38 μm × 170 µm FOV allowed characterization of calcium spark morphology and 2D mapping of the calcium transient time-to-half-maximum across the cell. Results: Blinded analysis of the data revealed sparks with greater amplitude in left ventricle myocytes. The time for the calcium transient to reach half-maximum amplitude in the central part of the cell was found to be, on average, 2 ms shorter than at the cell ends. Sparks co-localized with t-tubules were found to have significantly longer duration, larger area and spark mass than those further away from t-tubules. Conclusion: The high spatiotemporal resolution of the microscope and automated image-analysis enabled detailed 2D mapping and quantification of calcium dynamics of n = 60 myocytes, with the findings demonstrating multi-level spatial variation of calcium dynamics across the cell, supporting the dependence of synchrony and characteristics of calcium release on the underlying t-tubule structure

Inexact Model: A Framework for Optimization and Variational Inequalities

In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal method for variational inequalities with composite structure. This method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. We also generalize our framework for strongly convex objectives and strongly monotone variational inequalities.Comment: 41 page

Gradient Free Methods for Non-Smooth Convex Optimization with Heavy Tails on Convex Compact

Optimization problems, in which only the realization of a function or a zeroth-order oracle is available, have many applications in practice. An effective method for solving such problems is the approximation of the gradient using sampling and finite differences of the function values. However, some noise can be present in the zeroth-order oracle not allowing the exact evaluation of the function value, and this noise can be stochastic or adversarial. In this paper, we propose and study new easy-to-implement algorithms that are optimal in terms of the number of oracle calls for solving non-smooth optimization problems on a convex compact set with heavy-tailed stochastic noise (random noise has $(1+\kappa)$-th bounded moment) and adversarial noise. The first algorithm is based on the heavy-tail-resistant mirror descent and uses special transformation functions that allow controlling the tails of the noise distribution. The second algorithm is based on the gradient clipping technique. The paper provides proof of algorithms' convergence results in terms of high probability and in terms of expectation when a convex function is minimized. For functions satisfying a $r$-growth condition, a faster algorithm is proposed using the restart technique. Particular attention is paid to the question of how large the adversarial noise can be so that the optimality and convergence of the algorithms is guaranteed

Bregman Proximal Method for Efficient Communications under Similarity

We propose a novel distributed method for monotone variational inequalities and convex-concave saddle point problems arising in various machine learning applications such as game theory and adversarial training. By exploiting \textit{similarity} our algorithm overcomes communication bottleneck which is a major issue in distributed optimization. The proposed algorithm enjoys optimal communication complexity of $\delta/\epsilon$, where $\epsilon$ measures the non-optimality gap function, and $\delta$ is a parameter of similarity. All the existing distributed algorithms achieving this bound essentially utilize the Euclidean setup. In contrast to them, our algorithm is built upon Bregman proximal maps and it is compatible with an arbitrary Bregman divergence. Thanks to this, it has more flexibility to fit the problem geometry than algorithms with the Euclidean setup. Thereby the proposed method bridges the gap between the Euclidean and non-Euclidean setting. By using the restart technique, we extend our algorithm to variational inequalities with $\mu$-strongly monotone operator, resulting in optimal communication complexity of $\delta/\mu$ (up to a logarithmic factor). Our theoretical results are confirmed by numerical experiments on a stochastic matrix game.Comment: 14 page