A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization

We focus on decentralized stochastic non-convex optimization, where $n$ agents work together to optimize a composite objective function which is a sum of a smooth term and a non-smooth convex term. To solve this problem, we propose two single-time scale algorithms: Prox-DASA and Prox-DASA-GT. These algorithms can find $\epsilon$-stationary points in $\mathcal{O}(n^{-1}\epsilon^{-2})$ iterations using constant batch sizes (i.e., $\mathcal{O}(1)$). Unlike prior work, our algorithms achieve comparable complexity without requiring large batch sizes, more complex per-iteration operations (such as double loops), or stronger assumptions. Our theoretical findings are supported by extensive numerical experiments, which demonstrate the superiority of our algorithms over previous approaches. Our code is available at https://github.com/xuxingc/ProxDASA.Comment: UAI 202

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow

Multi-objective optimization (MOO) aims to optimize multiple, possibly conflicting objectives with widespread applications. We introduce a novel interacting particle method for MOO inspired by molecular dynamics simulations. Our approach combines overdamped Langevin and birth-death dynamics, incorporating a "dominance potential" to steer particles toward global Pareto optimality. In contrast to previous methods, our method is able to relocate dominated particles, making it particularly adept at managing Pareto fronts of complicated geometries. Our method is also theoretically grounded as a Wasserstein-Fisher-Rao gradient flow with convergence guarantees. Extensive experiments confirm that our approach outperforms state-of-the-art methods on challenging synthetic and real-world datasets

The complex genetic landscape of familial MDS and AML reveals pathogenic germline variants.

The inclusion of familial myeloid malignancies as a separate disease entity in the revised WHO classification has renewed efforts to improve the recognition and management of this group of at risk individuals. Here we report a cohort of 86 acute myeloid leukemia (AML) and myelodysplastic syndrome (MDS) families with 49 harboring germline variants in 16 previously defined loci (57%). Whole exome sequencing in a further 37 uncharacterized families (43%) allowed us to rationalize 65 new candidate loci, including genes mutated in rare hematological syndromes (ADA, GP6, IL17RA, PRF1 and SEC23B), reported in prior MDS/AML or inherited bone marrow failure series (DNAH9, NAPRT1 and SH2B3) or variants at novel loci (DHX34) that appear specific to inherited forms of myeloid malignancies. Altogether, our series of MDS/AML families offer novel insights into the etiology of myeloid malignancies and provide a framework to prioritize variants for inclusion into routine diagnostics and patient management

Source-specific biomarkers as proxies for Arctic and Antarctic sea ice

Over the last decade or so, certain source-specific C-25 highly branched isoprenoid (HBI) lipid biomarkers have emerged as useful proxies for Arctic and Antarctic sea ice. Thus, IP25 (Ice proxy with 25 carbon atoms) and IPSO25 (Ice proxy for the Southern Ocean with 25 carbon atoms) represent binary measures of past seasonal sea ice in the Arctic and Antarctic, respectively. A further tri-unsaturated HBI (generally referred to as HBI III) appears to provide proxy evidence for the region of open water found adjacent to sea ice (i.e. the marginal ice zone (MIZ)) in both polar regions. This review provides an update on current knowledge pertaining to each proxy. The first section focuses on describing those studies that have aimed to establish the underlying features of each proxy, including source identification and spatial distribution characteristics. The second section presents some important analytical considerations pertinent to the accurate identification and quantification of HBI biomarkers. The third section describes how each HBI proxy is normally interpreted within the sedimentary record for palaeo sea ice reconstruction purposes. This includes the interpretation of individual and combined biomarker profiles such as the PIP25 index and multivariate decision tree models. A summary of all previous palaeo sea ice reconstructions based on HBIs is also given, which includes examples that clarify or reinforce our understanding of the individual or combined biomarker signatures. Some knowledge gaps and areas for future research are also briefly described

Towards Computational and Sample Efficiency in Stochastic Optimization

Stochastic optimization is a crucial tool in machine learning, statistics, and operations research, and developing efficient algorithms for stochastic optimization is of great importance. This dissertation focuses on stochastic composite optimization, where the objective function is composed of a smooth expected value function and a deterministic non-smooth component. We propose a class of algorithms called proximal averaged stochastic approximation (Prox-ASA), which estimates the gradient using a moving average approach. We prove the theoretical convergence of Prox-ASA to a first-order stationary point in different settings, including expectation, high probability, and almost surely asymptotically. In addition, we show that Prox-ASA can be applied to address decentralized problems and stochastic compositional optimization problems. For the non-convex constrained setting with expensive projection, we propose a novel class of conditional gradient based algorithms for solving stochastic multi-level compositional optimization problems that obtain the same sample complexity of the single-level setting under standard assumptions. Lastly, we demonstrate that by leveraging interpolation-like conditions satisfied by overparameterized models, we can improve the oracle complexities of stochastic conditional gradient methods

A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimization

We propose a projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization, where the objective function is a nested composition of $T$ functions and the constraint set is a closed convex set. Our algorithm assumes access to noisy evaluations of the functions and their gradients, through a stochastic first-order oracle satisfying certain standard unbiasedness and second moment assumptions. We show that the number of calls to the stochastic first-order oracle and the linear-minimization oracle required by the proposed algorithm, to obtain an $\epsilon$-stationary solution, are of order $\mathcal{O}_T(\epsilon^{-2})$ and $\mathcal{O}_T(\epsilon^{-3})$ respectively, where $\mathcal{O}_T$ hides constants in $T$. Notably, the dependence of these complexity bounds on $\epsilon$ and $T$ are separate in the sense that changing one does not impact the dependence of the bounds on the other. Moreover, our algorithm is parameter-free and does not require any (increasing) order of mini-batches to converge unlike the common practice in the analysis of stochastic conditional gradient-type algorithms