102 research outputs found
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
On linear, fractional, and submodular optimization
In this thesis, we study four fundamental problems in the theory of optimization. 1. In fractional optimization, we are interested in minimizing a ratio of two functions over some domain. A well-known technique for solving this problem is the Newton– Dinkelbach method. We propose an accelerated version of this classical method and give a new analysis using the Bregman divergence. We show how it leads to improved or simplified results in three application areas. 2. The diameter of a polyhedron is the maximum length of a shortest path between any two vertices. The circuit diameter is a relaxation of this notion, whereby shortest paths are not restricted to edges of the polyhedron. For a polyhedron in standard equality form with constraint matrix A, we prove an upper bound on the circuit diameter that is quadratic in the rank of A and logarithmic in the circuit imbalance measure of A. We also give circuit augmentation algorithms for linear programming with similar iteration complexity. 3. The correlation gap of a set function is the ratio between its multilinear and concave extensions. We present improved lower bounds on the correlation gap of a matroid rank function, parametrized by the rank and girth of the matroid. We also prove that for a weighted matroid rank function, the worst correlation gap is achieved with uniform weights. Such improved lower bounds have direct applications in submodular maximization and mechanism design. 4. The last part of this thesis concerns parity games, a problem intimately related to linear programming. A parity game is an infinite-duration game between two players on a graph. The problem of deciding the winner lies in NP and co-NP, with no known polynomial algorithm to date. Many of the fastest (quasi-polynomial) algorithms have been unified via the concept of a universal tree. We propose a strategy iteration framework which can be applied on any universal tree
LIPIcs, Volume 261, ICALP 2023, Complete Volume
LIPIcs, Volume 261, ICALP 2023, Complete Volum
Neural function approximation on graphs: shape modelling, graph discrimination & compression
Graphs serve as a versatile mathematical abstraction of real-world phenomena in numerous scientific disciplines. This thesis is part of the Geometric Deep Learning subject area, a family of learning paradigms, that capitalise on the increasing volume of non-Euclidean data so as to solve real-world tasks in a data-driven manner. In particular, we focus on the topic of graph function approximation using neural networks, which lies at the heart of many relevant methods. In the first part of the thesis, we contribute to the understanding and design of Graph Neural Networks (GNNs). Initially, we investigate the problem of learning on signals supported on a fixed graph. We show that treating graph signals as general graph spaces is restrictive and conventional GNNs have limited expressivity. Instead, we expose a more enlightening perspective by drawing parallels between graph signals and signals on Euclidean grids, such as images and audio. Accordingly, we propose a permutation-sensitive GNN based on an operator analogous to shifts in grids and instantiate it on 3D meshes for shape modelling (Spiral Convolutions). Following, we focus on learning on general graph spaces and in particular on functions that are invariant to graph isomorphism. We identify a fundamental trade-off between invariance, expressivity and computational complexity, which we address with a symmetry-breaking mechanism based on substructure encodings (Graph Substructure Networks). Substructures are shown to be a powerful tool that provably improves expressivity while controlling computational complexity, and a useful inductive bias in network science and chemistry. In the second part of the thesis, we discuss the problem of graph compression, where we analyse the information-theoretic principles and the connections with graph generative models. We show that another inevitable trade-off surfaces, now between computational complexity and compression quality, due to graph isomorphism. We propose a substructure-based dictionary coder - Partition and Code (PnC) - with theoretical guarantees that can be adapted to different graph distributions by estimating its parameters from observations. Additionally, contrary to the majority of neural compressors, PnC is parameter and sample efficient and is therefore of wide practical relevance. Finally, within this framework, substructures are further illustrated as a decisive archetype for learning problems on graph spaces.Open Acces
Recommended from our members
Essays in transportation inequalities, entropic gradient flows and mean field approximations
This thesis consists of four chapters. In Chapter 1, we focus on a class of transportation inequalities known as the transportation-information inequalities. These inequalities bound optimal transportation costs in terms of relative Fisher information, and are known to characterize certain concentration properties of Markov processes around their invariant measures. We provide a characterization of the quadratic transportation-information inequality in terms of a dimension-free concentration property for i.i.d. copies of the underlying Markov process, identifying the precise high-dimensional concentration property encoded by this inequality. We also illustrate how this result is an instance of a general convex-analytic tensorization principle.
In Chapter 2, we study the entropic gradient flow property of McKean--Vlasov diffusions via a stochastic analysis approach. We formulate a trajectorial version of the relative entropy dissipation identity for these interacting diffusions, which describes the rate of relative entropy dissipation along every path of the diffusive motion. As a first application, we obtain a new interpretation of the gradient flow structure for the granular media equation. Secondly, we show how the trajectorial approach leads to a new derivation of the HWBI inequality.
In Chapter 3, we further extend the trajectorial approach to a class of degenerate diffusion equations that includes the porous medium equation. These equations are posed on a bounded domain and are subject to no-flux boundary conditions, so that their corresponding probabilistic representations are stochastic differential equations with normal reflection on the boundary. Our stochastic analysis approach again leads to a new derivation of the Wasserstein gradient flow property for these nonlinear diffusions, as well as to a simple proof of the HWI inequality in the present context.
Finally, in Chapter 4, we turn our attention to mean field approximation -- a method widely used to study the behavior of large stochastic systems of interacting particles. We propose a new approach to deriving quantitative mean field approximations for any strongly log-concave probability measure. Our framework is inspired by the recent theory of nonlinear large deviations, for which we offer an efficient non-asymptotic perspective in log-concave settings based on functional inequalities. We discuss three implications, in the contexts of continuous Gibbs measures on large graphs, high-dimensional Bayesian linear regression, and the construction of decentralized near-optimizers in high-dimensional stochastic control problems
Asymptotics and Statistical Inference in High-Dimensional Low-Rank Matrix Models
High-dimensional matrix and tensor data is ubiquitous in machine learning and statistics
and often exhibits low-dimensional structure. With the rise of these types of data is the need to develop statistical inference procedures that adequately address the low-dimensional structure in a principled manner. In this dissertation we study asymptotic theory and statistical inference in structured low-rank matrix models in high-dimensional regimes where the column and row dimensions of the matrix are allowed to grow, and we consider a variety of settings for which structured low-rank matrix models manifest.
Chapter 1 establishes the general framework for statistical analysis in high-dimensional low-rank matrix models, including introducing entrywise perturbation bounds, asymptotic theory, distributional theory, and statistical inference, illustrated throughout via the matrix denoising model. In Chapter 2, Chapter 3, and Chapter 4 we study the entrywise estimation of singular vectors and eigenvectors in different structured settings, with Chapter 2 considering heteroskedastic and dependent noise, Chapter 3 sparsity, and Chapter 4 additional tensor structure. In Chapter 5 we apply previous asymptotic theory to study a two-sample
test for equality of distribution in network analysis, and in Chapter 6 we study a model for shared community memberships across multiple networks, and we propose and analyze a joint spectral clustering algorithm that leverages newly developed asymptotic theory for this setting.
Throughout this dissertation we emphasize tools and techniques that are data-driven, nonparametric, and adaptive to signal strength, and, where applicable, noise distribution. The contents of Chapters 2-6 are based on the papers Agterberg et al. (2022b); Agterberg and Sulam (2022); Agterberg and Zhang (2022); Agterberg et al. (2020a) and Agterberg et al. (2022a) respectively, and Chapter 1 contains several novel results
LIPIcs, Volume 274, ESA 2023, Complete Volume
LIPIcs, Volume 274, ESA 2023, Complete Volum
Fundamentals
Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters
LIPIcs, Volume 258, SoCG 2023, Complete Volume
LIPIcs, Volume 258, SoCG 2023, Complete Volum
Advances in Bosonic Quantum Error Correction with Gottesman-Kitaev-Preskill Codes: Theory, Engineering and Applications
Encoding quantum information into a set of harmonic oscillators is considered
a hardware efficient approach to mitigate noise for reliable quantum
information processing. Various codes have been proposed to encode a qubit into
an oscillator -- including cat codes, binomial codes and
Gottesman-Kitaev-Preskill (GKP) codes. These bosonic codes are among the first
to reach a break-even point for quantum error correction. Furthermore, GKP
states not only enable close-to-optimal quantum communication rates in bosonic
channels, but also allow for error correction of an oscillator into many
oscillators. This review focuses on the basic working mechanism, performance
characterization, and the many applications of GKP codes, with emphasis on
recent experimental progress in superconducting circuit architectures and
theoretical progress in multimode GKP qubit codes and
oscillators-to-oscillators (O2O) codes. We begin with a preliminary
continuous-variable formalism needed for bosonic codes. We then proceed to the
quantum engineering involved to physically realize GKP states. We take a deep
dive into GKP stabilization and preparation in superconducting architectures
and examine proposals for realizing GKP states in the optical domain (along
with a concise review of GKP realization in trapped-ion platforms). Finally, we
present multimode GKP qubits and GKP-O2O codes, examine code performance and
discuss applications of GKP codes in quantum information processing tasks such
as computing, communication, and sensing.Comment: 77+5 pages, 31 figures. Minor bugs fixed in v2. comments are welcome
- …