1,215 research outputs found

    A Unifying Theory for Graph Transformation

    Get PDF
    The field of graph transformation studies the rule-based transformation of graphs. An important branch is the algebraic graph transformation tradition, in which approaches are defined and studied using the language of category theory. Most algebraic graph transformation approaches (such as DPO, SPO, SqPO, and AGREE) are opinionated about the local contexts that are allowed around matches for rules, and about how replacement in context should work exactly. The approaches also differ considerably in their underlying formal theories and their general expressiveness (e.g., not all frameworks allow duplication). This dissertation proposes an expressive algebraic graph transformation approach, called PBPO+, which is an adaptation of PBPO by Corradini et al. The central contribution is a proof that PBPO+ subsumes (under mild restrictions) DPO, SqPO, AGREE, and PBPO in the important categorical setting of quasitoposes. This result allows for a more unified study of graph transformation metatheory, methods, and tools. A concrete example of this is found in the second major contribution of this dissertation: a graph transformation termination method for PBPO+, based on decreasing interpretations, and defined for general categories. By applying the proposed encodings into PBPO+, this method can also be applied for DPO, SqPO, AGREE, and PBPO

    Classical and quantum algorithms for scaling problems

    Get PDF
    This thesis is concerned with scaling problems, which have a plethora of connections to different areas of mathematics, physics and computer science. Although many structural aspects of these problems are understood by now, we only know how to solve them efficiently in special cases.We give new algorithms for non-commutative scaling problems with complexity guarantees that match the prior state of the art. To this end, we extend the well-known (self-concordance based) interior-point method (IPM) framework to Riemannian manifolds, motivated by its success in the commutative setting. Moreover, the IPM framework does not obviously suffer from the same obstructions to efficiency as previous methods. It also yields the first high-precision algorithms for other natural geometric problems in non-positive curvature.For the (commutative) problems of matrix scaling and balancing, we show that quantum algorithms can outperform the (already very efficient) state-of-the-art classical algorithms. Their time complexity can be sublinear in the input size; in certain parameter regimes they are also optimal, whereas in others we show no quantum speedup over the classical methods is possible. Along the way, we provide improvements over the long-standing state of the art for searching for all marked elements in a list, and computing the sum of a list of numbers.We identify a new application in the context of tensor networks for quantum many-body physics. We define a computable canonical form for uniform projected entangled pair states (as the solution to a scaling problem), circumventing previously known undecidability results. We also show, by characterizing the invariant polynomials, that the canonical form is determined by evaluating the tensor network contractions on networks of bounded size

    Backpropagation Beyond the Gradient

    Get PDF
    Automatic differentiation is a key enabler of deep learning: previously, practitioners were limited to models for which they could manually compute derivatives. Now, they can create sophisticated models with almost no restrictions and train them using first-order, i. e. gradient, information. Popular libraries like PyTorch and TensorFlow compute this gradient efficiently, automatically, and conveniently with a single line of code. Under the hood, reverse-mode automatic differentiation, or gradient backpropagation, powers the gradient computation in these libraries. Their entire design centers around gradient backpropagation. These frameworks are specialized around one specific task—computing the average gradient in a mini-batch. This specialization often complicates the extraction of other information like higher-order statistical moments of the gradient, or higher-order derivatives like the Hessian. It limits practitioners and researchers to methods that rely on the gradient. Arguably, this hampers the field from exploring the potential of higher-order information and there is evidence that focusing solely on the gradient has not lead to significant recent advances in deep learning optimization. To advance algorithmic research and inspire novel ideas, information beyond the batch-averaged gradient must be made available at the same level of computational efficiency, automation, and convenience. This thesis presents approaches to simplify experimentation with rich information beyond the gradient by making it more readily accessible. We present an implementation of these ideas as an extension to the backpropagation procedure in PyTorch. Using this newly accessible information, we demonstrate possible use cases by (i) showing how it can inform our understanding of neural network training by building a diagnostic tool, and (ii) enabling novel methods to efficiently compute and approximate curvature information. First, we extend gradient backpropagation for sequential feedforward models to Hessian backpropagation which enables computing approximate per-layer curvature. This perspective unifies recently proposed block- diagonal curvature approximations. Like gradient backpropagation, the computation of these second-order derivatives is modular, and therefore simple to automate and extend to new operations. Based on the insight that rich information beyond the gradient can be computed efficiently and at the same time, we extend the backpropagation in PyTorch with the BackPACK library. It provides efficient and convenient access to statistical moments of the gradient and approximate curvature information, often at a small overhead compared to computing just the gradient. Next, we showcase the utility of such information to better understand neural network training. We build the Cockpit library that visualizes what is happening inside the model during training through various instruments that rely on BackPACK’s statistics. We show how Cockpit provides a meaningful statistical summary report to the deep learning engineer to identify bugs in their machine learning pipeline, guide hyperparameter tuning, and study deep learning phenomena. Finally, we use BackPACK’s extended automatic differentiation functionality to develop ViViT, an approach to efficiently compute curvature information, in particular curvature noise. It uses the low-rank structure of the generalized Gauss-Newton approximation to the Hessian and addresses shortcomings in existing curvature approximations. Through monitoring curvature noise, we demonstrate how ViViT’s information helps in understanding challenges to make second-order optimization methods work in practice. This work develops new tools to experiment more easily with higher-order information in complex deep learning models. These tools have impacted works on Bayesian applications with Laplace approximations, out-of-distribution generalization, differential privacy, and the design of automatic differentia- tion systems. They constitute one important step towards developing and establishing more efficient deep learning algorithms

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    Locality and Exceptional Points in Pseudo-Hermitian Physics

    Get PDF
    Pseudo-Hermitian operators generalize the concept of Hermiticity. Included in this class of operators are the quasi-Hermitian operators, which define a generalization of quantum theory with real-valued measurement outcomes and unitary time evolution. This thesis is devoted to the study of locality in quasi-Hermitian theory, the symmetries and conserved quantities associated with non-Hermitian operators, and the perturbative features of pseudo-Hermitian matrices. An implicit assumption of the tensor product model of locality is that the inner product factorizes with the tensor product. Quasi-Hermitian quantum theory generalizes the tensor product model by modifying the Born rule via a metric operator with nontrivial Schmidt rank. Local observable algebras and expectation values are examined in chapter 5. Observable algebras of two one-dimensional fermionic quasi-Hermitian chains are explicitly constructed. Notably, there can be spatial subsystems with no nontrivial observables. Despite devising a new framework for local quantum theory, I show that expectation values of local quasi-Hermitian observables can be equivalently computed as expectation values of Hermitian observables. Thus, quasi-Hermitian theories do not increase the values of nonlocal games set by Hermitian theories. Furthermore, Bell's inequality violations in quasi-Hermitian theories never exceed the Tsirelson bound of Hermitian quantum theory. A perturbative feature present in pseudo-Hermitian curves which has no Hermitian counterpart is the exceptional point, a branch point in the set of eigenvalues. An original finding presented in section 2.6.3 is a correspondence between cusp singularities of algebraic curves and higher-order exceptional points. Eigensystems of one-dimensional lattice models admit closed-form expressions that can be used to explore the new features of non-Hermitian physics. One-dimensional lattice models with a pair of non Hermitian defect potentials with balanced gain and loss, Δ±iÎł, are investigated in chapter 3. Conserved quantities and positive-definite metric operators are examined. When the defects are nearest neighbour, the entire spectrum simultaneously becomes complex when Îł increases beyond a second-order exceptional point. When the defects are at the edges of the chain and the hopping amplitudes are 2-periodic, as in the Su-Schrieffer-Heeger chain, the PT-phase transition is dictated by the topological phase of the system. In the thermodynamic limit, PT-symmetry spontaneously breaks in the topologically non-trivial phase due to the presence of edge states. Chiral symmetry and representation theory are utilized in chapter 4 to derive large classes of pseudo-Hermitian operators with closed-form intertwining operators. These intertwining operators include positive-definite metric operators in the quasi-Hermitian case. The PT-phase transition is explicitly determined in a special case

    Advances and Applications of DSmT for Information Fusion. Collected Works, Volume 5

    Get PDF
    This ïŹfth volume on Advances and Applications of DSmT for Information Fusion collects theoretical and applied contributions of researchers working in different ïŹelds of applications and in mathematics, and is available in open-access. The collected contributions of this volume have either been published or presented after disseminating the fourth volume in 2015 in international conferences, seminars, workshops and journals, or they are new. The contributions of each part of this volume are chronologically ordered. First Part of this book presents some theoretical advances on DSmT, dealing mainly with modiïŹed Proportional ConïŹ‚ict Redistribution Rules (PCR) of combination with degree of intersection, coarsening techniques, interval calculus for PCR thanks to set inversion via interval analysis (SIVIA), rough set classiïŹers, canonical decomposition of dichotomous belief functions, fast PCR fusion, fast inter-criteria analysis with PCR, and improved PCR5 and PCR6 rules preserving the (quasi-)neutrality of (quasi-)vacuous belief assignment in the fusion of sources of evidence with their Matlab codes. Because more applications of DSmT have emerged in the past years since the apparition of the fourth book of DSmT in 2015, the second part of this volume is about selected applications of DSmT mainly in building change detection, object recognition, quality of data association in tracking, perception in robotics, risk assessment for torrent protection and multi-criteria decision-making, multi-modal image fusion, coarsening techniques, recommender system, levee characterization and assessment, human heading perception, trust assessment, robotics, biometrics, failure detection, GPS systems, inter-criteria analysis, group decision, human activity recognition, storm prediction, data association for autonomous vehicles, identiïŹcation of maritime vessels, fusion of support vector machines (SVM), Silx-Furtif RUST code library for information fusion including PCR rules, and network for ship classiïŹcation. Finally, the third part presents interesting contributions related to belief functions in general published or presented along the years since 2015. These contributions are related with decision-making under uncertainty, belief approximations, probability transformations, new distances between belief functions, non-classical multi-criteria decision-making problems with belief functions, generalization of Bayes theorem, image processing, data association, entropy and cross-entropy measures, fuzzy evidence numbers, negator of belief mass, human activity recognition, information fusion for breast cancer therapy, imbalanced data classiïŹcation, and hybrid techniques mixing deep learning with belief functions as well

    Introduction to Riemannian Geometry and Geometric Statistics: from basic theory to implementation with Geomstats

    Get PDF
    International audienceAs data is a predominant resource in applications, Riemannian geometry is a natural framework to model and unify complex nonlinear sources of data.However, the development of computational tools from the basic theory of Riemannian geometry is laborious.The work presented here forms one of the main contributions to the open-source project geomstats, that consists in a Python package providing efficient implementations of the concepts of Riemannian geometry and geometric statistics, both for mathematicians and for applied scientists for whom most of the difficulties are hidden under high-level functions. The goal of this monograph is two-fold. First, we aim at giving a self-contained exposition of the basic concepts of Riemannian geometry, providing illustrations and examples at each step and adopting a computational point of view. The second goal is to demonstrate how these concepts are implemented in Geomstats, explaining the choices that were made and the conventions chosen. The general concepts are exposed and specific examples are detailed along the text.The culmination of this implementation is to be able to perform statistics and machine learning on manifolds, with as few lines of codes as in the wide-spread machine learning tool scikit-learn. We exemplify this with an introduction to geometric statistics

    Identification of Novel Properties of Metabolic Systems Through Null-Space Analysis

    Get PDF
    Metabolic models provide a mathematical description of the complex network of biochemical reactions that sustain life. Among these, genome-scale models capture the entire metabolism of an organism, by encompassing all known biochemical reactions encoded by its genome. They are invaluable tools for exploring the metabolic potential of an organism, such as by predicting its response to different stimuli and identifying which reactions are essential for its survival. However, as the understanding of metabolism continues to grow, so too has the size and complexity of metabolic models, making the need for novel techniques that can simplify networks and extract specific features from them ever more important. This thesis addresses this challenge by leveraging the underlying structure of the network embodied by these models. Three different approaches are presented. Firstly, an algorithm that uses convex analysis techniques to decompose flux measurements into a set of fundamental flux pathways is developed and applied to a genome-scale model of Campylobacter jejuni in order to investigate its absolute requirement for environmental oxygen. This approach aims to overcome the computational limitations associated with the traditional technique of elementary mode analysis. Secondly, a method that can reduce the size of models by removing redundancies is introduced. This method identifies alternative pathways that lead from the same start to end product and is useful for identifying systematic errors that arise from model construction and for revealing information about the network’s flexibility. Finally, a novel technique for relating metabolites based on relationships between their concentration changes, or alternatively their chemical similarity, is developed based on the invariant properties of the left null-space of the stoichiometry matrix. Although various methods for relating the composition of metabolites exist, this technique has the advantage of not requiring any information apart from the model’s structure and allowed for the development of an algorithm that can simplify models and their analysis by extracting pathways containing metabolites that have similar composition. Furthermore, a method that uses the left null-space to facilitate the identification of un-balanced reactions in models is also presented
    • 

    corecore