GRAPH REPRESENTATION LEARNING WITH BOX EMBEDDINGS

Graphs are ubiquitous data structures, present in many machine-learning tasks, such as link prediction of products and node classification of scientific papers. As gradient descent drives the training of most modern machine learning architectures, the ability to encode graph-structured data using a differentiable representation is essential to make use of this data. Most approaches encode graph structure in Euclidean space, however, it is non-trivial to model directed edges. The naive solution is to represent each node using a separate source and target vector, however, this can decouple the representation, making it harder for the model to capture information within longer paths in the graph. In this dissertation, we propose to model graphs by representing each node as a \textit{box} (a Cartesian product of intervals) where directed edges are captured by the relative containment of one box in another. Theoretical proof shows that our proposed box embeddings have the expressiveness to represent any \emph{directed acyclic graph}. We also perform rigorous empirical evaluations of vector, hyperbolic, and region-based geometric representations on several families of synthetic and real-world directed graphs. Extensive experimental results suggest that the box containment can allow for transitive relationships to be modeled easily. We further propose t-Box, a variant of box embeddings that learns the temperature together during training. t-Box uses a learned smoothing parameter to achieve better representational capacity than vector models in low dimensions, while also avoiding performance saturation common to other geometric models in high dimensions. Though promising, modeling directed graphs that both contain cycles and some element of transitivity, two properties common in real-world settings, is challenging. Box embeddings, which can be thought of as representing the graph as an intersection over some learned super-graphs, have a natural inductive bias toward modeling transitivity, but (as we prove) cannot model cycles. To address this issue, we propose binary code box embeddings, where a learned binary code selects a subset of graphs for intersection. We explore several variants, including global binary codes (amounting to a union over intersections) and per-vertex binary codes (allowing greater flexibility) as well as methods of regularization. Theoretical and empirical results show that the proposed models not only preserve a useful inductive bias of transitivity but also have sufficient representational capacity to model arbitrary graphs, including graphs with cycles. Lastly, we discuss the use case where box embeddings are not free parameters but are produced by functions. In particular, we explore whether neural networks can map node features into the box space. This is critical in many real-world scenarios. On the one hand, graphs are sparse and the majority of vertices only have few connections or are completely isolated. On the other hand, there may exist rich node features such as attributes and descriptions, that could be useful for prediction tasks. The experimental analysis points out both the effectiveness and insufficiency of multi-layer perceptron-based encoders under different circumstances

Asymmetric colorings of products of graphs and digraphs

We extend results about asymmetric colorings of finite Cartesian products of graphs to strong and direct products of graphs and digraphs. On the way we shorten proofs for the existence of prime factorizations of finite digraphs and characterize the structure of the automorphism groups of strong and direct products. The paper ends with results on asymmetric colorings of Cartesian products of finite and infinite digraphs.http://www.elsevier.com/locate/dam2020-08-15hj2019Mathematics and Applied Mathematic

Global well-posedness of the periodic cubic fourth order NLS in negative Sobolev spaces

We consider the Cauchy problem for the cubic fourth order nonlinear Schr\"odinger equation (4NLS) on the circle. In particular, we prove global well-posedness of the renormalized 4NLS in negative Sobolev spaces $H^s(\mathbb{T})$, $s > -\frac{1}{3}$, with enhanced uniqueness. The proof consists of two separate arguments. (i) We first prove global existence in $H^s(\mathbb{T})$, $s > -\frac{9}{20}$, via the short-time Fourier restriction norm method. By following the argument in Guo-Oh for the cubic NLS, this also leads to non-existence of solutions for the (non-renormalized) 4NLS in negative Sobolev spaces. (ii) We then prove enhanced uniqueness in $H^s(\mathbb{T})$, $s > -\frac{1}{3}$, by establishing an energy estimate for the difference of two solutions with the same initial condition. For this purpose, we perform an infinite iteration of normal form reductions on the $H^s$-energy functional, allowing us to introduce an infinite sequence of correction terms to the $H^s$-energy functional in the spirit of the $I$-method. In fact, the main novelty of this paper is this reduction of the $H^s$-energy functionals (for a single solution and for the difference of two solutions with the same initial condition) to sums of infinite series of multilinear terms of increasing degrees.Comment: 65 pages. Published in Forum Math. Sigm

Reflect-Push Methods Part I: Two Dimensional Techniques

We determine all maximum weight downsets in the product of two chains, where the weight function is a strictly increasing function of the rank. Many discrete isoperimetric problems can be reduced to the maximum weight downset problem. Our results generalize Lindsay's edge-isoperimetric theorem in two dimensions in several directions. They also imply and strengthen (in several directions) a result of Ahlswede and Katona concerning graphs with maximal number of adjacent pairs of edges. We find all optimal shifted graphs in the Ahlswede-Katona problem. Furthermore, the results of Ahlswede-Katona are extended to posets with a rank increasing and rank constant weight function. Our results also strengthen a special case of a recent result by Keough and Radcliffe concerning graphs with the fewest matchings. All of these results are achieved by applications of a key lemma that we call the reflect-push method. This method is geometric and combinatorial. Most of the literature on edge-isoperimetric inequalities focuses on finding a solution, and there are no general methods for finding all possible solutions. Our results give a general approach for finding all compressed solutions for the above edge-isoperimetric problems. By using the Ahlswede-Cai local-global principle, one can conclude that lexicographic solutions are optimal for many cases of higher dimensional isoperimetric problems. With this and our two dimensional results we can prove Lindsay's edge-isoperimetric inequality in any dimension. Furthermore, our results show that lexicographic solutions are the unique solutions for which compression techniques can be applied in this general setting

Domino statistics of the two-periodic Aztec diamond

Random domino tilings of the Aztec diamond shape exhibit interesting features and some of the statistical properties seen in random matrix theory. As a statistical mechanical model it can be thought of as a dimer model or as a certain random surface. We consider the Aztec diamond with a two-periodic weighting which exhibits all three possible phases that occur in these types of models, often referred to as solid, liquid and gas. To analyze this model, we use entries of the inverse Kasteleyn matrix which give the probability of any configuration of dominoes. A formula for these entries, for this particular model, was derived by Chhita and Young (2014). In this paper, we find a major simplification of this formula expressing entries of the inverse Kasteleyn matrix by double contour integrals which makes it possible to investigate their asymptotics. In a part of the Aztec diamond, where the asymptotic analysis is simpler, we use this formula to show that the entries of the inverse Kasteleyn matrix converge to the known entries of the full-plane inverse Kasteleyn matrices for the different phases. We also study the detailed asymptotics of the inverse Kasteleyn matrix at both the ‘liquid–solid’ and ‘liquid–gas’ boundaries, and find the extended Airy kernel in the next order asymptotics. Finally we provide a potential candidate for a combinatorial description of the liquid–gas boundary

Symmetric Edit Lenses: A New Foundation for Bidirectional Languages

Lenses are bidirectional transformations between pairs of connected structures capable of translating an edit on one structure into an edit on the other. Most of the extensive existing work on lenses has focused on the special case of asymmetric lenses, where one structures is taken as primary and the other is thought of as a projection or view. Some symmetric variants exist, where each structure contains information not present in the other, but these all lack the basic operation of composition. Additionally, existing accounts do not represent edits carefully, making incremental operation difficult or producing unsatisfactory synchronization candidates. We present a new symmetric formulation which works with descriptions of changes to structures, rather than with the structures themselves. We construct a semantic space of edit lenses between “editable structures”—monoids of edits with a partial monoid action for applying edits—with natural laws governing their behavior. We present generalizations of a number of known constructions on asymmetric lenses and settle some longstanding questions about their properties—in particular, we prove the existence of (symmetric monoidal) tensor products and sums and the non-existence of full categorical products and sums in a category of lenses. Universal algebra shows how to build iterator lenses for structured data such as lists and trees, yielding lenses for operations like mapping, filtering, and concatenation from first principles. More generally, we provide mapping combinators based on the theory of containers. Finally, we present a prototype implementation of the core theory and take a first step in addressing the challenge of translating between user gestures and the internal representation of edits

The aesthetics of science fiction spaceship design

In this thesis, we present a detailed analysis of the conventions that appear in fictional spaceship design, including a discussion of their origins, their uses in emulating certain traits, and reasons these conventions might be followed or ignored. We uncover these conventions by examining and comparing popular spaceship designs from the past sixty years, which we present in a detailed survey. We also examine an aesthetic interpretation of information theory, which can be used to describe the balance of uniformity amidst variety, and discuss specific strategies for incorporating these principles into the creation of spaceship surface details. Procedural modeling describes a set of techniques used to allow computers to generate digital content such as 3D digital models automatically. However, procedural modeling to date has focused on very specific areas: natural scenery such as trees and terrain, or cityscapes such as road maps and buildings. While these types of models are important and useful, they focus on a specific subset of the procedural modeling problem. Though procedural generation can be an invaluable tool for providing viable and dynamic content, it is troubling that so few types of objects have been studied in this area. Using the aesthetic and spaceship principles we define, we have developed a prototype system to procedurally generate the surface details of a large scale spaceship. Given a surface representing the frame of a spaceship, we apply geometry automatically in a coherent manner to achieve the appearance of a spaceship by emulating important traits

On the Stability of Structured Prediction

Many important applications of artificial intelligence---such as image segmentation, part-of-speech tagging and network classification---are framed as multiple, interdependent prediction tasks. These structured prediction problems are typically modeled using some form of joint inference over the outputs, to exploit the relational dependencies. Joint reasoning can significantly improve predictive accuracy, but it introduces a complication in the analysis of structured models: the stability of inference. In optimizations involving multiple interdependent variables, such as joint inference, a small change to the input or parameters could induce drastic changes in the solution. In this dissertation, I investigate the impact of stability in structured prediction. I explore two topics, connected by the stability of inference. First, I provide generalization bounds for learning from a limited number of examples with large internal structure. The effective learning rate can be significantly sharper than rates given in related work. Under certain conditions on the data distribution and stability of the predictor, the bounds decrease with both the number of examples and the size of each example, meaning one could potentially learn from a single giant example. Secondly, I investigate the benefits of learning with strongly convex variational inference. Using the duality between strong convexity and stability, I demonstrate, both theoretically and empirically, that learning with a strongly convex free energy can result in significantly more accurate marginal probabilities. One consequence of this work is a new technique that ``strongly convexifies" many free energies used in practice. These two seemingly unrelated threads are tied by the idea that stable inference leads to lower error, particularly in the limited example setting, thereby demonstrating that inference stability is of critical importance to the study and practice of structured prediction