1,919 research outputs found
Exploiting Structural Properties in the Analysis of High-dimensional Dynamical Systems
The physical and cyber domains with which we interact are filled with high-dimensional dynamical systems. In machine learning, for instance, the evolution of overparametrized neural networks can be seen as a dynamical system. In networked systems, numerous agents or nodes dynamically interact with each other. A deep understanding of these systems can enable us to predict their behavior, identify potential pitfalls, and devise effective solutions for optimal outcomes. In this dissertation, we will discuss two classes of high-dimensional dynamical systems with specific structural properties that aid in understanding their dynamic behavior.
In the first scenario, we consider the training dynamics of multi-layer neural networks. The high dimensionality comes from overparametrization: a typical network has a large depth and hidden layer width. We are interested in the following question regarding convergence: Do network weights converge to an equilibrium point corresponding to a global minimum of our training loss, and how fast is the convergence rate? The key to those questions is the symmetry of the weights, a critical property induced by the multi-layer architecture. Such symmetry leads to a set of time-invariant quantities, called weight imbalance, that restrict the training trajectory to a low-dimensional manifold defined by the weight initialization. A tailored convergence analysis is developed over this low-dimensional manifold, showing improved rate bounds for several multi-layer network models studied in the literature, leading to novel characterizations of the effect of weight imbalance on the convergence rate.
In the second scenario, we consider large-scale networked systems with multiple weakly-connected groups. Such a multi-cluster structure leads to a time-scale separation between the fast intra-group interaction due to high intra-group connectivity, and the slow inter-group oscillation, due to the weak inter-group connection. We develop a novel frequency-domain network coherence analysis that captures both the coherent behavior within each group, and the dynamical interaction between groups, leading to a structure-preserving model-reduction methodology for large-scale dynamic networks with multiple clusters under general node dynamics assumptions
Classical and quantum algorithms for scaling problems
This thesis is concerned with scaling problems, which have a plethora of connections to different areas of mathematics, physics and computer science. Although many structural aspects of these problems are understood by now, we only know how to solve them efficiently in special cases.We give new algorithms for non-commutative scaling problems with complexity guarantees that match the prior state of the art. To this end, we extend the well-known (self-concordance based) interior-point method (IPM) framework to Riemannian manifolds, motivated by its success in the commutative setting. Moreover, the IPM framework does not obviously suffer from the same obstructions to efficiency as previous methods. It also yields the first high-precision algorithms for other natural geometric problems in non-positive curvature.For the (commutative) problems of matrix scaling and balancing, we show that quantum algorithms can outperform the (already very efficient) state-of-the-art classical algorithms. Their time complexity can be sublinear in the input size; in certain parameter regimes they are also optimal, whereas in others we show no quantum speedup over the classical methods is possible. Along the way, we provide improvements over the long-standing state of the art for searching for all marked elements in a list, and computing the sum of a list of numbers.We identify a new application in the context of tensor networks for quantum many-body physics. We define a computable canonical form for uniform projected entangled pair states (as the solution to a scaling problem), circumventing previously known undecidability results. We also show, by characterizing the invariant polynomials, that the canonical form is determined by evaluating the tensor network contractions on networks of bounded size
Categorical Invariants of Graphs and Matroids
Graphs and matroids are two of the most important objects in combinatorics.We study invariants of graphs and matroids that behave well with respect to
certain morphisms by realizing these invariants as functors from a category of
graphs (resp. matroids).
For graphs, we study invariants that respect deletions and contractions ofedges. For an integer , we define a category of of graphs of genus at most
g where morphisms correspond to deletions and contractions. We prove that this
category is locally Noetherian and show that many graph invariants form finitely
generated modules over the category . This fact allows us to exihibit many
stabilization properties of these invariants. In particular we show that the torsion
that can occur in the homologies of the unordered configuration space of n points
in a graph and the matching complex of a graph are uniform over the entire family
of graphs with genus .
For matroids, we study valuative invariants of matroids. Given a matroid,one can define a corresponding polytope called the base polytope. Often, the base
polytope of a matroid can be decomposed into a cell complex made up of base
polytopes of other matroids. A valuative invariant of matroids is an invariant that
respects these polytope decompositions. We define a category of matroids
whose morphisms correspond to containment of base polytopes. We then define the
notion of a categorical matroid invariant which categorifies the notion of a valuative
invariant. Finally, we prove that the functor sending a matroid to its Orlik-Solomon
algebra is a categorical valuative invariant. This allows us to derive relations among
the Orlik-Solomon algebras of a matroid and matroids that decompose its base
polytope viewed as representations of any group whose action is compatible with
the polytope decomposition.
This dissertation includes previously unpublished co-authored material
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
Polynomial Identity Testing and the Ideal Proof System: PIT is in NP if and only if IPS can be p-simulated by a Cook-Reckhow proof system
The Ideal Proof System (IPS) of Grochow & Pitassi (FOCS 2014, J. ACM, 2018)
is an algebraic proof system that uses algebraic circuits to refute the
solvability of unsatisfiable systems of polynomial equations. One potential
drawback of IPS is that verifying an IPS proof is only known to be doable using
Polynomial Identity Testing (PIT), which is solvable by a randomized algorithm,
but whose derandomization, even into NSUBEXP, is equivalent to strong lower
bounds. However, the circuits that are used in IPS proofs are not arbitrary,
and it is conceivable that one could get around general PIT by leveraging some
structure in these circuits. This proposal may be even more tempting when IPS
is used as a proof system for Boolean Unsatisfiability, where the equations
themselves have additional structure.
Our main result is that, on the contrary, one cannot get around PIT as above:
we show that IPS, even as a proof system for Boolean Unsatisfiability, can be
p-simulated by a deterministically verifiable (Cook-Reckhow) proof system if
and only if PIT is in NP. We use our main result to propose a potentially new
approach to derandomizing PIT into NP
- …