A Proof of The Triangular Ashbaugh-Benguria-Payne-P\'{o}lya-Weinberger Inequality

In this paper, we show that for all triangles in the plane, the equilateral triangle maximizes the ratio of the first two Dirichlet-Laplacian eigenvalues. This is an extension of work by Siudeja, who proved the inequality in the case of acute triangles. The proof utilizes inequalities due to Siudeja and Freitas, together with improved variational bounds.Comment: 16 pages, 4 figure

Representation Learning via Manifold Flattening and Reconstruction

This work proposes an algorithm for explicitly constructing a pair of neural networks that linearize and reconstruct an embedded submanifold, from finite samples of this manifold. Our such-generated neural networks, called Flattening Networks (FlatNet), are theoretically interpretable, computationally feasible at scale, and generalize well to test data, a balance not typically found in manifold-based learning methods. We present empirical results and comparisons to other models on synthetic high-dimensional manifold data and 2D image data. Our code is publicly available.Comment: 44 pages, 19 figure

Integrating Deep Learning with the Theory of Nonlinear, Chaotic, and History-Dependent Dynamics

Deep learning approaches for modeling challenging dynamics are becoming more and more ubiquitous. However, as deep neural networks (DNNs) are not fully understood even in their own domain, interpretability of these models in dynamical settings can be a challenge. In this work, we first give a theoretical introduction to three common and challenging phenomena in differentiable dynamical systems: nonlinearity, chaos, and memory effects. After looking at these three challenges in theoretical detail, we develop from this theory two deep learning approaches for dynamical system modeling: the former closely resembling a recurrent neural network (RNN), and the latter closely resembling a Transformer. Through the theoretical development of these models, we formally analyze the role of each component in the overall learning problem, interpret meaningful information about the underlying dynamical system from the model, and design models that are robust to nonlinearity, chaos, and memory effects

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction

This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the feature space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding and decoding mappings sought can be formulated as a two-player minimax game between the encoder and decoderfor the learned representation. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedback from control systems and avoids expensive evaluating and minimizing of approximated distances between arbitrary distributions in either the data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performance of the encoder is competitive and arguably better than existing methods based on GAN, VAE, or a combination of both. Unlike existing generative models, the so-learned features of the multiple classes are structured instead of hidden: different classes are explicitly mapped onto corresponding independent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independent principal components within each subspace

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction

This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the feature space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding and decoding mappings sought can be formulated as a two-player minimax game between the encoder and decoderfor the learned representation. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedback from control systems and avoids expensive evaluating and minimizing of approximated distances between arbitrary distributions in either the data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performance of the encoder is competitive and arguably better than existing methods based on GAN, VAE, or a combination of both. Unlike existing generative models, the so-learned features of the multiple classes are structured instead of hidden: different classes are explicitly mapped onto corresponding independent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independent principal components within each subspace