Two Geometric Results regarding Hölder-Brascamp-Lieb Inequalities, and Two Novel Algorithms for Low-Rank Approximation

Broadly speaking, this thesis investigates mathematical questions motivated by computer science. The involved topics include communication avoiding algorithms, classical analysis, convex geometry, and low-rank matrix approximation. In total, the thesis consists of four self-contained sections, each adapted from papers the author has been a part of.The first two sections are both motivated by the Brascamp-Lieb inequalities, which are also often referred to as Hölder-Brascamp-Lieb inequalities. These inequalities have featured prominently in recent theoretical computer science work, due to connections to geometric complexity theory, harmonic analysis, communication-avoidance, and many other areas. Moreover, work generalizing the inequalities in various ways, such as to nonlinear versions, has been impactful to the study of differential equations.Section 1 studies the application of Hölder-Brascamp-Lieb (HBL) inequalities to the design of communication optimal algorithms. In particular, it describes optimal tiling (blocking) strategies for nested loops that lack data dependencies and exhibit affine memory access patterns. The problem roughly amounts to maximizing the volume of an object provided some of its linear images have bounded volume. The methods used are algorithmic.Another reason for the interest in these inequalities is because they are an interesting test case for non-convex optimization techniques. The optimal constant for a particular instance of the inequality is given by solving a non-convex optimization problem that is still highly structured. Of particular relevance to this thesis is that it can be formulated as a geodesically-convex problem, considered in the context of the manifold of positive definite matrices of determinant $1$. Even using the methods of Section 1, the procedure is not necessarily polynomial time, and this motivates further study of geodesic convexity.This lead to the work of Section 2, which discusses a notion of halfspace for Hadamard manifolds that is natural in the context of convex optimization. For this notion of halfspace, we generalize a classic result of Grunbaum, which itself is a corollary of Helly's theorem. Namely, given a probability distribution on the manifold, there is a point for which all halfspaces based at this point have at least 1/(n+1) of the mass, n being the dimension of the manifold. As an application, the gradient oracle complexity of geodesic convex optimization is polynomial in the parameters defining the problem. In particular it is polynomial in -log(epsilon), where epsilon is the desired error. This is a step toward the open question of whether such an algorithm exists.The remaining two sections of the paper present a different research direction, randomized numerical linear algebra. Numerical linear algebra has long been an important part of scientific computing. Due to the current trend of increasing matrix sizes and growing importance of fast, approximate solutions in industry, randomized methods are quickly increasing in popularity. Sections 3 and 4 in this thesis aim to show that randomized low-rank approximation algorithms satisfy many of the properties of classical rank-revealing factorizations.Section 3 introduces a Generalized Randomized QR-decomposition (RURV) that may be applied to arbitrary products of matrices and their inverses, without needing to explicitly compute the products or inverses. This factorization is a critical part of a communication-optimal spectral divide-and-conquer algorithm for the nonsymmetric eigenvalue problem. In this paper, we establish that this randomized QR-factorization satisfies the strong rank-revealing properties. We also formally prove its stability, making it suitable in applications. Finally, we present numerical experiments which demonstrate that our theoretical bounds capture the empirical behavior of the factorization.Section 4 concerns a Generalized LU-Factorization (GLU) for low-rank matrix approximation. We relate this to past approaches and extensively analyze its approximation properties. The established deterministic guarantees are combined with sketching ensembles satisfying Johnson-Lindenstrauss properties to present complete bounds. Particularly good performance is shown for the sub-sampled randomized Hadamard transform (SRHT) ensemble. Moreover, the factorization is shown to unify and generalize many past algorithms. It also helps to explain the effect of sketching on the growth factor during Gaussian Elimination

Communication Bounds for Convolutional Neural Networks

Convolutional neural networks (CNNs) are important in a wide variety of machine learning tasks and applications, so optimizing their performance is essential. Moving words of data between levels of a memory hierarchy or between processors on a network is much more expensive than the cost of arithmetic, so minimizing communication is critical to optimizing performance. In this paper, we present new lower bounds on data movement for mixed precision convolutions in both single-processor and parallel distributed memory models, as well as algorithms that outperform current implementations such as Im2Col. We obtain performance figures using GEMMINI, a machine learning accelerator, where our tiling provides improvements between 13% and 150% over a vendor supplied algorithm

Toward the Understanding of Irradiation Effects on Concrete: The Irradiated Minerals, Aggregates, and Concrete Database

The understanding of irradiation effects on concrete has become urgent due to the possible extension of the operating life of nuclear power plants. Although there are scarcity, uncertainties, and inconsistency in concrete irradiation data, literature indicated that significant reduction in concrete mechanical properties occurred mainly due to the radiation-induced volumetric expansion (RIVE) of aggregate at neutron fluence of 1.0x1019 n.cm-2 (Energy \u3e 10 KeV). This fluence is expected to be reached at 80 years of operation. Therefore, better understanding of aggregate RIVE could be obtained through understanding the RIVE of its mineral composition.A large amount of minerals and aggregates RIVE data were published recently in Russia, and reanalyzed by: (1) finding empirical models for minerals RIVEs; (2) upscaling minerals RIVEs to aggregate scale through homogenization; (3) comparing the upscaled and experimental RIVEs of aggregates to estimate crackings in them.Minerals empirical models were obtained by combining two different interpolation techniques with 90% confidence of RIVE estimation. Further analysis of minerals RIVEs indicated that silicate minerals have the highest RIVEs, and show different susceptibility to irradiation depending on: (1) the dimensionality of SiO4 polymerization; (2) the relative number of Si-O bond per unit cell; and (3) the relative bonding energy of unit cell.The upscaled RIVEs of aggregates were calculated at the same irradiation temperature (T) and neutron fluence (©) of experimental RIVEs. The Inverse Distance Weighting interpolation technique was used to normalize RIVEs at different conditions to a fixed condition of © Æ 1.0x1020n.cm-2 (E \u3e 10 KeV), and T Æ 80±C. A comparison of the two RIVEs showed that mineral composition and texture play a major role in RIVEs of aggregates. RIVEs of silicate-bearing aggregates were higher than RIVEs of carbonate-bearing aggregates. For all aggregates, high plagioclase feldspar content, medium-to-large mineral grain sizes, or both, have higher cracks in experimental RIVEs than other aggregates with similar mineral composition. Further observations indicated igneous intrusive aggregates have high RIVEs that might be due to residual strains stored in those aggregates during their formation under high pressure and temperature

The Robustness of Ecological Communities: Theory and Application.

As ecologists, we frequently rely on mathematical models to formulate and test our hypotheses concerning ecological communities. An important problem is whether and why interacting species coexist. Once our hypothesis for why coexistence happens is translated into the form of a model, we check to see whether the proposed mechanism could indeed lead to coexistence. Usually, the golden standard for evaluating coexistence has been to check whether the model possesses an all-positive, stable stationary state (where this state may be an equilibrium point, a limit cycle, or a chaotic or otherwise aperiodic orbit). This perspective, however, ignores another important aspect of the same problem: the robustness of the stationary state against parameter changes. We may find coexistence in a model, but if that coexistence collapses after even very slight parameter perturbations, it is not actually expected to hold. The purpose of this dissertation is fourfold. First, it aims at working out the quantitative, formal mathematical machinery for evaluating the robustness of ecological communities under complex circumstances, such as ones involving population structure or nonequilibrium community dynamics. Second, it applies this machinery to various ecological problems, ranging from the theoretical to the applied, to demonstrate the kinds of uses robustness analysis has. Among the models discussed are the sensitivity of a field-parametrized model of annual plant competition to parameter changes, the analysis of coexistence in the tolerance-fecundity tradeoff model, and predicting species diversity in a model of interspecific facilitation. Third, it takes a look at some of the consequences of robustness analysis for community patterns, arguing that the elementary biological fact that species are by and large discrete, well-defined entities is a natural consequence of the basic structure of ecological interactions, not of any model details. Fourth, the dissertation synthesizes some of the general conclusions of robustness analysis to formalize the concept of the ecological niche, revealing a fundamental unity between functional, temporal, and spatial mechanisms of diversity maintenance.PHDEcology and Evolutionary BiologyUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/102303/1/dysordys_1.pd

Third International Conference on Inverse Design Concepts and Optimization in Engineering Sciences (ICIDES-3)

Papers from the Third International Conference on Inverse Design Concepts and Optimization in Engineering Sciences (ICIDES) are presented. The papers discuss current research in the general field of inverse, semi-inverse, and direct design and optimization in engineering sciences. The rapid growth of this relatively new field is due to the availability of faster and larger computing machines