Analysis of non-unique solutions in mean field games

This thesis investigates cases when solutions to a mean field game (MFG) are non-unique. The symmetric Markov perfect information N-player game is considered and restricted to finite states and continuous time. The players' transitions are random with a parameter determined by their control. There is a unique joint distribution of the players for the symmetric Markov perfect equilibrium, but there can be multiple solutions to the MFG equations. This thesis focuses on understanding the behaviors of the many MFG solutions for the 2-state case. This thesis explores methods to determine which MFG solution represents the fluid limit trajectories of the N-player system for large populations. This thesis investigates the MFG map which acts on the MFG distributions and outputs a prediction of the population's distribution based on the expected response of any given player. The MFG solutions are exactly the fixed points of the MFG map. The MFG solution that approximates large population trajectories is conjectured to be the only stable point for the MFG map. There is a second concept investigated, social cost, which is the average accumulated cost per player. But as is shown, the social cost is not a good indicator of which MFG solution approximates large population trajectories. A set, called the bifurcation set, is defined by there being some possibility of multiple trajectories of a large population. Another important set is the indifference set, which indicates when the transition rate of the players to a state is positively reinforced by an increase of the empirical distribution of that state. However, numerical results are given, indicating that the fluid limit trajectory may relate to stability of the MFG map. It appears the MFG map is difficult to handle in many ways; stability of the mapping is difficult to show, even in a simple example and there are numerical anomalies such that non-fixed points appear to be numerically stable under rigorous tests

Stable configurations for population and social dynamics

This dissertation investigates global and local minima in two models: the Lotka--Volterra model for population dynamics and a tractable polarized opinion social dynamic model. This dissertation contains stability results of the Lotka--Volterra model when induced by a cycle graph food web network. Results such as orbits, chaos and the probability of stability are given. A result showing convexity of the weighted connections of the food web is sufficient for global stability is given as well. Stability results of food webs which are perturbed from the cycle graph are explored as well for comparison. This dissertation goes on to investigate how algebraic relationships within the community matrix predict stability for the generalized Lotka--Volterra model. In particular, it is shown that there is a strong relationship between the transversal eigenvalues with respect to a subset of the species in a system and the Schur compliment of the Jacobian at the interior fixed point with the submatrix determined by the same subset of species. This relationship gives an alternate proof to many well known results. This dissertation also analyzes the global and local stability of an opinion dynamic model which consists of a W-well potential and a graph Laplacian for coupling. The global minimizers and their lack of confinement to an orthant are investigated. The number of local minimizers are also investigated for various W-well potentials. This dissertation investigates the different types of bifurcations that can be seen depending on the differential properties of the W-potentials

The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation

Function approximation is widely used in reinforcement learning to handle the computational difficulties associated with very large state spaces. However, function approximation introduces errors which may lead to instabilities when using approximate dynamic programming techniques to obtain the optimal policy. Therefore, techniques such as lookahead for policy improvement and m-step rollout for policy evaluation are used in practice to improve the performance of approximate dynamic programming with function approximation. We quantitatively characterize, for the first time, the impact of lookahead and m-step rollout on the performance of approximate dynamic programming (DP) with function approximation: (i) without a sufficient combination of lookahead and m-step rollout, approximate DP may not converge, (ii) both lookahead and m-step rollout improve the convergence rate of approximate DP, and (iii) lookahead helps mitigate the effect of function approximation and the discount factor on the asymptotic performance of the algorithm. Our results are presented for two approximate DP methods: one which uses least-squares regression to perform function approximation and another which performs several steps of gradient descent of the least-squares objective in each iteration.Comment: 36 pages, 4 figure

GME: GPU-based Microarchitectural Extensions to Accelerate Homomorphic Encryption

Fully Homomorphic Encryption (FHE) enables the processing of encrypted data without decrypting it. FHE has garnered significant attention over the past decade as it supports secure outsourcing of data processing to remote cloud services. Despite its promise of strong data privacy and security guarantees, FHE introduces a slowdown of up to five orders of magnitude as compared to the same computation using plaintext data. This overhead is presently a major barrier to the commercial adoption of FHE. In this work, we leverage GPUs to accelerate FHE, capitalizing on a well-established GPU ecosystem available in the cloud. We propose GME, which combines three key microarchitectural extensions along with a compile-time optimization to the current AMD CDNA GPU architecture. First, GME integrates a lightweight on-chip compute unit (CU)-side hierarchical interconnect to retain ciphertext in cache across FHE kernels, thus eliminating redundant memory transactions. Second, to tackle compute bottlenecks, GME introduces special MOD-units that provide native custom hardware support for modular reduction operations, one of the most commonly executed sets of operations in FHE. Third, by integrating the MOD-unit with our novel pipelined $64$-bit integer arithmetic cores (WMAC-units), GME further accelerates FHE workloads by $19\%$. Finally, we propose a Locality-Aware Block Scheduler (LABS) that exploits the temporal locality available in FHE primitive blocks. Incorporating these microarchitectural features and compiler optimizations, we create a synergistic approach achieving average speedups of $796\times$, $14.2\times$, and $2.3\times$ over Intel Xeon CPU, NVIDIA V100 GPU, and Xilinx FPGA implementations, respectively

GME: GPU-based Microarchitectural Extensions to Accelerate Homomorphic Encryption

Fully Homomorphic Encryption (FHE) enables the processing of encrypted data without decrypting it. FHE has garnered significant attention over the past decade as it supports secure outsourcing of data processing to remote cloud services. Despite its promise of strong data privacy and security guarantees, FHE introduces a slowdown of up to five orders of magnitude as compared to the same computation using plaintext data. This overhead is presently a major barrier to the commercial adoption of FHE. While prior efforts recommend moving to custom accelerators to accelerate FHE computing, these solutions lack cost-effectiveness and scalability. In this work, we leverage GPUs to accelerate FHE, capitalizing on a well-established GPU ecosystem that is available in the cloud. We propose GME, which combines three key microarchitectural extensions along with a compile-time optimization to the current AMD CDNA GPU architecture. First, GME integrates a lightweight on-chip compute unit (CU)-side hierarchical interconnect to retain ciphertext in cache across FHE kernels, thus eliminating redundant memory transactions and improving performance. Second, to tackle compute bottlenecks, GME introduces special MOD-units that provide native custom hardware support for modular reduction operations, one of the most commonly executed sets of operations in FHE. Third, by integrating the MOD-unit with our novel pipelined 64-bit integer arithmetic cores (WMAC-units), GME further accelerates FHE workloads by 19%. Finally, we propose a Locality-Aware Block Scheduler (LABS) that improves FHE workload performance, exploiting the temporal locality available in FHE primitive blocks. Incorporating these microarchitectural features and compiler optimizations, we create a synergistic approach achieving average speedups of 796×, 14.2×, and 2.3× over Intel Xeon CPU, NVIDIA V100 GPU, and Xilinx FPGA implementations, respectively

Topological Deep Learning: Going Beyond Graph Data

Topological deep learning is a rapidly growing field that pertains to the development of deep learning models for data supported on topological domains such as simplicial complexes, cell complexes, and hypergraphs, which generalize many domains encountered in scientific computations. In this paper, we present a unifying deep learning framework built upon a richer data structure that includes widely adopted topological domains. Specifically, we first introduce combinatorial complexes, a novel type of topological domain. Combinatorial complexes can be seen as generalizations of graphs that maintain certain desirable properties. Similar to hypergraphs, combinatorial complexes impose no constraints on the set of relations. In addition, combinatorial complexes permit the construction of hierarchical higher-order relations, analogous to those found in simplicial and cell complexes. Thus, combinatorial complexes generalize and combine useful traits of both hypergraphs and cell complexes, which have emerged as two promising abstractions that facilitate the generalization of graph neural networks to topological spaces. Second, building upon combinatorial complexes and their rich combinatorial and algebraic structure, we develop a general class of message-passing combinatorial complex neural networks (CCNNs), focusing primarily on attention-based CCNNs. We characterize permutation and orientation equivariances of CCNNs, and discuss pooling and unpooling operations within CCNNs in detail. Third, we evaluate the performance of CCNNs on tasks related to mesh shape analysis and graph learning. Our experiments demonstrate that CCNNs have competitive performance as compared to state-of-the-art deep learning models specifically tailored to the same tasks. Our findings demonstrate the advantages of incorporating higher-order relations into deep learning models in different applications

Integration of genetic and genomics resources in einkorn wheat enables precision mapping of important traits

Einkorn wheat (Triticum monococcum) is an ancient grain crop and a close relative of the diploid progenitor (T. urartu) of polyploid wheat. It is the only diploid wheat species having both domesticated and wild forms and therefore provides an excellent system to identify domestication genes and genes for traits of interest to utilize in wheat improvement. Here, we leverage genomic advancements for einkorn wheat using an einkorn reference genome assembly combined with skim-sequencing of a large genetic population of 812 recombinant inbred lines (RILs) developed from a cross between a wild and a domesticated T. monococcum accession. We identify 15,919 crossover breakpoints delimited to a median and average interval of 114 Kbp and 219 Kbp, respectively. This high-resolution mapping resource enables us to perform fine-scale mapping of one qualitative (red coleoptile) and one quantitative (spikelet number per spike) trait, resulting in the identification of small physical intervals (400 Kb to 700 Kb) with a limited number of candidate genes. Furthermore, an important domestication locus for brittle rachis is also identified on chromosome 7A. This resource presents an exciting route to perform trait discovery in diploid wheat for agronomically important traits and their further deployment in einkorn as well as tetraploid pasta wheat and hexaploid bread wheat cultivars