Analysis of non-unique solutions in mean field games

This thesis investigates cases when solutions to a mean field game (MFG) are non-unique. The symmetric Markov perfect information N-player game is considered and restricted to finite states and continuous time. The players' transitions are random with a parameter determined by their control. There is a unique joint distribution of the players for the symmetric Markov perfect equilibrium, but there can be multiple solutions to the MFG equations. This thesis focuses on understanding the behaviors of the many MFG solutions for the 2-state case. This thesis explores methods to determine which MFG solution represents the fluid limit trajectories of the N-player system for large populations. This thesis investigates the MFG map which acts on the MFG distributions and outputs a prediction of the population's distribution based on the expected response of any given player. The MFG solutions are exactly the fixed points of the MFG map. The MFG solution that approximates large population trajectories is conjectured to be the only stable point for the MFG map. There is a second concept investigated, social cost, which is the average accumulated cost per player. But as is shown, the social cost is not a good indicator of which MFG solution approximates large population trajectories. A set, called the bifurcation set, is defined by there being some possibility of multiple trajectories of a large population. Another important set is the indifference set, which indicates when the transition rate of the players to a state is positively reinforced by an increase of the empirical distribution of that state. However, numerical results are given, indicating that the fluid limit trajectory may relate to stability of the MFG map. It appears the MFG map is difficult to handle in many ways; stability of the mapping is difficult to show, even in a simple example and there are numerical anomalies such that non-fixed points appear to be numerically stable under rigorous tests

The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation

Function approximation is widely used in reinforcement learning to handle the computational difficulties associated with very large state spaces. However, function approximation introduces errors which may lead to instabilities when using approximate dynamic programming techniques to obtain the optimal policy. Therefore, techniques such as lookahead for policy improvement and m-step rollout for policy evaluation are used in practice to improve the performance of approximate dynamic programming with function approximation. We quantitatively characterize, for the first time, the impact of lookahead and m-step rollout on the performance of approximate dynamic programming (DP) with function approximation: (i) without a sufficient combination of lookahead and m-step rollout, approximate DP may not converge, (ii) both lookahead and m-step rollout improve the convergence rate of approximate DP, and (iii) lookahead helps mitigate the effect of function approximation and the discount factor on the asymptotic performance of the algorithm. Our results are presented for two approximate DP methods: one which uses least-squares regression to perform function approximation and another which performs several steps of gradient descent of the least-squares objective in each iteration.Comment: 36 pages, 4 figure

A GPU-accelerated immersive audio-visual framework for interaction with molecular dynamics using consumer depth sensors

© the Partner Organisations 2014. With advances in computational power, the rapidly growing role of computational/simulation methodologies in the physical sciences, and the development of new human-computer interaction technologies, the field of interactive molecular dynamics seems destined to expand. In this paper, we describe and benchmark the software algorithms and hardware setup for carrying out interactive molecular dynamics utilizing an array of consumer depth sensors. The system works by interpreting the human form as an energy landscape, and superimposing this landscape on a molecular dynamics simulation to chaperone the motion of the simulated atoms, affecting both graphics and sonified simulation data. GPU acceleration has been key to achieving our target of 60 frames per second (FPS), giving an extremely fluid interactive experience. GPU acceleration has also allowed us to scale the system for use in immersive 360° spaces with an array of up to ten depth sensors, allowing several users to simultaneously chaperone the dynamics. The flexibility of our platform for carrying out molecular dynamics simulations has been considerably enhanced by wrappers that facilitate fast communication with a portable selection of GPU-accelerated molecular force evaluation routines. In this paper, we describe a 360°atmospheric molecular dynamics simulation we have run in a chemistry/physics education context. We also describe initial tests in which users have been able to chaperone the dynamics of 10-alanine peptide embedded in an explicit water solvent. Using this system, both expert and novice users have been able to accelerate peptide rare event dynamics by 3-4 orders of magnitude. This journal i

Filament Depolymerization Can Explain Chromosome Pulling during Bacterial Mitosis

Chromosome segregation is fundamental to all cells, but the force-generating mechanisms underlying chromosome translocation in bacteria remain mysterious. Caulobacter crescentus utilizes a depolymerization-driven process in which a ParA protein structure elongates from the new cell pole, binds to a ParB-decorated chromosome, and then retracts via disassembly, pulling the chromosome across the cell. This poses the question of how a depolymerizing structure can robustly pull the chromosome that disassembles it. We perform Brownian dynamics simulations with a simple, physically consistent model of the ParABS system. The simulations suggest that the mechanism of translocation is “self-diffusiophoretic”: by disassembling ParA, ParB generates a ParA concentration gradient so that the ParA concentration is higher in front of the chromosome than behind it. Since the chromosome is attracted to ParA via ParB, it moves up the ParA gradient and across the cell. We find that translocation is most robust when ParB binds side-on to ParA filaments. In this case, robust translocation occurs over a wide parameter range and is controlled by a single dimensionless quantity: the product of the rate of ParA disassembly and a characteristic relaxation time of the chromosome. This time scale measures the time it takes for the chromosome to recover its average shape after it is has been pulled. Our results suggest explanations for observed phenomena such as segregation failure, filament-length-dependent translocation velocity, and chromosomal compaction

The Australasian Resuscitation In Sepsis Evaluation : fluids or vasopressors in emergency department sepsis (ARISE FLUIDS), a multi-centre observational study describing current practice in Australia and New Zealand

Objectives: To describe haemodynamic resuscitation practices in ED patients with suspected sepsis and hypotension. Methods: This was a prospective, multicentre, observational study conducted in 70 hospitals in Australia and New Zealand between September 2018 and January 2019. Consecutive adults presenting to the ED during a 30-day period at each site, with suspected sepsis and hypotension (systolic blood pressure <100 mmHg) despite at least 1000 mL fluid resuscitation, were eligible. Data included baseline demographics, clinical and laboratory variables and intravenous fluid volume administered, vasopressor administration at baseline and 6- and 24-h post-enrolment, time to antimicrobial administration, intensive care admission, organ support and in-hospital mortality. Results: A total of 4477 patients were screened and 591 were included with a mean (standard deviation) age of 62 (19) years, Acute Physiology and Chronic Health Evaluation II score 15.2 (6.6) and a median (interquartile range) systolic blood pressure of 94 mmHg (87–100). Median time to first intravenous antimicrobials was 77 min (42–148). A vasopressor infusion was commenced within 24 h in 177 (30.2%) patients, with noradrenaline the most frequently used (n = 138, 78%). A median of 2000 mL (1500–3000) of intravenous fluids was administered prior to commencing vasopressors. The total volume of fluid administered from pre-enrolment to 24 h was 4200 mL (3000–5661), with a range from 1000 to 12 200 mL. Two hundred and eighteen patients (37.1%) were admitted to an intensive care unit. Overall in-hospital mortality was 6.2% (95% confidence interval 4.4–8.5%). Conclusion: Current resuscitation practice in patients with sepsis and hypotension varies widely and occupies the spectrum between a restricted volume/earlier vasopressor and liberal fluid/later vasopressor strategy

Global Systems-Level Analysis of Hfq and SmpB Deletion Mutants in Salmonella: Implications for Virulence and Global Protein Translation

Using sample-matched transcriptomics and proteomics measurements it is now possible to begin to understand the impact of post-transcriptional regulatory programs in Enterobacteria. In bacteria post-transcriptional regulation is mediated by relatively few identified RNA-binding protein factors including CsrA, Hfq and SmpB. A mutation in any one of these three genes, csrA, hfq, and smpB, in Salmonella is attenuated for mouse virulence and unable to survive in macrophages. CsrA has a clearly defined specificity based on binding to a specific mRNA sequence to inhibit translation. However, the proteins regulated by Hfq and SmpB are not as clearly defined. Previous work identified proteins regulated by hfq using purification of the RNA-protein complex with direct sequencing of the bound RNAs and found binding to a surprisingly large number of transcripts. In this report we have used global proteomics to directly identify proteins regulated by Hfq or SmpB by comparing protein abundance in the parent and isogenic hfq or smpB mutant. From these same samples we also prepared RNA for microarray analysis to determine if alteration of protein expression was mediated post-transcriptionally. Samples were analyzed from bacteria grown under four different conditions; two laboratory conditions and two that are thought to mimic the intracellular environment. We show that mutants of hfq and smpB directly or indirectly modulate at least 20% and 4% of all possible Salmonella proteins, respectively, with limited correlation between transcription and protein expression. These proteins represent a broad spectrum of Salmonella proteins required for many biological processes including host cell invasion, motility, central metabolism, LPS biosynthesis, two-component regulatory systems, and fatty acid metabolism. Our results represent one of the first global analyses of post-transcriptional regulons in any organism and suggest that regulation at the translational level is widespread and plays an important role in virulence regulation and environmental adaptation for Salmonella

Conferring Thermostability to Mesophilic Proteins through Optimized Electrostatic Surfaces

Recently, there have been several experimental reports of proteins displaying appreciable stability gains through mutation of one or two amino acid residues. Here, we employ a simple theoretical model to quickly screen mutant structures for increased thermostability through optimization of the protein's electrostatic surface. Our results are able to reproduce the experimental observation that elimination of like-charge repulsions and creation of opposite-charge attractions on the protein surface is an efficient method to confer thermostability to a mesophilic protein. Using Poisson-Boltzmann electrostatics, we calculate relative protein stabilities for the exhaustive surface mutagenesis of the cold shock, RNase T1, and CheY proteins. Comparison with 25 experimentally characterized cold shock protein mutants reveals an average correlation of 0.86. The model is also quantitatively accurate when reproducing the experimental D49A and D49H mutant stabilities of RNase T1. This work represents the first comprehensive in silico screening of mutant candidates likely to confer thermostability to mesophilic proteins through optimization of surface electrostatics. Systematic single mutant, followed by double mutant, screening yields a limited number of mutant structures displaying significant stability gains suitable for subsequent experimental characterization