Diverse Conventions for Human-AI Collaboration

Conventions are crucial for strong performance in cooperative multi-agent games, because they allow players to coordinate on a shared strategy without explicit communication. Unfortunately, standard multi-agent reinforcement learning techniques, such as self-play, converge to conventions that are arbitrary and non-diverse, leading to poor generalization when interacting with new partners. In this work, we present a technique for generating diverse conventions by (1) maximizing their rewards during self-play, while (2) minimizing their rewards when playing with previously discovered conventions (cross-play), stimulating conventions to be semantically different. To ensure that learned policies act in good faith despite the adversarial optimization of cross-play, we introduce \emph{mixed-play}, where an initial state is randomly generated by sampling self-play and cross-play transitions and the player learns to maximize the self-play reward from this initial state. We analyze the benefits of our technique on various multi-agent collaborative games, including Overcooked, and find that our technique can adapt to the conventions of humans, surpassing human-level performance when paired with real users.Comment: 25 pages, 9 figures, 37th Conference on Neural Information Processing Systems (NeurIPS 2023

The elusive brain perivascular fibroblast: a potential role in vascular stability and homeostasis

In the brain, perivascular fibroblasts (PVFs) reside within the perivascular spaces (PVSs) of arterioles and large venules, however their physiological and pathophysiological roles remain largely unknown. PVFs express numerous extracellular matrix proteins that are found in the basement membrane and PVS surrounding large diameter vessels. PVFs are sandwiched between the mural cell layer and astrocytic endfeet, where they are poised to interact with mural cells, perivascular macrophages, and astrocytes. We draw connections between the more well-studied PVF pro-fibrotic response in ischemic injury and the less understood thickening of the vascular wall and enlargement of the PVS described in dementia and neurodegenerative diseases. We postulate that PVFs may be responsible for stability and homeostasis of the brain vasculature, and may also contribute to changes within the PVS during disease

PantheonRL: A MARL Library for Dynamic Training Interactions

We present PantheonRL, a multiagent reinforcement learning software package for dynamic training interactions such as round-robin, adaptive, and ad-hoc training. Our package is designed around flexible agent objects that can be easily configured to support different training interactions, and handles fully general multiagent environments with mixed rewards and n agents. Built on top of StableBaselines3, our package works directly with existing powerful deep RL algorithms. Finally, PantheonRL comes with an intuitive yet functional web user interface for configuring experiments and launching multiple asynchronous jobs. Our package can be found at https://github.com/Stanford-ILIAD/PantheonRL.Comment: 3 pages, 3 figures. Published in Proceedings of the 36th AAAI Conference on Artificial Intelligence (Demo Track) 202

Smart State Management

A method of smart state management to make a software stateful is disclosed. This method involves determining where to define states within a software using a classification learning mechanism

Pericyte Structural Remodeling in Cerebrovascular Health and Homeostasis

The biology of brain microvascular pericytes is an active area of research and discovery, as their interaction with the endothelium is critical for multiple aspects of cerebrovascular function. There is growing evidence that pericyte loss or dysfunction is involved in the pathogenesis of Alzheimer’s disease, vascular dementia, ischemic stroke and brain injury. However, strategies to mitigate or compensate for this loss remain limited. In this review, we highlight a novel finding that pericytes in the adult brain are structurally dynamic in vivo, and actively compensate for loss of endothelial coverage by extending their far-reaching processes to maintain contact with regions of exposed endothelium. Structural remodeling of pericytes may present an opportunity to foster pericyte-endothelial communication in the adult brain and should be explored as a potential means to counteract pericyte loss in dementia and cerebrovascular disease. We discuss the pathophysiological consequences of pericyte loss on capillary function, and the biochemical pathways that may control pericyte remodeling. We also offer guidance for observing pericytes in vivo, such that pericyte structural remodeling can be more broadly studied in mouse models of cerebrovascular disease