275 research outputs found
Diverse Conventions for Human-AI Collaboration
Conventions are crucial for strong performance in cooperative multi-agent
games, because they allow players to coordinate on a shared strategy without
explicit communication. Unfortunately, standard multi-agent reinforcement
learning techniques, such as self-play, converge to conventions that are
arbitrary and non-diverse, leading to poor generalization when interacting with
new partners. In this work, we present a technique for generating diverse
conventions by (1) maximizing their rewards during self-play, while (2)
minimizing their rewards when playing with previously discovered conventions
(cross-play), stimulating conventions to be semantically different. To ensure
that learned policies act in good faith despite the adversarial optimization of
cross-play, we introduce \emph{mixed-play}, where an initial state is randomly
generated by sampling self-play and cross-play transitions and the player
learns to maximize the self-play reward from this initial state. We analyze the
benefits of our technique on various multi-agent collaborative games, including
Overcooked, and find that our technique can adapt to the conventions of humans,
surpassing human-level performance when paired with real users.Comment: 25 pages, 9 figures, 37th Conference on Neural Information Processing
Systems (NeurIPS 2023
The elusive brain perivascular fibroblast: a potential role in vascular stability and homeostasis
In the brain, perivascular fibroblasts (PVFs) reside within the perivascular spaces (PVSs) of arterioles and large venules, however their physiological and pathophysiological roles remain largely unknown. PVFs express numerous extracellular matrix proteins that are found in the basement membrane and PVS surrounding large diameter vessels. PVFs are sandwiched between the mural cell layer and astrocytic endfeet, where they are poised to interact with mural cells, perivascular macrophages, and astrocytes. We draw connections between the more well-studied PVF pro-fibrotic response in ischemic injury and the less understood thickening of the vascular wall and enlargement of the PVS described in dementia and neurodegenerative diseases. We postulate that PVFs may be responsible for stability and homeostasis of the brain vasculature, and may also contribute to changes within the PVS during disease
PantheonRL: A MARL Library for Dynamic Training Interactions
We present PantheonRL, a multiagent reinforcement learning software package
for dynamic training interactions such as round-robin, adaptive, and ad-hoc
training. Our package is designed around flexible agent objects that can be
easily configured to support different training interactions, and handles fully
general multiagent environments with mixed rewards and n agents. Built on top
of StableBaselines3, our package works directly with existing powerful deep RL
algorithms. Finally, PantheonRL comes with an intuitive yet functional web user
interface for configuring experiments and launching multiple asynchronous jobs.
Our package can be found at https://github.com/Stanford-ILIAD/PantheonRL.Comment: 3 pages, 3 figures. Published in Proceedings of the 36th AAAI
Conference on Artificial Intelligence (Demo Track) 202
Smart State Management
A method of smart state management to make a software stateful is disclosed. This method involves determining where to define states within a software using a classification learning mechanism
Pericyte Structural Remodeling in Cerebrovascular Health and Homeostasis
The biology of brain microvascular pericytes is an active area of research and discovery, as their interaction with the endothelium is critical for multiple aspects of cerebrovascular function. There is growing evidence that pericyte loss or dysfunction is involved in the pathogenesis of Alzheimer’s disease, vascular dementia, ischemic stroke and brain injury. However, strategies to mitigate or compensate for this loss remain limited. In this review, we highlight a novel finding that pericytes in the adult brain are structurally dynamic in vivo, and actively compensate for loss of endothelial coverage by extending their far-reaching processes to maintain contact with regions of exposed endothelium. Structural remodeling of pericytes may present an opportunity to foster pericyte-endothelial communication in the adult brain and should be explored as a potential means to counteract pericyte loss in dementia and cerebrovascular disease. We discuss the pathophysiological consequences of pericyte loss on capillary function, and the biochemical pathways that may control pericyte remodeling. We also offer guidance for observing pericytes in vivo, such that pericyte structural remodeling can be more broadly studied in mouse models of cerebrovascular disease
- …