Collaborative Adaptation: Learning to Recover from Unforeseen Malfunctions in Multi-Robot Teams

Cooperative multi-agent reinforcement learning (MARL) approaches tackle the challenge of finding effective multi-agent cooperation strategies for accomplishing individual or shared objectives in multi-agent teams. In real-world scenarios, however, agents may encounter unforeseen failures due to constraints like battery depletion or mechanical issues. Existing state-of-the-art methods in MARL often recover slowly -- if at all -- from such malfunctions once agents have already converged on a cooperation strategy. To address this gap, we present the Collaborative Adaptation (CA) framework. CA introduces a mechanism that guides collaboration and accelerates adaptation from unforeseen failures by leveraging inter-agent relationships. Our findings demonstrate that CA enables agents to act on the knowledge of inter-agent relations, recovering from unforeseen agent failures and selecting appropriate cooperative strategies.Comment: Presented at Multi-Agent Dynamic Games (MADGames) workshop at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023

Impact of Relational Networks in Multi-Agent Learning: A Value-Based Factorization View

Effective coordination and cooperation among agents are crucial for accomplishing individual or shared objectives in multi-agent systems. In many real-world multi-agent systems, agents possess varying abilities and constraints, making it necessary to prioritize agents based on their specific properties to ensure successful coordination and cooperation within the team. However, most existing cooperative multi-agent algorithms do not take into account these individual differences, and lack an effective mechanism to guide coordination strategies. We propose a novel multi-agent learning approach that incorporates relationship awareness into value-based factorization methods. Given a relational network, our approach utilizes inter-agents relationships to discover new team behaviors by prioritizing certain agents over other, accounting for differences between them in cooperative tasks. We evaluated the effectiveness of our proposed approach by conducting fifteen experiments in two different environments. The results demonstrate that our proposed algorithm can influence and shape team behavior, guide cooperation strategies, and expedite agent learning. Therefore, our approach shows promise for use in multi-agent systems, especially when agents have diverse properties.Comment: Accepted to International Conference on Decision and Control (IEEE CDC 2023

Influence of Team Interactions on Multi-Robot Cooperation: A Relational Network Perspective

Relational networks within a team play a critical role in the performance of many real-world multi-robot systems. To successfully accomplish tasks that require cooperation and coordination, different agents (e.g., robots) necessitate different priorities based on their positioning within the team. Yet, many of the existing multi-robot cooperation algorithms regard agents as interchangeable and lack a mechanism to guide the type of cooperation strategy the agents should exhibit. To account for the team structure in cooperative tasks, we propose a novel algorithm that uses a relational network comprising inter-agent relationships to prioritize certain agents over others. Through appropriate design of the team's relational network, we can guide the cooperation strategy, resulting in the emergence of new behaviors that accomplish the specified task. We conducted six experiments in a multi-robot setting with a cooperative task. Our results demonstrate that the proposed method can effectively influence the type of solution that the algorithm converges to by specifying the relationships between the agents, making it a promising approach for tasks that require cooperation among agents with a specified team structure.Comment: Accepted to Multi-Robot and Multi-Agent Systems (IEEE MRS 2023

Down Regulation of a Matrix Degrading Cysteine Protease Cathepsin L, by Acetaldehyde: Role of C/EBPα

BACKGROUND: The imbalance between extra cellular matrix (ECM) synthesis and degradation is critical aspect of various hepatic pathologies including alcohol induced liver fibrosis. This study was carried out to investigate the effect of acetaldehyde on expression of an extra cellular matrix degrading protease cathepsin L (CTSL) in HepG2 cells. METHODOLOGY AND RESULTS: We measured the enzymatic activity, protein and, mRNA levels of CTSL in acetaldehyde treated and untreated cells. The binding of CAAT enhancer binding protein α (C/EBP α) to CTSL promoter and its key role in the transcription from this promoter and conferring responsiveness to acetaldehyde was established by site directed mutagenesis, electrophoretic mobility shift assay (EMSA), chromatin immunoprecipitation (ChIP) assays and siRNA technology. Acetaldehyde treatment significantly decreased CTSL activity and protein levels in HepG2 cells. A similar decrease in the mRNA levels and promoter activity was also observed. This decrease by acetaldehyde was attributed to the fall in the liver enriched transcription factor C/EBP α levels and it's binding to the CTSL promoter. Mutagenesis of C/EBP α binding motifs revealed the key role of this factor in CTSL transcription as well as conferring responsiveness to acetaldehyde. The siRNA mediated silencing of the C/EBP α expression mimicked the effect of acetaldehyde on CTSL levels and its promoter activity. It also abolished the responsiveness of this promoter to acetaldehyde. CONCLUSION: Acetaldehyde down regulates the C/EBP α mediated CTSL expression in hepatic cell lines. The decreased expression of CTSL may at least in part contribute to ECM deposition in liver which is a hallmark of alcoholic liver fibrosis

ER Stress-Inducible Factor CHOP Affects the Expression of Hepcidin by Modulating C/EBPalpha Activity

Endoplasmic reticulum (ER) stress induces a complex network of pathways collectively termed the unfolded protein response (UPR). The clarification of these pathways has linked the UPR to the regulation of several physiological processes. However, its crosstalk with cellular iron metabolism remains unclear, which prompted us to examine whether an UPR affects the expression of relevant iron-related genes. For that purpose, the HepG2 cell line was used as model and the UPR was activated by dithiothreitol (DTT) and homocysteine (Hcys). Here, we report that hepcidin, a liver secreted hormone that shepherds iron homeostasis, exhibits a biphasic pattern of expression following UPR activation: its levels decreased in an early stage and increased with the maintenance of the stress response. Furthermore, we show that immediately after stressing the ER, the stress-inducible transcription factor CHOP depletes C/EBPα protein pool, which may in turn impact on the activation of hepcidin transcription. In the later period of the UPR, CHOP levels decreased progressively, enhancing C/EBPα-binding to the hepcidin promoter. In addition, analysis of ferroportin and ferritin H revealed that the transcript levels of these iron-genes are increased by the UPR signaling pathways. Taken together, our findings suggest that the UPR can have a broad impact on the maintenance of cellular iron homeostasis

Iron Behaving Badly: Inappropriate Iron Chelation as a Major Contributor to the Aetiology of Vascular and Other Progressive Inflammatory and Degenerative Diseases

The production of peroxide and superoxide is an inevitable consequence of aerobic metabolism, and while these particular "reactive oxygen species" (ROSs) can exhibit a number of biological effects, they are not of themselves excessively reactive and thus they are not especially damaging at physiological concentrations. However, their reactions with poorly liganded iron species can lead to the catalytic production of the very reactive and dangerous hydroxyl radical, which is exceptionally damaging, and a major cause of chronic inflammation. We review the considerable and wide-ranging evidence for the involvement of this combination of (su)peroxide and poorly liganded iron in a large number of physiological and indeed pathological processes and inflammatory disorders, especially those involving the progressive degradation of cellular and organismal performance. These diseases share a great many similarities and thus might be considered to have a common cause (i.e. iron-catalysed free radical and especially hydroxyl radical generation). The studies reviewed include those focused on a series of cardiovascular, metabolic and neurological diseases, where iron can be found at the sites of plaques and lesions, as well as studies showing the significance of iron to aging and longevity. The effective chelation of iron by natural or synthetic ligands is thus of major physiological (and potentially therapeutic) importance. As systems properties, we need to recognise that physiological observables have multiple molecular causes, and studying them in isolation leads to inconsistent patterns of apparent causality when it is the simultaneous combination of multiple factors that is responsible. This explains, for instance, the decidedly mixed effects of antioxidants that have been observed, etc...Comment: 159 pages, including 9 Figs and 2184 reference