marl-jax: Multi-Agent Reinforcement Leaning Framework

Recent advances in Reinforcement Learning (RL) have led to many exciting applications. These advancements have been driven by improvements in both algorithms and engineering, which have resulted in faster training of RL agents. We present marl-jax, a multi-agent reinforcement learning software package for training and evaluating social generalization of the agents. The package is designed for training a population of agents in multi-agent environments and evaluating their ability to generalize to diverse background agents. It is built on top of DeepMind's JAX ecosystem~\cite{deepmind2020jax} and leverages the RL ecosystem developed by DeepMind. Our framework marl-jax is capable of working in cooperative and competitive, simultaneous-acting environments with multiple agents. The package offers an intuitive and user-friendly command-line interface for training a population and evaluating its generalization capabilities. In conclusion, marl-jax provides a valuable resource for researchers interested in exploring social generalization in the context of MARL. The open-source code for marl-jax is available at: \href{https://github.com/kinalmehta/marl-jax}{https://github.com/kinalmehta/marl-jax}Comment: Accepted at ECML-PKDD 2023 Demo Trac

Effects of Spectral Normalization in Multi-agent Reinforcement Learning

A reliable critic is central to on-policy actor-critic learning. But it becomes challenging to learn a reliable critic in a multi-agent sparse reward scenario due to two factors: 1) The joint action space grows exponentially with the number of agents 2) This, combined with the reward sparseness and environment noise, leads to large sample requirements for accurate learning. We show that regularising the critic with spectral normalization (SN) enables it to learn more robustly, even in multi-agent on-policy sparse reward scenarios. Our experiments show that the regularised critic is quickly able to learn from the sparse rewarding experience in the complex SMAC and RWARE domains. These findings highlight the importance of regularisation in the critic for stable learning

Traffic Density Management using Round Robin scheduling with Varied Time Quantum and Traffic Analysis

This paper suggests a solution to the problem of traffic management in metropolitan cities. The current technique of managing traffic uses traffic signals with fixed time cycles for each direction of traffic, without taking the amount of vehicles into consideration. This paper illustrates an adaptive system where signal cycles change based on traffic densities of each direction of vehicles. This system will calculate density using infrared sensors that will perform vehicle counting, along with road dimensions. This density is added to a relational database. Further analysis is done to calculate other important parameters that will contribute to deciding the time cycle that the signal will assign to that direction. The main aim of this study is to reduce the amount of time a vehicle has to spend at an intersection, or any such point on a road where traffic is controlled by signals and to analyze the traffic flow at a cross junction

SARS-CoV-2: comparison of IgG levels at 9 months post second dose of vaccination in COVID-survivor and COVID-naïve healthcare workers

Background: Natural (asymptomatic/symptomatic COVID-19 infection) and artificial (vaccination) exposure to the pathogen represent two modes of acquiring active immunity. No definitive guidelines exist regarding whether COVID-survivors (with infection/re-infection/re-re-infection in the three COVID-19 waves) require a modified vaccination schedule. Most countries are offering a third vaccine dose and many are contemplating a fourth dose. Our aim was to gauge the IgG-antibody levels 9m post second vaccination in healthcare workers (HCW) and compare these with IgG-levels 1m post-vaccination in the same cohort for any decline, and to compare the post-vaccination IgG-levels in COVID-survivors and COVID-naïve HCW at 9m.Methods: This prospective observational single-centric cohort study included 63 HCW of either sex, aged 18-70y who completed 9m post-vaccination. The IgG-titre was tested at 9-10m post second vaccination in COVID-survivors and COVID-naïve HCW.Results: At 1m and 9m post-vaccination IgG-levels in COVID-survivors (23.097±4.58 and 15.103±4.367 respectively; p<0.0001) and COVID-naïve HCW (16.277±6.36 and 9.793±6.928 respectively; p=0.0013) had unequal variance (Welsch test; p=0.0022 at 9m). 9/31 COVID-naïve HCW but none of the 32 COVID-survivors tested COVID-positive in the second wave post second vaccination. 11/31 and 3/32 HCW belonging to the former and latter groups developed COVID-19 in the third wave consequently deferring their third/precautionary vaccination.Conclusions: Although HCW with IgG-levels in all brackets developed COVID-19, the severity of symptoms corresponded with the IgG-levels. COVID-19 is here to stay, but in peaceful co-existence in endemic proportions. Considering evidence that immunity acquired by vaccination/natural infection is ephemeral, re-invention of vaccines to match the ever-mutating virus is foreseen.

Posterior migration of Ahmed glaucoma valve tube in a patient with Reiger anomaly: a case report

BACKGROUND: To describe, a yet non-documented complication of GDI surgery (glaucoma drainage incision surgery) - anterior to posterior segment migration of Ahmed Glaucoma Valve (AGV) tube. CASE PRESENTATION: We report a young 9 year old boy, diagnosed with refractory glaucoma with Reiger anomaly. History included of poor vision in both eyes, left more than right with glare since childhood. He underwent GDI surgery with AGV implantation following which he developed posterior migration of AGV tube. The detailed ocular history, ophthalmic findings, clinical course, surgical management and development of the posterior tube migration is discussed. CONCLUSION: Posterior Migration of AGV tube has yet not been described. Also there is a role of expectant management of the complication in this case as evidenced by the benign course of events

MRI Appearance of Florid Cystic Endosalpingiosis of the Uterus: a Case Report

Endosalpingiosis is a non-neoplastic proliferation of ectopic tubal epithelium. It may be found incidentally or the patients may present with chronic pelvic pain. It may resemble a gynecologic malignancy on imaging findings and clinicians and radiologists should be aware of this benign entity to render a correct diagnosis and to avoid over-treatment. We report here the MR imaging appearance of a case of florid cystic endosalpingiosis

Albiglutide and cardiovascular outcomes in patients with type 2 diabetes and cardiovascular disease (Harmony Outcomes): a double-blind, randomised placebo-controlled trial

Background: Glucagon-like peptide 1 receptor agonists differ in chemical structure, duration of action, and in their effects on clinical outcomes. The cardiovascular effects of once-weekly albiglutide in type 2 diabetes are unknown. We aimed to determine the safety and efficacy of albiglutide in preventing cardiovascular death, myocardial infarction, or stroke. Methods: We did a double-blind, randomised, placebo-controlled trial in 610 sites across 28 countries. We randomly assigned patients aged 40 years and older with type 2 diabetes and cardiovascular disease (at a 1:1 ratio) to groups that either received a subcutaneous injection of albiglutide (30–50 mg, based on glycaemic response and tolerability) or of a matched volume of placebo once a week, in addition to their standard care. Investigators used an interactive voice or web response system to obtain treatment assignment, and patients and all study investigators were masked to their treatment allocation. We hypothesised that albiglutide would be non-inferior to placebo for the primary outcome of the first occurrence of cardiovascular death, myocardial infarction, or stroke, which was assessed in the intention-to-treat population. If non-inferiority was confirmed by an upper limit of the 95% CI for a hazard ratio of less than 1·30, closed testing for superiority was prespecified. This study is registered with ClinicalTrials.gov, number NCT02465515. Findings: Patients were screened between July 1, 2015, and Nov 24, 2016. 10 793 patients were screened and 9463 participants were enrolled and randomly assigned to groups: 4731 patients were assigned to receive albiglutide and 4732 patients to receive placebo. On Nov 8, 2017, it was determined that 611 primary endpoints and a median follow-up of at least 1·5 years had accrued, and participants returned for a final visit and discontinuation from study treatment; the last patient visit was on March 12, 2018. These 9463 patients, the intention-to-treat population, were evaluated for a median duration of 1·6 years and were assessed for the primary outcome. The primary composite outcome occurred in 338 (7%) of 4731 patients at an incidence rate of 4·6 events per 100 person-years in the albiglutide group and in 428 (9%) of 4732 patients at an incidence rate of 5·9 events per 100 person-years in the placebo group (hazard ratio 0·78, 95% CI 0·68–0·90), which indicated that albiglutide was superior to placebo (p&lt;0·0001 for non-inferiority; p=0·0006 for superiority). The incidence of acute pancreatitis (ten patients in the albiglutide group and seven patients in the placebo group), pancreatic cancer (six patients in the albiglutide group and five patients in the placebo group), medullary thyroid carcinoma (zero patients in both groups), and other serious adverse events did not differ between the two groups. There were three (&lt;1%) deaths in the placebo group that were assessed by investigators, who were masked to study drug assignment, to be treatment-related and two (&lt;1%) deaths in the albiglutide group. Interpretation: In patients with type 2 diabetes and cardiovascular disease, albiglutide was superior to placebo with respect to major adverse cardiovascular events. Evidence-based glucagon-like peptide 1 receptor agonists should therefore be considered as part of a comprehensive strategy to reduce the risk of cardiovascular events in patients with type 2 diabetes. Funding: GlaxoSmithKline

A REVIEW PAPER ON SECURITY IN MOBILE ADHOC NETWORK

Mobile Ad-hoc network (MANET) is a collection of self configuring, multi-hop wireless network. Due to the mobility and dynamic nature of MANET, network is not secure. MANET is more vulnerable to different types of attacks and security threats because of its characteristics. A routing protocol in a mobile Ad hoc network should be against both inside and outside attackers. Most of the routing protocols in MANETs assume that all the nodes in a network will cooperate to each other while forwarding data packets to other nodes. But intermediate nodes may cause several problems like it can deny to forward the packet, can also extract useful information from the packet or may modify the content of packet. Such nodes are referred as malicious nodes. We present a survey of the main types of routing protocols and some security threats and various detection scheme against attack.. This paper also classifies several common attacks against the ad-hoc networks routing protocols based upon the techniques that could be used by attackers to exploit routing messages

Extrauterine adenomyoma with uterus like features: A rare entity presenting 17 years post hysterectomy

We report a case of a 47-year-old female (posthysterectomy) with bleeding per vaginam. Imaging studies showed a large abdomino-pelvic mass diagnosed as extrauterine adenomyoma with uterus-like features. This pathological entity is extremely uncommon with only few case reports available in the reported literature. This case is being highlighted for its rarity and to discuss the possible theories for origin of this uncommon condition