Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning

In multi-timescale multi-agent reinforcement learning (MARL), agents interact across different timescales. In general, policies for time-dependent behaviors, such as those induced by multiple timescales, are non-stationary. Learning non-stationary policies is challenging and typically requires sophisticated or inefficient algorithms. Motivated by the prevalence of this control problem in real-world complex systems, we introduce a simple framework for learning non-stationary policies for multi-timescale MARL. Our approach uses available information about agent timescales to define a periodic time encoding. In detail, we theoretically demonstrate that the effects of non-stationarity introduced by multiple timescales can be learned by a periodic multi-agent policy. To learn such policies, we propose a policy gradient algorithm that parameterizes the actor and critic with phase-functioned neural networks, which provide an inductive bias for periodicity. The framework's ability to effectively learn multi-timescale policies is validated on a gridworld and building energy management environment.Comment: Accepted at IEEE CDC'23. 7 pages, 6 figure

Shock cooling of a red-supergiant supernova at redshift 3 in lensed images

The core-collapse supernova of a massive star rapidly brightens when a shock, produced following the collapse of its core, reaches the stellar surface. As the shock-heated star subsequently expands and cools, its early-time light curve should have a simple dependence on the progenitor's size and therefore final evolutionary state. Measurements of the progenitor's radius from early light curves exist for only a small sample of nearby supernovae, and almost all lack constraining ultraviolet observations within a day of explosion. The several-day time delays and magnifying ability of galaxy-scale gravitational lenses, however, should provide a powerful tool for measuring the early light curves of distant supernovae, and thereby studying massive stellar populations at high redshift. Here we analyse individual rest-frame ultraviolet-through-optical exposures taken with the Hubble Space Telescope that simultaneously capture, in three separate gravitationally lensed images, the early phases of a supernova at redshift $z \approx 3$ beginning within $5.8\pm 3.1$ hr of explosion. The supernova, seen at a lookback time of $\sim11.5$ billion years, is strongly lensed by an early-type galaxy in the Abell 370 cluster. We constrain the pre-explosion radius to be $533^{+154}_{-119}$ solar radii, consistent with a red supergiant. Highly confined and massive circumstellar material at the same radius can also reproduce the light curve, but is unlikely since no similar low-redshift examples are known.Comment: 69 pages, 12 figures/tables (4 main text, 8 extended data). Published in Natur

Flashlights: More than A Dozen High-Significance Microlensing Events of Extremely Magnified Stars in Galaxies at Redshifts z=0.7-1.5

Once only accessible in nearby galaxies, we can now study individual stars across much of the observable universe aided by galaxy-cluster gravitational lenses. When a star, compact object, or multiple such objects in the foreground galaxy-cluster lens become aligned, they can magnify a background individual star, and the timescale of a magnification peak can limit its size to tens of AU. The number and frequency of microlensing events therefore opens a window into the population of stars and compact objects, as well as high-redshift stars. To assemble the first statistical sample of stars in order to constrain the initial mass function (IMF) of massive stars at redshift z=0.7-1.5, the abundance of primordial black holes in galaxy-cluster dark matter, and the IMF of the stars making up the intracluster light, we are carrying out a 192-orbit program with the Hubble Space Telescope called "Flashlights," which is now two-thirds complete owing to scheduling challenges. We use the ultrawide F200LP and F350LP long-pass WFC3 UVIS filters and conduct two 16-orbit visits separated by one year. Having an identical roll angle during both visits, while difficult to schedule, yields extremely clean subtraction. Here we report the discovery of more than a dozen bright microlensing events, including multiple examples in the famous "Dragon Arc" discovered in the 1980s, as well as the "Spocks" and "Warhol" arcs that have hosted already known supergiants. The ultradeep observer-frame ultraviolet-through-optical imaging is sensitive to hot stars, which will complement deep James Webb Space Telescope infrared imaging. We are also acquiring Large Binocular Telescope LUCI and Keck-I MOSFIRE near-infrared spectra of the highly magnified arcs to constrain their recent star-formation histories

Genome-wide detection of a TFIID localization element from an initial human disease mutation

Eukaryotic core promoters are often characterized by the presence of consensus motifs such as the TATA box or initiator elements, which attract and direct the transcriptional machinery to the transcription start site. However, many human promoters have none of the known core promoter motifs, suggesting that undiscovered promoter motifs exist in the genome. We previously identified a mutation in the human Ankyrin-1 (ANK-1) promoter that causes the disease ankyrin-deficient Hereditary Spherocytosis (HS). Although the ANK-1 promoter is CpG rich, no discernable basal promoter elements had been identified. We showed that the HS mutation disrupted the binding of the transcription factor TFIID, the major component of the pre-initiation complex. We hypothesized that the mutation identified a candidate promoter element with a more widespread role in gene regulation. We examined 17 181 human promoters for the experimentally validated binding site, called the TFIID localization sequence (DLS) and found three times as many promoters containing DLS than TATA motifs. Mutational analyses of DLS sequences confirmed their functional significance, as did the addition of a DLS site to a minimal Sp1 promoter. Our results demonstrate that novel promoter elements can be identified on a genome-wide scale through observations of regulatory disruptions that cause human disease

Molecular and pathological signatures of epithelial–mesenchymal transitions at the cancer invasion front

Reduction of epithelial cell–cell adhesion via the transcriptional repression of cadherins in combination with the acquisition of mesenchymal properties are key determinants of epithelial–mesenchymal transition (EMT). EMT is associated with early stages of carcinogenesis, cancer invasion and recurrence. Furthermore, the tumor stroma dictates EMT through intensive bidirectional communication. The pathological analysis of EMT signatures is critically, especially to determine the presence of cancer cells at the resection margins of a tumor. When diffusion barriers disappear, EMT markers may be detected in sera from cancer patients. The detection of EMT signatures is not only important for diagnosis but can also be exploited to enhance classical chemotherapy treatments. In conclusion, further detailed understanding of the contextual cues and molecular mediators that control EMT will be required in order to develop diagnostic tools and small molecule inhibitors with potential clinical implications

A Universal Power-law Prescription for Variability from Synthetic Images of Black Hole Accretion Flows

We present a framework for characterizing the spatiotemporal power spectrum of the variability expected from the horizon-scale emission structure around supermassive black holes, and we apply this framework to a library of general relativistic magnetohydrodynamic (GRMHD) simulations and associated general relativistic ray-traced images relevant for Event Horizon Telescope (EHT) observations of Sgr A*. We find that the variability power spectrum is generically a red-noise process in both the temporal and spatial dimensions, with the peak in power occurring on the longest timescales and largest spatial scales. When both the time-averaged source structure and the spatially integrated light-curve variability are removed, the residual power spectrum exhibits a universal broken power-law behavior. On small spatial frequencies, the residual power spectrum rises as the square of the spatial frequency and is proportional to the variance in the centroid of emission. Beyond some peak in variability power, the residual power spectrum falls as that of the time-averaged source structure, which is similar across simulations; this behavior can be naturally explained if the variability arises from a multiplicative random field that has a steeper high-frequency power-law index than that of the time-averaged source structure. We briefly explore the ability of power spectral variability studies to constrain physical parameters relevant for the GRMHD simulations, which can be scaled to provide predictions for black holes in a range of systems in the optically thin regime. We present specific expectations for the behavior of the M87* and Sgr A* accretion flows as observed by the EHT

Impact of COVID-19 on cardiovascular testing in the United States versus the rest of the world

Objectives: This study sought to quantify and compare the decline in volumes of cardiovascular procedures between the United States and non-US institutions during the early phase of the coronavirus disease-2019 (COVID-19) pandemic. Background: The COVID-19 pandemic has disrupted the care of many non-COVID-19 illnesses. Reductions in diagnostic cardiovascular testing around the world have led to concerns over the implications of reduced testing for cardiovascular disease (CVD) morbidity and mortality. Methods: Data were submitted to the INCAPS-COVID (International Atomic Energy Agency Non-Invasive Cardiology Protocols Study of COVID-19), a multinational registry comprising 909 institutions in 108 countries (including 155 facilities in 40 U.S. states), assessing the impact of the COVID-19 pandemic on volumes of diagnostic cardiovascular procedures. Data were obtained for April 2020 and compared with volumes of baseline procedures from March 2019. We compared laboratory characteristics, practices, and procedure volumes between U.S. and non-U.S. facilities and between U.S. geographic regions and identified factors associated with volume reduction in the United States. Results: Reductions in the volumes of procedures in the United States were similar to those in non-U.S. facilities (68% vs. 63%, respectively; p = 0.237), although U.S. facilities reported greater reductions in invasive coronary angiography (69% vs. 53%, respectively; p < 0.001). Significantly more U.S. facilities reported increased use of telehealth and patient screening measures than non-U.S. facilities, such as temperature checks, symptom screenings, and COVID-19 testing. Reductions in volumes of procedures differed between U.S. regions, with larger declines observed in the Northeast (76%) and Midwest (74%) than in the South (62%) and West (44%). Prevalence of COVID-19, staff redeployments, outpatient centers, and urban centers were associated with greater reductions in volume in U.S. facilities in a multivariable analysis. Conclusions: We observed marked reductions in U.S. cardiovascular testing in the early phase of the pandemic and significant variability between U.S. regions. The association between reductions of volumes and COVID-19 prevalence in the United States highlighted the need for proactive efforts to maintain access to cardiovascular testing in areas most affected by outbreaks of COVID-19 infection