Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved significant success in single-agent decision making tasks. However, it is challenging to extend Q-learning to large-scale multi-agent scenarios, due to the explosion of action space resulting from the complex dynamics between the environment and the agents. In this paper, we propose to make the computation of multi-agent Q-learning tractable by treating the Q-function (w.r.t. state and joint-action) as a high-order high-dimensional tensor and then approximate it with factorized pairwise interactions. Furthermore, we utilize a composite deep neural network architecture for computing the factorized Q-function, share the model parameters among all the agents within the same group, and estimate the agents' optimal joint actions through a coordinate descent type algorithm. All these simplifications greatly reduce the model complexity and accelerate the learning process. Extensive experiments on two different multi-agent problems demonstrate the performance gain of our proposed approach in comparison with strong baselines, particularly when there are a large number of agents.Comment: 7 pages, 5 figures, DAI 201

Fijian sea krait behavior relates to fine - scale environmental heterogeneity in old - growth coastal forest: the importance of integrated land – sea management for protecting amphibious animals

The importance of terrestrial coastal ecosystems for maintaining healthy coral reef ecosystems remains understudied. Sea kraits are amphibious snakes that require healthy coral reefs for foraging, but little is known about their requirements of terrestrial habitats, where they slough their skin, digest prey, and breed. Using concurrent microclimate measurements and behavior surveys, we show that a small, topographically flat atoll in Fiji with coastal forest provides many microhabitats that relate to the behaviors of Yellow Lipped Sea Kraits, Laticauda colubrina. Microclimates were significantly related to canopy cover, leaf litter depth, and distance from the high-water mark (HWM). Sea kraits were almost exclusively observed in coastal forest within 30 m of the HWM. Sloughing of skins only occurred within crevices of mature or dying trees. Resting L. colubrina were significantly more likely to occur at locations with higher mean diurnal temperatures, lower leaf litter depths, and shorter distances from the HWM. On Leleuvia, behavior of L. colubrina therefore relates to environmental heterogeneity created by old-growth coastal forests, particularly canopy cover and crevices in mature and dead tree trunks. The importance of healthy coastal habitats, both terrestrial and marine, for L. colubrina suggests it could be a good flagship species for advocating integrated land-sea management. Furthermore, our study highlights the importance of coastal forests and topographically flat atolls for biodiversity conservation. Effective conservation management of amphibious species that utilize land- and seascapes is therefore likely to require a holistic approach that incorporates connectivity among ecosystems and environmental heterogeneity at all relevant scales

Satellite coverage assessment considering cloud cover

Due to cloud cover, the performance provided by optical or lidar satellite systems may differ from the predicted performance. This work presents a means of rapidly assessing the coverage that can be expected from a satellite constellation considering cloud probability. A latitude-specific pseudo-circumference is defined and calculated using a cloud factor to provide insights into the relative coverage available to different regions. Comparison of the results from the presented method with simulated and historical data collected by the Sentinel-2 constellation shows the potential of the proposed method to calculate expected time between cloud free images when averaged over the long-term. Appropriate choice of cloud model is found to be an important consideration to ensure the validity of the method

Finding navigable paths through tidal flats with Synthetic Aperture Radar

Tidal flats are some of the most dynamic coastal environments in the world, where traditional coastal mapping and monitoring provide insufficient temporal resolution to reliably map channels and sand flats. Satellite-based Synthetic Aperture Radar (SAR) enables regular cloud-penetrating detection of water flowing through channels within the tidal flats, referred to as tidal channels. This paper presents a method for detecting a path through tidal channels, using satellite imagery, that supports our understanding and safe exploitation of this valuable coastal environment. This approach is the first proposed to identify navigable paths in all conditions, with SAR images susceptible to variation due to weather and tidal conditions. Tidal channels are known to vary in SAR presentation, and we find that tidal flat presentation is also influenced by conditions. The most influential factor is the wind, with high winds causing an inversion in how both tidal flats and tidal channels present in SAR images. The presented method for the automatic detection of tidal channels accounts for this variability by using previous channel paths as a reference to reliably correct imagery and detect the latest path. The final algorithm produces paths with minor errors in 17.6% of images; the error rate increases to 71.7%, with an almost tenfold increase in errors, when the SAR image and paths are not adjusted to account for conditions. This capability has been used to support the Nith Inshore Rescue in attending call-outs from their base in Glencaple, UK, while the insights from monitoring tidal channels for a year demonstrate how periods of high river flow preceded major changes in the channel path

Ichthyosis follicularis, alopecia, and photophobia (IFAP) syndrome

The IFAP syndrome is a rare X-linked genetic disorder reported in nearly 40 patients. It is characterized by the triad of Ichthyosis Follicularis, Alopecia, and Photophobia from birth. Other features such as short stature, intellectual disability, and seizures may develop in the first few years of life. Skin histopathology is non-specific and consists of dilated hair follicles with keratin plugs extending above the surface of the skin, decreased or absent sebaceous glands, and decreased desmosomes in number and size. The disorder results from mutations in the MBTPS2 gene that impairs cholesterol homeostasis and the ability to cope with endoplasmic reticulum stress. Follicular hyperkeratosis can be treated using topical keratolytics, emollients and urea preparations. A moderate response to acitretin therapy has been noted in some patients. Intensive lubrication of the ocular surface is essential. Life expectancy in patients with IFAP syndrome can vary from death in the neonatal period to normal surviving. Cardiopulmonary complications remain the major cause of death

Three red suns in the sky: A transiting, terrestrial planet in a triple M-dwarf system at 6.9 pc

We present the discovery from Transiting Exoplanet Survey Satellite (TESS) data of LTT 1445Ab. At a distance of 6.9 pc, it is the second nearest transiting exoplanet system found to date, and the closest one known for which the primary is an M dwarf. The host stellar system consists of three mid-to-late M dwarfs in a hierarchical configuration, which are blended in one TESS pixel. We use MEarth data and results from the Science Processing Operations Center data validation report to determine that the planet transits the primary star in the system. The planet has a radius of ${1.38}_{-0.12}^{+0.13}$ ${R}_{\oplus }$, an orbital period of ${5.35882}_{-0.00031}^{+0.00030}$ days, and an equilibrium temperature of ${433}_{-27}^{+28}$ K. With radial velocities from the High Accuracy Radial Velocity Planet Searcher, we place a 3σ upper mass limit of 8.4 ${M}_{\oplus }$ on the planet. LTT 1445Ab provides one of the best opportunities to date for the spectroscopic study of the atmosphere of a terrestrial world. We also present a detailed characterization of the host stellar system. We use high-resolution spectroscopy and imaging to rule out the presence of any other close stellar or brown dwarf companions. Nineteen years of photometric monitoring of A and BC indicate a moderate amount of variability, in agreement with that observed in the TESS light-curve data. We derive a preliminary astrometric orbit for the BC pair that reveals an edge-on and eccentric configuration. The presence of a transiting planet in this system hints that the entire system may be co-planar, implying that the system may have formed from the early fragmentation of an individual protostellar core.Accepted manuscrip

The effect of training in reduced energy density eating and food self-monitoring accuracy on weight loss maintenance

Background: Failure to maintain weight losses in lifestyle change programs continues to be a major problem and warrants investigation of innovative approaches to weight control.Objective: The goal of this study was to compare two novel group interventions, both aimed at improving weight loss maintenance, with a control group.Methods and Procedures: A total of 103 women lost weight on a meal replacement&ndash;supplemented diet and were then randomized to one of three conditions for the 14-week maintenance phase: cognitive-behavioral treatment (CBT); CBT with an enhanced food monitoring accuracy (EFMA) program; or these two interventions plus a reduced energy density eating (REDE) program. Assessments were conducted periodically through an 18-month postintervention. Outcome measures included weight and self-reported dietary intake. Data were analyzed using completers only as well as baseline-carried-forward imputation.Results: Participants lost an average of 7.6 plusminus 2.6 kg during the weight loss phase and 1.8 plusminus 2.3 kg during the maintenance phase. Results do not suggest that the EFMA intervention was successful in improving food monitoring accuracy. The REDE group decreased the energy density (ED) of their diets more so than the other two groups. However, neither the REDE nor the EFMA condition showed any advantage in weight loss maintenance. All groups regained weight between 6- and 18-month follow-ups.Discussion: Although no incremental weight maintenance benefit was observed in the EFMA or EFMA + REDE groups, the improvement in the ED of the REDE group\u27s diet, if shown to be sustainable in future studies, could have weight maintenance benefits.<br /