Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved significant success in single-agent decision making tasks. However, it is challenging to extend Q-learning to large-scale multi-agent scenarios, due to the explosion of action space resulting from the complex dynamics between the environment and the agents. In this paper, we propose to make the computation of multi-agent Q-learning tractable by treating the Q-function (w.r.t. state and joint-action) as a high-order high-dimensional tensor and then approximate it with factorized pairwise interactions. Furthermore, we utilize a composite deep neural network architecture for computing the factorized Q-function, share the model parameters among all the agents within the same group, and estimate the agents' optimal joint actions through a coordinate descent type algorithm. All these simplifications greatly reduce the model complexity and accelerate the learning process. Extensive experiments on two different multi-agent problems demonstrate the performance gain of our proposed approach in comparison with strong baselines, particularly when there are a large number of agents.Comment: 7 pages, 5 figures, DAI 201

Astrometry and geodesy with radio interferometry: experiments, models, results

Summarizes current status of radio interferometry at radio frequencies between Earth-based receivers, for astrometric and geodetic applications. Emphasizes theoretical models of VLBI observables that are required to extract results at the present accuracy levels of 1 cm and 1 nanoradian. Highlights the achievements of VLBI during the past two decades in reference frames, Earth orientation, atmospheric effects on microwave propagation, and relativity.Comment: 83 pages, 19 Postscript figures. To be published in Rev. Mod. Phys., Vol. 70, Oct. 199

Myosin Sequestration Regulates Sarcomere Function, Cardiomyocyte Energetics, and Metabolism, Informing the Pathogenesis of Hypertrophic Cardiomyopathy

BACKGROUND: Hypertrophic cardiomyopathy (HCM) is caused by pathogenic variants in sarcomere protein genes that evoke hypercontractility, poor relaxation, and increased energy consumption by the heart and increased patient risks for arrhythmias and heart failure. Recent studies show that pathogenic missense variants in myosin, the molecular motor of the sarcomere, are clustered in residues that participate in dynamic conformational states of sarcomere proteins. We hypothesized that these conformations are essential to adapt contractile output for energy conservation and that pathophysiology of HCM results from destabilization of these conformations. METHODS: We assayed myosin ATP binding to define the proportion of myosins in the super relaxed state (SRX) conformation or the disordered relaxed state (DRX) conformation in healthy rodent and human hearts, at baseline and in response to reduced hemodynamic demands of hibernation or pathogenic HCM variants. To determine the relationships between myosin conformations, sarcomere function, and cell biology, we assessed contractility, relaxation, and cardiomyocyte morphology and metabolism, with and without an allosteric modulator of myosin ATPase activity. We then tested whether the positions of myosin variants of unknown clinical significance that were identified in patients with HCM, predicted functional consequences and associations with heart failure and arrhythmias. RESULTS: Myosins undergo physiological shifts between the SRX conformation that maximizes energy conservation and the DRX conformation that enables cross-bridge formation with greater ATP consumption. Systemic hemodynamic requirements, pharmacological modulators of myosin, and pathogenic myosin missense mutations influenced the proportions of these conformations. Hibernation increased the proportion of myosins in the SRX conformation, whereas pathogenic variants destabilized these and increased the proportion of myosins in the DRX conformation, which enhanced cardiomyocyte contractility, but impaired relaxation and evoked hypertrophic remodeling with increased energetic stress. Using structural locations to stratify variants of unknown clinical significance, we showed that the variants that destabilized myosin conformations were associated with higher rates of heart failure and arrhythmias in patients with HCM. CONCLUSIONS: Myosin conformations establish work-energy equipoise that is essential for life-long cellular homeostasis and heart function. Destabilization of myosin energy-conserving states promotes contractile abnormalities, morphological and metabolic remodeling, and adverse clinical outcomes in patients with HCM. Therapeutic restabilization corrects cellular contractile and metabolic phenotypes and may limit these adverse clinical outcomes in patients with HCM

Gravity Recovery and Interior Laboratory (GRAIL): Extended Mission and End-Game Status

The Gravity Recovery and Interior Laboratory (GRAIL) [1], NASA s eleventh Discovery mission, successfully executed its Primary Mission (PM) in lunar orbit between March 1, 2012 and May 29, 2012. GRAIL s Extended Mission (XM) initiated on August 30, 2012 and was successfully completed on December 14, 2012. The XM provided an additional three months of gravity mapping at half the altitude (23 km) of the PM (55 km), and is providing higherresolution gravity models that are being used to map the upper crust of the Moon in unprecedented detail

One-year outcomes after low-dose intracoronary alteplase during primary percutaneous coronary intervention. The T-TIME randomized trial

No abstract available

New genetic loci implicated in fasting glucose homeostasis and their impact on type 2 diabetes risk.

Levels of circulating glucose are tightly regulated. To identify new loci influencing glycemic traits, we performed meta-analyses of 21 genome-wide association studies informative for fasting glucose, fasting insulin and indices of beta-cell function (HOMA-B) and insulin resistance (HOMA-IR) in up to 46,186 nondiabetic participants. Follow-up of 25 loci in up to 76,558 additional subjects identified 16 loci associated with fasting glucose and HOMA-B and two loci associated with fasting insulin and HOMA-IR. These include nine loci newly associated with fasting glucose (in or near ADCY5, MADD, ADRA2A, CRY2, FADS1, GLIS3, SLC2A2, PROX1 and C2CD4B) and one influencing fasting insulin and HOMA-IR (near IGF1). We also demonstrated association of ADCY5, PROX1, GCK, GCKR and DGKB-TMEM195 with type 2 diabetes. Within these loci, likely biological candidate genes influence signal transduction, cell proliferation, development, glucose-sensing and circadian regulation. Our results demonstrate that genetic studies of glycemic traits can identify type 2 diabetes risk loci, as well as loci containing gene variants that are associated with a modest elevation in glucose levels but are not associated with overt diabetes

Hundreds of variants clustered in genomic loci and biological pathways affect human height

Most common human traits and diseases have a polygenic pattern of inheritance: DNA sequence variants at many genetic loci influence the phenotype. Genome-wide association (GWA) studies have identified more than 600 variants associated with human traits, but these typically explain small fractions of phenotypic variation, raising questions about the use of further studies. Here, using 183,727 individuals, we show that hundreds of genetic variants, in at least 180 loci, influence adult height, a highly heritable and classic polygenic trait. The large number of loci reveals patterns with important implications for genetic studies of common human diseases and traits. First, the 180 loci are not random, but instead are enriched for genes that are connected in biological pathways (P = 0.016) and that underlie skeletal growth defects (P < 0.001). Second, the likely causal gene is often located near the most strongly associated variant: in 13 of 21 loci containing a known skeletal growth gene, that gene was closest to the associated variant. Third, at least 19 loci have multiple independently associated variants, suggesting that allelic heterogeneity is a frequent feature of polygenic traits, that comprehensive explorations of already-discovered loci should discover additional variants and that an appreciable fraction of associated loci may have been identified. Fourth, associated variants are enriched for likely functional effects on genes, being over-represented among variants that alter amino-acid structure of proteins and expression levels of nearby genes. Our data explain approximately 10% of the phenotypic variation in height, and we estimate that unidentified common variants of similar effect sizes would increase this figure to approximately 16% of phenotypic variation (approximately 20% of heritable variation). Although additional approaches are needed to dissect the genetic architecture of polygenic human traits fully, our findings indicate that GWA studies can identify large numbers of loci that implicate biologically relevant genes and pathways.

Genetically determined height and coronary artery disease.

BACKGROUND: The nature and underlying mechanisms of an inverse association between adult height and the risk of coronary artery disease (CAD) are unclear. METHODS: We used a genetic approach to investigate the association between height and CAD, using 180 height-associated genetic variants. We tested the association between a change in genetically determined height of 1 SD (6.5 cm) with the risk of CAD in 65,066 cases and 128,383 controls. Using individual-level genotype data from 18,249 persons, we also examined the risk of CAD associated with the presence of various numbers of height-associated alleles. To identify putative mechanisms, we analyzed whether genetically determined height was associated with known cardiovascular risk factors and performed a pathway analysis of the height-associated genes. RESULTS: We observed a relative increase of 13.5% (95% confidence interval [CI], 5.4 to 22.1; P<0.001) in the risk of CAD per 1-SD decrease in genetically determined height. There was a graded relationship between the presence of an increased number of height-raising variants and a reduced risk of CAD (odds ratio for height quartile 4 versus quartile 1, 0.74; 95% CI, 0.68 to 0.84; P<0.001). Of the 12 risk factors that we studied, we observed significant associations only with levels of low-density lipoprotein cholesterol and triglycerides (accounting for approximately 30% of the association). We identified several overlapping pathways involving genes associated with both development and atherosclerosis. CONCLUSIONS: There is a primary association between a genetically determined shorter height and an increased risk of CAD, a link that is partly explained by the association between shorter height and an adverse lipid profile. Shared biologic processes that determine achieved height and the development of atherosclerosis may explain some of the association. (Funded by the British Heart Foundation and others.).Supported by the British Heart Foundation, the United Kingdom National Institute for Health Research, the European Union project CVgenes@target, and a grant from the Leducq Foundation.This is the final published version. It first appeared at http://www.nejm.org/doi/full/10.1056/NEJMoa1404881

Fused Traditional and Geometric Morphometrics Demonstrate Pinniped Whisker Diversity

Vibrissae (whiskers) are important components of the mammalian tactile sensory system, and primarily function as detectors of vibrotactile information from the environment. Pinnipeds possess the largest vibrissae among mammals and their vibrissal hair shafts demonstrate a diversity of shapes. The vibrissae of most phocid seals exhibit a beaded morphology with repeating sequences of crests and troughs along their length. However, there are few detailed analyses of pinniped vibrissal morphology, and these are limited to a few species. Therefore, we comparatively characterized differences in vibrissal hair shaft morphologies among phocid species with a beaded profile, phocid species with a smooth profile, and otariids with a smooth profile using traditional and geometric morphometric methods. Traditional morphometric measurements (peak-to-peak distance, crest width, trough width and total length) were collected using digital photographs. Elliptic Fourier analysis (geometric morphometrics) was used to quantify the outlines of whole vibrissae. The traditional and geometric morphometric datasets were subsequently combined by mathematically scaling each to true rank, followed by a single eigendecomposition. Quadratic discriminant function analysis demonstrated that 79.3, 97.8 and 100% of individuals could be correctly classified to their species based on vibrissal shape variables in the traditional, geometric and combined morphometric analyses, respectively. Phocids with beaded vibrissae, phocids with smooth vibrissae, and otariids each occupied distinct morphospace in the geometric morphometric and combined data analyses. Otariids split into two groups in the geometric morphometric analysis and gray seals appeared intermediate between beaded- and smooth-whiskered species in the traditional and combined analyses. Vibrissal hair shafts modulate the transduction of environmental stimuli to the mechanoreceptors in the follicle-sinus complex (F-SC), which results in vibrotactile reception, but it is currently unclear how the diversity of shapes affects environmental signal modulation