Toward Robust Long Range Policy Transfer

Humans can master a new task within a few trials by drawing upon skills acquired through prior experience. To mimic this capability, hierarchical models combining primitive policies learned from prior tasks have been proposed. However, these methods fall short comparing to the human's range of transferability. We propose a method, which leverages the hierarchical structure to train the combination function and adapt the set of diverse primitive polices alternatively, to efficiently produce a range of complex behaviors on challenging new tasks. We also design two regularization terms to improve the diversity and utilization rate of the primitives in the pre-training phase. We demonstrate that our method outperforms other recent policy transfer methods by combining and adapting these reusable primitives in tasks with continuous action space. The experiment results further show that our approach provides a broader transferring range. The ablation study also shows the regularization terms are critical for long range policy transfer. Finally, we show that our method consistently outperforms other methods when the quality of the primitives varies.Comment: Accepted by AAAI 202

Meteorin regulates mesendoderm development by enhancing nodal expression

During gastrulation, distinct lineage specification into three germ layers, the mesoderm, endoderm and ectoderm, occurs through an elaborate harmony between signaling molecules along the embryonic proximo-distal and anterior-posterior axes, and Nodal signaling plays a key role in the early embryonic development governing embryonic axis formation, mesoderm and endoderm specification, and left-right asymmetry determination. However, the mechanism by which Nodal expression is regulated is largely unknown. Here, we show that Meteorin regulates Nodal expression and is required for mesendoderm development. It is highly expressed in the inner cell mass of blastocysts and further in the epiblast and extra-embryonic ectoderm during gastrulation. Genetic ablation of the Meteorin gene resulted in early embryonic lethality, presumably due to impaired lineage allocation and subsequent cell accumulation. Embryoid body culture using Meteorin-null embryonic stem (ES) cells showed reduced Nodal expression and concomitant impairment of mesendoderm specification. Meteorin-null embryos displayed reduced levels of Nodal transcripts before the gastrulation stage, and impaired expression of Goosecoid, a definitive endoderm marker, during gastrulation, while the proximo-distal and anterior-posterior axes and primitive streak formation were preserved. Our results show that Meteorin is a novel regulator of Nodal transcription and is required to maintain sufficient Nodal levels for endoderm formation, thereby providing new insights in the regulation of mesendoderm allocation.open1113sciescopu

Partial correlation matrix estimation using ridge penalty followed by thresholding and re-estimation

Motivated by the problem of construction gene co-expression network, we propose a statistical framework for estimating high-dimensional partial correlation matrix by a three-step approach. We first obtain a penalized estimate of a partial correlation matrix using ridge penalty. Next we select the non-zero entries of the partial correlation matrix by hypothesis testing. Finally we reestimate the partial correlation coefficients at these non-zero entries. In the second step, the null distribution of the test statistics derived from penalized partial correlation estimates has not been established. We address this challenge by estimating the null distribution from the empirical distribution of the test statistics of all the penalized partial correlation estimates. Extensive simulation studies demonstrate the good performance of our method. Application on a yeast cell cycle gene expression data shows that our method delivers better predictions of the protein-protein interactions than the Graphic Lasso