Learning for a robot:deep reinforcement learning, imitation learning, transfer learning

Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed

A towards-multidimensional screening approach to predict candidate genes of rheumatoid arthritis based on SNP, structural and functional annotations

<p>Abstract</p> <p>Background</p> <p>According to the Genetic Analysis Workshops (GAW), hundreds of thousands of SNPs have been tested for association with rheumatoid arthritis. Traditional genome-wide association studies (GWAS) have been developed to identify susceptibility genes using a "most significant SNPs/genes" model. However, many minor- or modest-risk genes are likely to be missed after adjustment of multiple testing. This screening process uses a strict selection of statistical thresholds that aim to identify susceptibility genes based only on statistical model, without considering multi-dimensional biological similarities in sequence arrangement, crystal structure, or functional categories/biological pathways between candidate and known disease genes.</p> <p>Methods</p> <p>Multidimensional screening approaches combined with traditional statistical genetics methods can consider multiple biological backgrounds of genetic mutation, structural, and functional annotations. Here we introduce a newly developed multidimensional screening approach for rheumatoid arthritis candidate genes that considers all SNPs with nominal evidence of Bayesian association (<it>BFLn > 0</it>), and structural and functional similarities of corresponding genes or proteins.</p> <p>Results</p> <p>Our multidimensional screening approach extracted all risk genes (<it>BFLn > 0</it>) by odd ratios of hypothesis H₁to H₀, and determined whether a particular group of genes shared underlying biological similarities with known disease genes. Using this method, we found 6614 risk SNPs in our Bayesian screen result set. Finally, we identified 146 likely causal genes for rheumatoid arthritis, including CD4, FGFR1, and KDR, which have been reported as high risk factors by recent studies. We must denote that 790 (96.1%) of genes identified by GWAS could not easily be classified into related functional categories or biological processes associated with the disease, while our candidate genes shared underlying biological similarities (<it>e.g</it>. were in the same pathway or GO term) and contributed to disease etiology, but where common variations in each of these genes make modest contributions to disease risk. We also found 6141 risk SNPs that were too minor to be detected by conventional approaches, and associations between 58 candidate genes and rheumatoid arthritis were verified by literature retrieved from the NCBI PubMed module.</p> <p>Conclusions</p> <p>Our proposed approach to the analysis of GAW16 data for rheumatoid arthritis was based on an underlying biological similarities-based method applied to candidate and known disease genes. Application of our method could identify likely causal candidate disease genes of rheumatoid arthritis, and could yield biological insights that not detected when focusing only on genes that give the strongest evidence by multiple testing. We hope that our proposed method complements the "most significant SNPs/genes" model, and provides additional insights into the pathogenesis of rheumatoid arthritis and other diseases, when searching datasets for hundreds of genetic variances.</p

Improved Cell-Free RNA and Protein Synthesis System

Cell-free RNA and protein synthesis (CFPS) is becoming increasingly used for protein production as yields increase and costs decrease. Advances in reconstituted CFPS systems such as the Protein synthesis Using Recombinant Elements (PURE) system offer new opportunities to tailor the reactions for specialized applications including in vitro protein evolution, protein microarrays, isotopic labeling, and incorporating unnatural amino acids. In this study, using firefly luciferase synthesis as a reporter system, we improved PURE system productivity up to 5 fold by adding or adjusting a variety of factors that affect transcription and translation, including Elongation factors (EF-Ts, EF-Tu, EF-G, and EF4), ribosome recycling factor (RRF), release factors (RF1, RF2, RF3), chaperones (GroEL/ES), BSA and tRNAs. The work provides a more efficient defined in vitro transcription and translation system and a deeper understanding of the factors that limit the whole system efficiency

Finite-temperature violation of the anomalous transverse Wiedemann-Franz law

The Wiedemann-Franz (WF) law links the ratio of electronic charge and heat conductivity to fundamental constants. It has been tested in numerous solids, but the extent of its relevance to the anomalous transverse transport, which represents the topological nature of the wave function, remains an open question. Here we present a study of anomalous transverse response in the noncollinear antiferromagnet Mn$_{3}$Ge extended from room temperature down to sub-Kelvin temperature and find that the anomalous Lorenz ratio remains close to the Sommerfeld value up to 100 K, but not above. The finite-temperature violation of the WF correlation is caused by a mismatch between the thermal and electrical summations of the Berry curvature, rather than the inelastic scattering as observed in ordinary metals. This interpretation is backed by our theoretical calculations, which reveals a competition between the temperature and the Berry curvature distribution. The accuracy of the experiment is supported by the verification of the Bridgman relation between the anomalous Ettingshausen and Nernst effects. Our results identify the anomalous Lorenz ratio as an extremely sensitive probe of Berry spectrum near the chemical potential.Comment: 9 pages,6 figures, Supplemental Material include

FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction

Click-through rate (CTR) prediction is one of the fundamental tasks for online advertising and recommendation. While multi-layer perceptron (MLP) serves as a core component in many deep CTR prediction models, it has been widely recognized that applying a vanilla MLP network alone is inefficient in learning multiplicative feature interactions. As such, many two-stream interaction models (e.g., DeepFM and DCN) have been proposed by integrating an MLP network with another dedicated network for enhanced CTR prediction. As the MLP stream learns feature interactions implicitly, existing research focuses mainly on enhancing explicit feature interactions in the complementary stream. In contrast, our empirical study shows that a well-tuned two-stream MLP model that simply combines two MLPs can even achieve surprisingly good performance, which has never been reported before by existing work. Based on this observation, we further propose feature gating and interaction aggregation layers that can be easily plugged to make an enhanced two-stream MLP model, FinalMLP. In this way, it not only enables differentiated feature inputs but also effectively fuses stream-level interactions across two streams. Our evaluation results on four open benchmark datasets as well as an online A/B test in our industrial system show that FinalMLP achieves better performance than many sophisticated two-stream CTR models. Our source code will be available at MindSpore/models.Comment: Accepted by AAAI 2023. Code available at https://xpai.github.io/FinalML

Tuning the anomalous Nernst and Hall effects with shifting the chemical potential in Fe-doped and Ni-doped Co3_3Sn2_2S2_2

Co$_3$Sn$_2$S$_2$ is believed to be a magnetic Weyl semimetal. It displays large anomalous Hall, Nernst and thermal Hall effects with a remarkably large anomalous Hall angle. Here, we present a comprehensive study of how substituting Co by Fe or Ni affects the electrical and thermoelectric transport. We find that doping alters the amplitude of the anomalous transverse coefficients. The maximum decrease in the amplitude of the low-temperature anomalous Hall conductivity $\sigma^A_{ij}$ is twofold. Comparing our results with theoretical calculations of the Berry spectrum assuming a rigid shift of the Fermi level, we find that given the modest shift in the position of the chemical potential induced by doping, the experimentally observed variation occurs five times faster than expected. Doping affects the amplitude and the sign of the anomalous Nernst coefficient. Despite these drastic changes, the amplitude of the $\alpha^A_{ij}/\sigma^A_{ij}$ ratio at the Curie temperature remains close to $\approx 0.5 k_B/e$, in agreement with the scaling relationship observed across many topological magnets.Comment: 8 pages, 9 figure

Expression profiling and integrative analysis of the CESA/CSL superfamily in rice

<p>Abstract</p> <p>Background</p> <p>The cellulose synthase and cellulose synthase-like gene superfamily (<it>CESA</it>/<it>CSL</it>) is proposed to encode enzymes for cellulose and non-cellulosic matrix polysaccharide synthesis in plants. Although the rice (<it>Oryza sativa </it>L.) genome has been sequenced for a few years, the global expression profiling patterns and functions of the <it>OsCESA</it>/<it>CSL </it>superfamily remain largely unknown.</p> <p>Results</p> <p>A total of 45 identified members of <it>OsCESA</it>/<it>CSL </it>were classified into two clusters based on phylogeny and motif constitution. Duplication events contributed largely to the expansion of this superfamily, with Cluster I and II mainly attributed to tandem and segmental duplication, respectively. With microarray data of 33 tissue samples covering the entire life cycle of rice, fairly high <it>OsCESA </it>gene expression and rather variable <it>OsCSL </it>expression were observed. While some members from each <it>CSL </it>family (<it>A1</it>, <it>C9</it>, <it>D2</it>, <it>E1</it>, <it>F6 </it>and <it>H1</it>) were expressed in all tissues examined, many of <it>OsCSL </it>genes were expressed in specific tissues (stamen and radicles). The expression pattern of <it>OsCESA</it>/<it>CSL </it>and <it>OsBC1L </it>which extensively co-expressed with <it>OsCESA</it>/<it>CSL </it>can be divided into three major groups with ten subgroups, each showing a distinct co-expression in tissues representing typically distinct cell wall constitutions. In particular, <it>OsCESA1, -3 & -8 </it>and <it>OsCESA4, -7 & -9 </it>were strongly co-expressed in tissues typical of primary and secondary cell walls, suggesting that they form as a cellulose synthase complex; these results are similar to the findings in <it>Arabidopsis</it>. <it>OsCESA5</it>/<it>OsCESA6 </it>is likely partially redundant with <it>OsCESA3 </it>for OsCESA complex organization in the specific tissues (plumule and radicle). Moreover, the phylogenetic comparison in rice, <it>Arabidopsis </it>and other species can provide clues for the prediction of orthologous gene expression patterns.</p> <p>Conclusions</p> <p>The study characterized the <it>CESA</it>/<it>CSL </it>of rice using an integrated approach comprised of phylogeny, transcriptional profiling and co-expression analyses. These investigations revealed very useful clues on the major roles of <it>CESA</it>/<it>CSL</it>, their potentially functional complement and their associations for appropriate cell wall synthesis in higher plants.</p