Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning

Training a task-completion dialogue agent via reinforcement learning (RL) is costly because it requires many interactions with real users. One common alternative is to use a user simulator. However, a user simulator usually lacks the language complexity of human interlocutors and the biases in its design may tend to degrade the agent. To address these issues, we present Deep Dyna-Q, which to our knowledge is the first deep RL framework that integrates planning for task-completion dialogue policy learning. We incorporate into the dialogue agent a model of the environment, referred to as the world model, to mimic real user response and generate simulated experience. During dialogue policy learning, the world model is constantly updated with real user experience to approach real user behavior, and in turn, the dialogue agent is optimized using both real experience and simulated experience. The effectiveness of our approach is demonstrated on a movie-ticket booking task in both simulated and human-in-the-loop settings.Comment: 11 pages, 8 figures, Accepted in ACL 201

Nurse employment contracts in Chinese hospitals: impact of inequitable benefit structures on nurse and patient satisfaction

PURPOSE: Ongoing economic and health system reforms in China have transformed nurse employment in Chinese hospitals. Employment of ‘bianzhi’ nurses, a type of position with state-guaranteed lifetime employment that has been customary since 1949, is decreasing while there is an increase in the contract-based nurse employment with limited job security and reduced benefits. The consequences of inequities between the two types of nurses in terms of wages and job-related benefits are unknown. This study examined current rates of contract-based nurse employment and the effects of the new nurse contract employment strategy on nurse and patient outcomes in Chinese hospitals. METHODS: This cross-sectional study used geographically representative survey data collected from 2008 to 2010 from 181 hospitals in six provinces, two municipalities, and one autonomous region in China. Logistic regression models were used to estimate the association between contract-based nurse utilization, dissatisfaction among contract-based nurses, nurse intentions to leave their positions, and patient satisfaction, controlling for nurse, patient, and hospital characteristics. PRINCIPAL RESULTS: Hospital-level utilization of contract-based nurses varies greatly from 0 to 91%, with an average of 51%. Contract-based nurses were significantly more dissatisfied with their remuneration and benefits than ‘bianzhi’ nurses who have more job security (P <0.01). Contract-based nurses who were dissatisfied with their salary and benefits were more likely to intend to leave their current positions (P <0.01). Hospitals with high levels of dissatisfaction with salary and benefits among contract-based nurses were rated lower and less likely to be recommended by patients (P < 0.05). CONCLUSIONS: Our results suggest a high utilization of contract-based nurses in Chinese hospitals, and that the inequities in benefits between contract-based nurses and ‘bianzhi’ nurses may adversely affect both nurse and patient satisfaction in hospitals. Our study provides empirical support for the ‘equal pay for equal work’ policy emphasized by the China Ministry of Health’s recent regulations, and calls for efforts in Chinese hospitals to eliminate the disparities between ‘bianzhi’ and contract-based nurses

Experimental observation of highly anisotropic elastic properties of two-dimensional black arsenic

Anisotropic two-dimensional layered materials with low-symmetric lattices have attracted increasing attention due to their unique orientation-dependent mechanical properties. Black arsenic (b-As), with the puckered structure, exhibits extreme in-plane anisotropy in optical, electrical and thermal properties. However, experimental research on mechanical properties of b-As is very rare, although theoretical calculations predicted the exotic elastic properties of b-As, such as anisotropic Young's modulus and negative Poisson's ratio. Herein, experimental observations on highly anisotropic elastic properties of b-As were demonstrated using our developed in situ tensile straining setup based on the effective microelectromechanical system. The cyclic and repeatable load-displacement curves proved that Young's modulus along zigzag direction was ~1.6 times greater than that along armchair direction, while the anisotropic ratio of ultimate strain reached ~2.5, attributed to hinge structure in armchair direction. This study could provide significant insights to design novel anisotropic materials and explore their potential applications in nanomechanics and nanodevices.Comment: 19 pages, 5 figure

Highly Anisotropic Elastic Properties of Suspended Black Arsenic Nanoribbons

Anisotropy, as an exotic degree of freedom, enables us to discover the emergent two-dimensional (2D) layered nanomaterials with low in-plane symmetry and to explore their outstanding properties and promising applications. 2D black arsenic (b-As) with puckered structure has garnered increasing attention these years owing to its extreme anisotropy with respect to the electrical, thermal, and optical properties. However, the investigation on mechanical properties of 2D b-As is still lacking, despite much effort on theoretical simulations. Herein, we report the highly anisotropic elastic properties of suspended b-As nanoribbons via atomic force microscope-based nanoindentation. It was found that the extracted Young's modulus of b-As nanoribbons exhibits remarkable anisotropy, which approximates to 72.2 +- 5.4 and 44.3 +- 1.4 GPa along zigzag and armchair directions, respectively. The anisotropic ratio reaches up to ~ 1.6. We expect that these results could lay a solid foundation for the potential applications of 2D anisotropic nanomaterials in the next-generation nanomechanics and optoelectronics.Comment: 17 pages, 5 figure

Nociception and hypersensitivity involve distinct neurons and molecular transducers in Drosophila

Significance: Functional plasticity of the nociceptive circuit is a remarkable feature and is of clinical relevance. As an example, nociceptors lower their threshold upon tissue injury, a process known as allodynia that would facilitate healing by guarding the injured areas. However, long-lasting hypersensitivity could lead to chronic pain, a debilitating disease not effectively treated. Therefore, it is crucial to dissect the mechanisms underlying basal nociception and nociceptive hypersensitivity. In both vertebrate and invertebrate species, conserved transient receptor potential (Trp) channels are the primary transducers of noxious stimuli. Here, we provide a precedent that in Drosophila larvae, heat sensing in the nociception and hypersensitivity states is mediated by distinct heat-sensitive neurons and TrpA1 alternative isoforms

Whole-genome resequencing reveals genetic diversity and selection characteristics of dairy goat

The dairy goat is one of the earliest dairy livestock species, which plays an important role in the economic development, especially for developing countries. With the development of agricultural civilization, dairy goats have been widely distributed across the world. However, few studies have been conducted on the specific characteristics of dairy goat. In this study, we collected the whole-genome data of 89 goat individuals by sequencing 48 goats and employing 41 publicly available goats, including five dairy goat breeds (Saanen, Nubian, Alpine, Toggenburg, and Guanzhong dairy goat; n = 24, 15, 11, 6, 6), and three goat breeds (Guishan goat, Longlin goat, Yunshang Black goat; n = 6, 15, 6). Through compared the genomes of dairy goat and non-dairy goat to analyze genetic diversity and selection characteristics of dairy goat. The results show that the eight goats could be divided into three subgroups of European, African, and Chinese indigenous goat populations, and we also found that Australian Nubian, Toggenburg, and Australian Alpine had the highest linkage disequilibrium, the lowest level of nucleotide diversity, and a higher inbreeding coefficient, indicating that they were strongly artificially selected. In addition, we identified several candidate genes related to the specificity of dairy goat, particularly genes associated with milk production traits (GHR, DGAT2, ELF5, GLYCAM1, ACSBG2, ACSS2), reproduction traits (TSHR, TSHB, PTGS2, ESR2), immunity traits (JAK1, POU2F2, LRRC66). Our results provide not only insights into the evolutionary history and breed characteristics of dairy goat, but also valuable information for the implementation and improvement of dairy goat cross breeding program

Characterization and identification of the integrin family in silkworm, Bombyx mori

YesAs an important economic insect, Bombyx mori is also a useful model organism for lepidopteran insect. Integrins are evolutionarily conserved fromsponges to humans, and play vital roles inmany physiological and pathological processes. To explore their diverse functions of integrins in insect, eleven integrins including sixα and five β subunitswere cloned and characterized fromsilkworm. Our results showed that integrins fromsilkwormown more family members compared to other invertebrates. Among those α subunits, integrins α1, α2, and the other four subunits belong to PS1, PS2, and PS3 groups, respectively. The β subunits mainly gather in the insect βν group except the β1 subunit which belongs to the insect β group. Expression profiles demonstrated that the integrins exhibited distinct patterns, but were mainly expressed in hemocytes. α1 and β2 subunits are the predominant ones either in the embryogenesis or larva stages. Interestingly, integrins were significantly up-regulated after stimulated by 20-hydroxyecdysone (20-E) in vivo. These results indicate that integrins performdiverse functions in hemocytes of silkworm. Overall, our results provide a newinsight into the functional and evolutionary features of integrins.National Basic Research Programof China (No. 2012cb114603), the Research Fund for the Doctoral Program of Higher Education of China (20130182110003), the Natural Science Foundation of Chongqing (cstc2013jcyjys0007), and the Fundamental Research Funds for the Central Universities (SWU111014)