Goal-oriented Dialogue Policy Learning from Failures

Reinforcement learning methods have been used for learning dialogue policies. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the very few successful dialogues in early learning phase. Hindsight experience replay (HER) enables learning from failures, but the vanilla HER is inapplicable to dialogue learning due to the implicit goals. In this work, we develop two complex HER methods providing different trade-offs between complexity and performance, and, for the first time, enabled HER-based dialogue policy learning. Experiments using a realistic user simulator show that our HER methods perform better than existing experience replay methods (as applied to deep Q-networks) in learning rate

A Comparative study of Chinese and American address terms

In cross-cultural situations, choices of address terms often reflect cultural differences. Although a good number of studies have discussed address terms in mono-linguistic settings, literature directly related to cross-cultural address terms is scarce. The current study intends to investigate common forms of address terms in Chinese and American cultures. Two hypotheses are examined: 1) Differences between Americans and Chinese in their choices of address terms are governed by cultural norms such as politeness, as well as by contexts or styles, and 2) The Chinese students in the U.S., who are undergoing the process of assimilation and acculturation, tend to accommodate the American culture and be more like the Americans in their choices of address terms. Twenty-seven American and 24 Chinese subjects completed a 12-item survey. Data was analyzed by descriptive statistics and visual presentations and through the Kolmogorov-Smimov tests of population difference. The results indicate that while most American respondents tend to use either first name or no name in most informal settings or status conscious settings, Chinese respondents under the context in China would use more diversified choices. In addition, acculturation plays a role in Chinese respondents’ language change in terms of the choices of address terms. The relationship between age and the choice of address terms is also discussed

The role of human serum carnosinase-1 in diabetic nephropathy

Diabetische nephropaty (DN), een nierziekte die ontstaat ten gevolge van diabetes, is de meest voorkomende oorzaak voor nierfalen. Uit verschillende genetische studies is gebleken dat niet alle patienten met diabetes even gevoelig zijn voor het ontstaan van DN, maar dat dit ten dele wordt bepaald door de genetische compositie van de betroffen person. De gevoeligheid om DN te ontwikkelen berust hierbij op zogenaamde variatie in de sequentie van bepaalde genen. Deze genen worden ook wel “susceptibility genes” genoemd. Tot nu toe zijn er uit de literatuur verscheidene van zulke DN susceptibility genes bekend, één van deze is het CNDP1 gen. Het CNDP1 gen codeert voor het enzyme carnosinase (ook wel CN-1 genoemd) hetgeen vooral in de lever wordt geproduceerd en in het bloed uitgescheiden. Hoewel het nog niet geheel duidelijk is waarom de genvariatie van CNDP1 betrokken is bij de gevoeligheid om DN te ontwikkelen, is inmiddels wel aangetoond dat personen die 2 kopieen van het zogenaamde CNDP1 Mannheim allel hebben (homozygoot), een geringere concentratie van CN-1 in serum hebben en bovendien een geringere kans hebben om DN te ontwikkelen. In de studies die in dit proefschrift zijn beschreven, is getracht deze associatie verder te onderzoeken met betrekking tot geslacht (is de associatie sterker bij mannen of vrouwen), en of CN-1 concentraties in serum van diabetische patienten die homozygoot zijn voor het Mannheim allel maar toch DN ontwikkelen, hoger is als bij patienten die dit niet ontwikkelen. Tevens zijn er er ook in vitro en in vivo studies gedaan om de beschermende rol van carnosine bij het onstaan van diabetisch complicaties te onderzoeke

Learning and Reasoning for Robot Sequential Decision Making under Uncertainty

Robots frequently face complex tasks that require more than one action, where sequential decision-making (SDM) capabilities become necessary. The key contribution of this work is a robot SDM framework, called LCORPP, that supports the simultaneous capabilities of supervised learning for passive state estimation, automated reasoning with declarative human knowledge, and planning under uncertainty toward achieving long-term goals. In particular, we use a hybrid reasoning paradigm to refine the state estimator, and provide informative priors for the probabilistic planner. In experiments, a mobile robot is tasked with estimating human intentions using their motion trajectories, declarative contextual knowledge, and human-robot interaction (dialog-based and motion-based). Results suggest that, in efficiency and accuracy, our framework performs better than its no-learning and no-reasoning counterparts in office environment.Comment: In proceedings of 34th AAAI conference on Artificial Intelligence, 202

KR3^3: An Architecture for Knowledge Representation and Reasoning in Robotics

This paper describes an architecture that combines the complementary strengths of declarative programming and probabilistic graphical models to enable robots to represent, reason with, and learn from, qualitative and quantitative descriptions of uncertainty and knowledge. An action language is used for the low-level (LL) and high-level (HL) system descriptions in the architecture, and the definition of recorded histories in the HL is expanded to allow prioritized defaults. For any given goal, tentative plans created in the HL using default knowledge and commonsense reasoning are implemented in the LL using probabilistic algorithms, with the corresponding observations used to update the HL history. Tight coupling between the two levels enables automatic selection of relevant variables and generation of suitable action policies in the LL for each HL action, and supports reasoning with violation of defaults, noisy observations and unreliable actions in large and complex domains. The architecture is evaluated in simulation and on physical robots transporting objects in indoor domains; the benefit on robots is a reduction in task execution time of 39% compared with a purely probabilistic, but still hierarchical, approach.Comment: The paper appears in the Proceedings of the 15th International Workshop on Non-Monotonic Reasoning (NMR 2014