Feedback-Based Self-Learning in Large-Scale Conversational AI Agents

Today, most large-scale conversational AI agents (e.g. Alexa, Siri, or Google Assistant) are built using manually annotated data to train the different components of the system. Typically, the accuracy of the ML models in these components are improved by manually transcribing and annotating data. As the scope of these systems increase to cover more scenarios and domains, manual annotation to improve the accuracy of these components becomes prohibitively costly and time consuming. In this paper, we propose a system that leverages user-system interaction feedback signals to automate learning without any manual annotation. Users here tend to modify a previous query in hopes of fixing an error in the previous turn to get the right results. These reformulations, which are often preceded by defective experiences caused by errors in ASR, NLU, ER or the application. In some cases, users may not properly formulate their requests (e.g. providing partial title of a song), but gleaning across a wider pool of users and sessions reveals the underlying recurrent patterns. Our proposed self-learning system automatically detects the errors, generate reformulations and deploys fixes to the runtime system to correct different types of errors occurring in different components of the system. In particular, we propose leveraging an absorbing Markov Chain model as a collaborative filtering mechanism in a novel attempt to mine these patterns. We show that our approach is highly scalable, and able to learn reformulations that reduce Alexa-user errors by pooling anonymized data across millions of customers. The proposed self-learning system achieves a win/loss ratio of 11.8 and effectively reduces the defect rate by more than 30% on utterance level reformulations in our production A/B tests. To the best of our knowledge, this is the first self-learning large-scale conversational AI system in production.Comment: 8 pages, 2 figure

In Vivo Quantification and Mathematical Description of Osteogenesis in Tissue Engineering Scaffold

INTRODUCTION Developing a successful bone tissue engineering strategy entails translation of experimental findings to clinical needs. A major leap forward towards this goal is a quantitative tool to predict spatial and temporal bone formation in scaffold. We hypothesized that bone formation in an osteoconductive scaffold follows diffusion phenomenon. In order to identify the proposed model, we implanted PLA/β-TCP scaffolds in distal femur of rats and measured bone formation using longitudinal micro-CT imaging. We then validated the proposed model using two other published in vivo models

In vivo cyclic loading as a potent stimulatory signal for bone formation inside tissue engineering scaffolds

In clinical situations, bone defects are often located at load bearing sites. Tissue engineering scaffolds are future bone substitutes and hence they will be subjected to mechanical stimulation. The goal of this study was to test if cyclic loading can be used as stimulatory signal for bone formation in a bone scaffold. PLA/ 5% β-TCP scaffolds were implanted in both distal femoral epiphyses of eight rats. Right knees were stimulated (10N, 4Hz, 5min) five times, every two days, starting from the third day after surgery while left knees served as control. Finite element study of the in vivo model showed that the strain applied to the scaffold is similar to physiological strains. Using micro-CT, all knees were scanned five times after the surgery and the related bone parameters of the newly formed bone were quantified. Statistical modeling was used to estimate the evolution of these parameters as a function of time and loading. The results showed that mechanical stimulation had two effects on bone volume (BV): an initial decrease in BV at week 2, and a long-term increase in the rate of bone formation by 28%. At week 13, the BV was then significantly higher in the loaded scaffolds

In vivo assessment of local effects after application of bone screws delivering bisphosphonates into a compromised cancellous bone site

Background: The primary stability of cancellous screw is difficult to obtain in bone of compromised quality and failure of screw fixation is common. To overcome this problem, it is proposed to locally deliver bisphosphonate from the screw An in vivo validation of the increase cancellous screw fixation is then needed in compromised bone. Methods: In this study, we used an overdrilling procedure, which enables consistent modelling of reduced screw stability comparable to compromised cancellous bone. Forty eight adult NZW rabbits were used in this study and all animals underwent bilateral femur implantation. One leg was implanted with the screw containing the bisphosphonate (biocoated group) while the other was used as control (control group) with the screw only. Mechanical testing and micro-CT imaging was used to assess the effect of local drug delivery of Zoledronate on screws fixation at 5 time points. Findings: At the early time points (1, 5, and 10 days), no significant difference could be seen between the biocoated and control groups. At 6 weeks, the bone volume fraction was significantly higher in the trabecular region of the biocoated group. However, this increase did not have a significant effect on the pull-out force. At the last time point, 11 weeks, both the bone volume fraction and the pull-out force were significantly higher in the biocoated group. Interpretation: The results of this study suggest that, in compromised bone, local delivery of bisphosphonate enhances the stability of bone screws

Biomechanical evaluation of porous biodegradable scaffolds for revision knee arthroplasty

Tibial bone defect is a critical problem for revision knee arthroplasty. Instead of using metallic spacer or cement, biodegradable scaffolds could be an alternative solution. A numerical model of a revision knee arthroplasty was thus developed to estimate the mechanical resistance of the scaffold in this demanding situation. The tibia, scaffold, and prosthesis were represented by simplified parameterised geometries. The maximal gait cycle force was applied asymmetrically to simulate a critical loading. Several parameters were analysed: 1) inter-individual variability, 2) cortical bone stiffness, 3) cortical bone thickness, 4) prosthesis fixation quality, and 5) scaffold thickness. The calculated scaffold strain was compared to its experimental ultimate strain. Among the tested parameters, failure was only predicted with scaffold thickness below 5 mm. This study suggests that biodegradable bone scaffolds could be used to fill bone defects in revision knee arthroplasty, but scaffold size seems to be the limiting factor

Constructing 'homeland' and 'target' cities in the 'war on terror'.

No abstract available for this item

Monitoring bone fetal cells seeded in PLA scaffolds using bioluminescence imaging

In our laboratory, we have shown previously that bone fetal cells enhance osteogenesis in polymeric scaffolds. However, little is known about the longevity of these cells in vivo, and hence their function. A successful tissue engineering approach needs understanding the role osteoinductive cells as well as optimization of their durability in the scaffold. Bioluminescence imaging (BLI) is a potent non-invasive imaging procedure which allows continuous assessing of the behavior of cells in vivo. A collaboration has been made with University Hospital of Geneva, where they have successfully developed a technique for screening myoblasts in vivo using BLI. The goal of this project is to screen the bone fetal cells seeded in polymeric scaffolds in vitro and in vivo using BLI