Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spaces

In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. A part of this effort is the policy optimisation task, which attempts to find a policy describing how to respond to humans, in the form of a function taking the current state of the dialogue and returning the response of the system. In this paper, we investigate deep reinforcement learning approaches to solve this problem. Particular attention is given to actor-critic methods, off-policy reinforcement learning with experience replay, and various methods aimed at reducing the bias and variance of estimators. When combined, these methods result in the previously proposed ACER algorithm that gave competitive results in gaming environments. These environments however are fully observable and have a relatively small action set so in this paper we examine the application of ACER to dialogue policy optimisation. We show that this method beats the current state-of-the-art in deep learning approaches for spoken dialogue systems. This not only leads to a more sample efficient algorithm that can train faster, but also allows us to apply the algorithm in more difficult environments than before. We thus experiment with learning in a very large action space, which has two orders of magnitude more actions than previously considered. We find that ACER trains significantly faster than the current state-of-the-art.Toshiba Research Europe Ltd, Cambridge Research Laboratory - RG85875 EPSRC Research Council - RG8079

Towards end-to-end multi-domain dialogue modelling

This work was funded by a Google Faculty Re- search Award (RG91111), an EPSRC studentship (RG80792), an EPSRC grant (EP/M018946/1) and by Toshiba Research Europe Ltd, Cam- bridge Research Laboratory (RG85875

Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels. However, collecting these annotations is expensive and time-consuming, holding back development in the area of dialogue modelling. In this paper, we investigate semi-supervised learning methods that are able to reduce the amount of required intermediate labelling. We find that by leveraging un-annotated data instead, the amount of turn-level annotations of dialogue state can be significantly reduced when building a neural dialogue system. Our analysis on the MultiWOZ corpus, covering a range of domains and topics, finds that annotations can be reduced by up to 30\% while maintaining equivalent system performance. We also describe and evaluate the first end-to-end dialogue model created for the MultiWOZ corpus.Comment: This article is published at EMNLP-IJCNLP 201

Theoretical and experimental investigation of thiourea derivatives: synthesis, crystal structure, in-silico and in-vitro biological evaluation

ABSTRACT. In this study, five different thiourea derivatives were synthesized from aryl amines according to the reported method. 1-Benzoyl-3-(4-methoxyphenyl)thiourea (2) was confirmed with single crystal XRD analysis while 1-benzoyl-3-phenylthiourea (1), 1-benzoyl-3-(4-hydroxyphenyl)thiourea (3), 1-benzoyl-3-(2-nitrophenyl) thiourea (4) and 1-benzoyl-3-p-tolylthiourea (5) were elucidated with FTIR and NMR techniques. The geometry optimization of the targeted molecules was accomplished with density functional theory applying B3LYP function. The experimental (XRD) and calculated (DFT) bond angles and bond lengths were compared. The frontier molecular orbitals and molecular electrostatic potential were computed to determine the charge density distribution and possible sites for electrophilic and nucleophilic reactions of the crystalline compound. The synthesized compounds were evaluated as an anti-radical scavenger and enzyme (esterases and protease) inhibitor using in-vitro models. The results confirmed that the synthesized molecules have good anti-oxidant property while a moderate enzyme inhibiting activity. Docking study was conducted with acetylcholine and butyrylcholine esterase which suggested that molecules under study have a potential to inhibit these esterases and protease enzymes. On the basis of in-vitro studies, it is concluded that compound 2 is most active against all tested assays.                     KEY WORDS: Thiourea, 2,2-Diphenyl-1-picrylhydrazyl, Enzyme inhibition, Density functional theory, Docking studies   Bull. Chem. Soc. Ethiop. 2021, 35(3), 587-600. DOI: https://dx.doi.org/10.4314/bcse.v35i3.1

Waste to biodiesel: A preliminary assessment for Saudi Arabia

This study presents a preliminary assessment of biodiesel production from waste sources available in the Kingdom of Saudi Arabia (KSA) for energy generation and solution for waste disposal issues. A case study was developed under three different scenarios: (S1) KSA population only in 2017, (S2) KSA population and pilgrims in 2017, and (S3) KSA population and pilgrims by 2030 using the fat fraction of the municipal solid waste. It was estimated that S1, S2, and S3 scenarios could produce around 1.08, 1.10 and 1.41 million tons of biodiesel with the energy potential of 43423, 43949 and 56493 TJ respectively. Furthermore, annual savings of US $55.89, 56.56 and 72.71 million can be generated from landfill diversion of food waste and added to the country's economy. However, there are challenges in commercialization of waste to biodiesel facilities in KSA, including waste collection and separation, impurities, reactor design and biodiesel quality

Synergies in the co-location of food manufacturing and biorefining

In food and drink manufacturing, costs must be relentlessly minimised because margins for most products are low. At the same time, the business case for biorefining of lignocellulosic feedstocks has been positive in only a small number of cases. Since the two industries use similar feedstocks and processing equipment, there should be potential for significant sharing of resources for economic and environmental gain, particularly with regard to energy, if they were co-located. This paper reviews the nature, issues and opportunities for this sort of resource sharing between food industries and biorefineries. It then illustrates the opportunity by modelling a food product (coffee bean roasting) co-located with lignocellulosic biorefining of its downstream by-product (spent coffee grounds) where biofuels are not the target output, identifying and evaluating the resource efficiencies and economics involved. The analysis shows that there can be significant benefits, but that the exact nature of the food and biorefinery products and the biorefining pathways are the key dependencies. Further research should produce a comprehensive league table of co-location opportunities for the benefit of both industries to enhance both their economics and their sustainability metrics through well-targeted synergies