Goal-oriented Dialogue Policy Learning from Failures

Reinforcement learning methods have been used for learning dialogue policies. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the very few successful dialogues in early learning phase. Hindsight experience replay (HER) enables learning from failures, but the vanilla HER is inapplicable to dialogue learning due to the implicit goals. In this work, we develop two complex HER methods providing different trade-offs between complexity and performance, and, for the first time, enabled HER-based dialogue policy learning. Experiments using a realistic user simulator show that our HER methods perform better than existing experience replay methods (as applied to deep Q-networks) in learning rate

Data-Driven Statistical Mechanical and Symmetry Insights into Collective Coordinates in Small Optical Matter Clusters

Optical matter (OM) systems are a class of active non-equilibrium materials. One of the most interesting variants consists of nano-particles (NPs) that form 2-dimensional ordered structures when illuminated and trapped by a focused laser beam. The force field developed by the electrodynamic interactions that hold the NPs together is non-conservative. Depending on the number of NPs and the phase, amplitude and polarization properties of the incident electromagnetic field, there are several different metastable ordered structures that can be formed. The relative stabilities of these structures can be tuned by adjusting the aforementioned laser beam properties. Therefore, the beam power, beam shape, spatial phase profile, and polarization of the light create a rich parameter subspace to explore stabilization, control and design of particular OM structures and their dynamics. Each of the one or more different ordered OM structures that form in a focused laser beam constitutes a (metastable) non-equilibrium steady state (NESS), which can, for example, be used to build optical matter machines that do mechanical work under a laser beam. In order to study the mechanically dynamic and light scattering properties of OM systems, I have developed and employed a data-driven approach based on principal component analysis (PCA) and harmonic linear discriminant analysis (HLDA) to determine the collective modes of non-conservative and overdamped OM structures. The approach is demonstrated via electrodynamics-Langevin dynamics simulations six electrodynamically-bound nanoparticles coupled to an incident laser beam. I then use this data-driven approach to build the PCA-HLDA reaction coordinates between stable states connected by Markov state model (MSM), compute entropy production rate, and analyze light scattering properties as well as induced-polarization. These studies represent a systematic endeavor to understanding and eventually controlling optical matter systems. This approach is also promising to the study of other non-conservative and overdamped active matter systems

Equipment Reliability Data Collection: A Journey to Operational Excellence in the Offshore Industry

PresentationThe offshore industry has witnessed catastrophic incidents, which continue to occur. There is a need to learn from various best practices and incidents and continue to move towards safer operations. Data in different forms on equipment reliability, near misses, key performance indicators and more exists within organizations and agencies in this industry. Most of these databases, if collected and connected could be used to prevent and/or assess the consequences of an event. Near-miss databases can help to assess the barriers that would prevent an event from escalating to consequences, and the reliability databases can be used to assess the barriers that can prevent an event. This paper describes the experience, initiatives and major challenges of the Ocean Energy Safety Institute (OESI) and the Mary Kay O’Connor Process Safety Center (MKOPSC) in data collection projects and initiatives. The paper concludes with next steps to see how existing databases could be improved in areas such as data quality, data validation, increased accessibility and searchability and provides a list of potential research projects