Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.Comment: This paper is accepted by ECML-PKDD 201

Engineering chromium related single photon emitters in single crystal diamond

Color centers in diamond as single photon emitters, are leading candidates for future quantum devices due to their room temperature operation and photostability. The recently discovered chromium related centers are particularly attractive since they possess narrow bandwidth emission and a very short lifetime. In this paper we investigate the fabrication methodologies to engineer these centers in monolithic diamond. We show that the emitters can be successfully fabricated by ion implantation of chromium in conjunction with oxygen or sulfur. Furthermore, our results indicate that the background nitrogen concentration is an important parameter, which governs the probability of success to generate these centers.Comment: 14 pages, 5 figure

S1×S2S^1 \times S^2 wormholes and topological charge

I investigate solutions to the Euclidean Einstein-matter field equations with topology $S^1 \times S^2 \times R$ in a theory with a massless periodic scalar field and electromagnetism. These solutions carry winding number of the periodic scalar as well as magnetic flux. They induce violations of a quasi-topological conservation law which conserves the product of magnetic flux and winding number on the background spacetime. I extend these solutions to a model with stable loops of superconducting cosmic string, and interpret them as contributing to the decay of such loops.Comment: 18 pages (includes 6 figs.), harvmac and epsf, CU-TP-62

A Study of Obscuration in Catadioptric Lenses

In this paper we will examine the effect of obscuration upon the various features we desired to image with a 157nm microstepper utilising a catadioptric lens. We will show the effect the obscuration has upon imaging when using not only conventional illumination and binary masks, but also when using a range of enhancement techniques such as off-axis illumination and phase-shifting masks. We will show how use of a large obscuration, whilst enhancing the signals for the densest features, actually degrades the signal for more isolated features. The level of obscuration must also take into account cross duty-ratio effects, i.e. the distribution of diffraction energy, for phase shifted features of various sizes. In this situation where a small sigma would be used a large level of obscuration can significantly increase biases. The choice of obscuration can have a major effect upon the imaging capabilities of a tool. In future, when the use of catadioptric lenses may be more widespread (for example this may happen at 157nm) it may be desirable to have the option to vary this obscuration dependant upon the pattern being imaged

Serious Games Application for Memory Training Using Egocentric Images

Mild cognitive impairment is the early stage of several neurodegenerative diseases, such as Alzheimer's. In this work, we address the use of lifelogging as a tool to obtain pictures from a patient's daily life from an egocentric point of view. We propose to use them in combination with serious games as a way to provide a non-pharmacological treatment to improve their quality of life. To do so, we introduce a novel computer vision technique that classifies rich and non rich egocentric images and uses them in serious games. We present results over a dataset composed by 10,997 images, recorded by 7 different users, achieving 79% of F1-score. Our model presents the first method used for automatic egocentric images selection applicable to serious games.Comment: 11 page

The Potential and Challenges of CAD with Equational Constraints for SC-Square

Cylindrical algebraic decomposition (CAD) is a core algorithm within Symbolic Computation, particularly for quantifier elimination over the reals and polynomial systems solving more generally. It is now finding increased application as a decision procedure for Satisfiability Modulo Theories (SMT) solvers when working with non-linear real arithmetic. We discuss the potentials from increased focus on the logical structure of the input brought by the SMT applications and SC-Square project, particularly the presence of equational constraints. We also highlight the challenges for exploiting these: primitivity restrictions, well-orientedness questions, and the prospect of incrementality.Comment: Accepted into proceedings of MACIS 201