Understanding representation learning for deep reinforcement learning

Representation learning is essential to practical success of reinforcement learning. Through a state representation, an agent can describe its environment to efficiently explore the state space, generalize to new states and perform credit assignment from delayed feedback. These representations may be state abstractions, hand-engineered or fixed features or implied by a neural network. In this thesis, we investigate several desirable theoretical properties of state representations and, using this categorization, design novel principled RL algorithms aiming at learning these state representations at scale through deep learning. First, we consider state abstractions induced by behavioral metrics and their generalization properties. We show that supporting the continuity of the value function is central to generalization in reinforcement learning. Together with this formalization, we provide an empirical evaluation comparing various metrics and demonstrating the importance of the choice of a neighborhood in RL algorithms. Then, we draw on statistical learning theory to characterize what it means for arbitrary state features to generalize in RL. We introduce a new notion called effective dimension of a representation that drives the generalization to unseen states and demonstrate its usefulness for value-based deep reinforcement learning in Atari games. The third contribution of this dissertation is a scalable algorithm to learn a state representation from a very large number of auxiliary tasks through deep learning. It is a stochastic gradient descent method to learn the principal components of a target matrix by means of a neural network from a handful of entries. Finally, the last part presents our findings on how the state representation in reinforcement learning influences the quality of an agent’s predictions but is also shaped by these predictions. We provide a formal mathematical model for studying this phenomenon and show how these theoretical results can be leveraged to improve the quality of the learning process

The DKAP Project The Country Report of Vietnam

Viet Nam is at the beginning of the Fourth Industrial Revolution. In order to grasp the opportunities that the revolution has brought about, and to successfully build the society of digital citizens, there must be the demand of enhancing the capacity and capability for students to meet international standards in terms of Information and Communications Technology (ICT) skills. Viet Nam was selected as one of the four countries (Viet Nam, Bangladesh, Fiji, and the Republic of Korea) to join UNESCO Bangkok’s “Digital Kids Asia Pacific (DKAP)” project, a comparative cross-national study with the aim to seek the understanding and address children’s ICT practices, attitudes, behaviors, and competency levels within an educational context. Thanks to the project, the Vietnamese research team completely conducted the survey in twenty (20) schools from five (5) provinces in Viet Nam. With the data on the digital citizenship competency levels of 1,061 10th grade students, the research team discovered the valuable findings to draw an initial big picture for Vietnamese policy makers, educators, and teachers about digital citizenship competencies of 15-year-old Vietnamese students

On the regularization of solution of an inverse ultraparabolic equation associated with perturbed final data

In this paper, we study the inverse problem for a class of abstract ultraparabolic equations which is well-known to be ill-posed. We employ some elementary results of semi-group theory to present the formula of solution, then show the instability cause. Since the solution exhibits unstable dependence on the given data functions, we propose a new regularization method to stabilize the solution. then obtain the error estimate. A numerical example shows that the method is efficient and feasible. This work slightly extends to the earlier results in Zouyed et al. \cite{key-9} (2014).Comment: 19 pages, 4 figures, 1 tabl

Magneto-Optical Stern-Gerlach Effect in Atomic Ensemble

We study the birefringence of the quantized polarized light in a magneto-optically manipulated atomic ensemble as a generalized Stern-Gerlach Effect of light. To explain this engineered birefringence microscopically, we derive an effective Shr\"odinger equation for the spatial motion of two orthogonally polarized components, which behave as a spin with an effective magnetic moment leading to a Stern-Gerlach split in an nonuniform magnetic field. We show that electromagnetic induced transparency (EIT) mechanism can enhance the magneto-optical Stern-Gerlach effect of light in the presence of a control field with a transverse spatial profile and a inhomogeneous magnetic field.Comment: 7 pages, 5 figure