Action selection in modular reinforcement learning

textModular reinforcement learning is an approach to resolve the curse of dimensionality problem in traditional reinforcement learning. We design and implement a modular reinforcement learning algorithm, which is based on three major components: Markov decision process decomposition, module training, and global action selection. We define and formalize module class and module instance concepts in decomposition step. Under our framework of decomposition, we train each modules efficiently using SARSA($\lambda$) algorithm. Then we design, implement, test, and compare three action selection algorithms based on different heuristics: Module Combination, Module Selection, and Module Voting. For last two algorithms, we propose a method to calculate module weights efficiently, by using standard deviation of Q-values of each module. We show that Module Combination and Module Voting algorithms produce satisfactory performance in our test domain.Computer Science

Molecular beam optical study of gold sulfide and gold oxide

Gold-sulfur and gold-oxygen bonds are key components to numerous established and emerging technologies that have applications as far ranging as medical imaging, catalysis, electronics, and material science. A major theoretical challenge for describing this bonding is correctly accounting for the large relativistic and electron correlation effects. Such effects are best studied in diatomic, AuX, molecules. Recently, the observed AuS electronic state energy ordering was measured and compared to a simple molecular orbital diagram prediction\footnote{D. L. Kokkin, R. Zhang, T. C. Steimle, I. A. Wyse, B. W. Pearlman and T. D. Varberg, \textit {J. Phys. Chem. A.}, {\textbf{119(48)}}, 4412, 2015.}. Here we more thoroughly investigate the nature of the electronic states of both AuS and AuO from the analysis of high-resolution (FWHM$\cong35$MHz) optical Zeeman spectroscopy of the (0,0)\textit {B}$^{2}$$\Sigma$$^{-}$$-$\textit {X}$^{2}$$\Pi$$_{3/2}$ bands. The determined fine and hyperfine parameters for the \textit {B}$^{2}$$\Sigma$$^{-}$ state of AuO differ from those extracted from the analysis of a hot, Doppler-limited, spectrum\footnote{L. C. O'Brien, B. A. Borchert, A. Farquhar, S. Shaji, J. J. O'Brien and R. W. Field, \textit {J. Mol. Spectrosc.}, {\textbf{252(2)}}, 136, 2008.}. It is demonstrated that the nature of the \textit {B}$^{2}$$\Sigma$$^{-}$ states of AuO and AuS are radically different. The magnetic tuning of AuO and AuS indicates that the \textit {B}$^{2}$$\Sigma$$^{-}$ states are heavily contaminated

The PERMANENT ELECTRIC DIPOLE MOMENT AND HYPERFINE INTERACTION IN GOLD SULFIDE, AuS

The bonding and electrostatic properties of gold containing molecules are highly influenced by the large relativistic and electron correlation effects.footnote{P. Pyykko; textit {Angew Chem. Int[43] }, {textbf{4412}},(2004).} Here we report on the electricpermanent dipole moment measurement and hyperfine interaction analysis of the $^{2}$$Delta$$_{3/2}$-$^{2}$$Pi$$_{3/2}$ and $^{2}$$Delta$$_{5/2}$-$^{2}$$Pi$$_{3/2}$ bands of AuS. A cold molecular beam sample of gold sulfide was generated using a supersonic laser ablation source. The electronic bands were recorded at high resolution (35 MHz, FWHM) using laser excitation spectroscopy both field-free and in the presence of a static electric field. The observed hyperfine spectral features were assigned and a set of spectroscopic parameters for the $^{2}$$Delta$ and $^{2}$$Pi$ states were obtained. The Stark induced shifts of selected low-rotational features were analyzed to determine the permanent electric dipole moments in both the ground and excited states

A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions

Traditional recommendation systems are faced with two long-standing obstacles, namely, data sparsity and cold-start problems, which promote the emergence and development of Cross-Domain Recommendation (CDR). The core idea of CDR is to leverage information collected from other domains to alleviate the two problems in one domain. Over the last decade, many efforts have been engaged for cross-domain recommendation. Recently, with the development of deep learning and neural networks, a large number of methods have emerged. However, there is a limited number of systematic surveys on CDR, especially regarding the latest proposed methods as well as the recommendation scenarios and recommendation tasks they address. In this survey paper, we first proposed a two-level taxonomy of cross-domain recommendation which classifies different recommendation scenarios and recommendation tasks. We then introduce and summarize existing cross-domain recommendation approaches under different recommendation scenarios in a structured manner. We also organize datasets commonly used. We conclude this survey by providing several potential research directions about this field