Regret Bounds for Reinforcement Learning with Policy Advice

In some reinforcement learning problems an agent may be provided with a set of input policies, perhaps learned from prior experience or provided by advisors. We present a reinforcement learning with policy advice (RLPA) algorithm which leverages this input set and learns to use the best policy in the set for the reinforcement learning task at hand. We prove that RLPA has a sub-linear regret of \tilde O(\sqrt{T}) relative to the best input policy, and that both this regret and its computational complexity are independent of the size of the state and action space. Our empirical simulations support our theoretical analysis. This suggests RLPA may offer significant advantages in large domains where some prior good policies are provided

Letter From the Editor

Welcome to the third issue of The Richmond Journal of Law and Technology\u27s seventh publication term! The 2000-2001 academic year has proved to be one of the most productive and exciting in the Journal\u27s decorated history. Our Editorial Board and staff have done a phenomenal job on the Journal\u27s seventh volume and we are very proud of the issue we have worked to prepare for you today

Student Counseling Program in Twenty Iowa Colleges

A survey was made of certain student personnel practices in twenty Iowa colleges by submitting a questionnaire and obtaining follow-up data one year later. While most colleges provide student counseling service, specific measures and procedures vary widely. There is no uniformity in regard to the testing program or the means for assisting students with How to Study problems. Generally speaking, any professional counseling effort is supplemented by adviser service rendered by regular faculty members

Aeorodynamic characteristics of an air-exchanger system for the 40- by 80-foot wind tunnel at Ames Research Center

A 1/50-scale model of the 40- by 80-Foot Wind Tunnel at Ames Research Center was used to study various air-exchange configurations. System components were tested throughout a range of parameters, and approximate analytical relationships were derived to explain the observed characteristics. It is found that the efficiency of the air exchanger could be increased (1) by adding a shaped wall to smoothly turn the incoming air downstream, (2) by changing to a contoured door at the inlet to control the flow rate, and (3) by increasing the size of the exhaust opening. The static pressures inside the circuit then remain within the design limits at the higher tunnel speeds if the air-exchange rate is about 5% or more. Since the model is much smaller than the full-scale facility, it is not possible to completely duplicate the tunnel, and it will be necessary to measure such characteristics as flow rate and tunnel pressures during implementation of the remodeled facility. The aerodynamic loads estimated for the inlet door and for nearby walls are also presented

A preconditioning scheme for minimum energy path finding methods

Popular methods for identifying transition paths between energy minima, such as the nudged elastic band and string methods, typically do not incorporate potential energy curvature information, leading to slow relaxation to the minimum energy path for typical potential energy surfaces encountered in molecular simulation. We propose a preconditioning scheme which, combined with a new adaptive time step selection algorithm, substantially reduces the computational cost of transition path finding algorithms. We demonstrate the improved performance of our approach in a range of examples including vacancy and dislocation migration modeled with both interatomic potentials and density functional theory

Sensitivity and dimensionality of atomic environment representations used for machine learning interatomic potentials

Faithfully representing chemical environments is essential for describing materials and molecules with machine learning approaches. Here, we present a systematic classification of these representations and then investigate (i) the sensitivity to perturbations and (ii) the effective dimensionality of a variety of atomic environment representations and over a range of material datasets. Representations investigated include atom centered symmetry functions, Chebyshev Polynomial Symmetry Functions (CHSF), smooth overlap of atomic positions, many-body tensor representation, and atomic cluster expansion. In area (i), we show that none of the atomic environment representations are linearly stable under tangential perturbations and that for CHSF, there are instabilities for particular choices of perturbation, which we show can be removed with a slight redefinition of the representation. In area (ii), we find that most representations can be compressed significantly without loss of precision and, further, that selecting optimal subsets of a representation method improves the accuracy of regression models built for a given dataset

Relativistic photoelectron spectra in the ionization of atoms by elliptically polarized light

Relativistic tunnel ionization of atoms by intense, elliptically polarized light is considered. The relativistic version of the Landau-Dykhne formula is employed. The general analytical expression is obtained for the relativistic photoelectron spectra. The most probable angle of electron emission, the angular distribution near this angle, the position of the maximum and the width of the energy spectrum are calculated. In the weak field limit we obtain the familiar non-relativistic results. For the case of circular polarization our analytical results are in agreement with recent derivations of Krainov [V.P. Krainov, J. Phys. B, {\bf 32}, 1607 (1999)].Comment: 8 pages, 2 figures, accepted for publication in Journal of Physics

A Posteriori Error Estimate and Adaptivity for QM/MM Models of Crystalline Defects

Hybrid quantum/molecular mechanics models (QM/MM methods) are widely used in material and molecular simulations when pure MM models cannot ensure adequate accuracy but pure QM models are computationally prohibitive. Adaptive QM/MM coupling methods feature on-the-fly classification of atoms, allowing the QM and MM subsystems to be updated as needed. The state-of-art "machine-learned interatomic potentials (MLIPs)" can be applied as the MM models for consistent QM/MM methods with rigorously justified accuracy. In this work, we propose a robust adaptive QM/MM method for practical material defect simulation, which is based on a developed residual-based error estimator. The error estimator provides both upper and lower bounds for the approximation error, demonstrating its reliability and efficiency. In particular, we introduce three minor approximations such that the error estimator can be evaluated efficiently without losing much accuracy. To update the QM/MM partitions anisotropically, a novel adaptive algorithm is proposed, where a free interface motion problem based on the proposed error estimator is solved by employing the fast marching method. We implement and validate the robustness of the adaptive algorithm on numerical simulations for various complex crystalline defects

Relativistic semiclassical approach in strong-field nonlinear photoionization

Nonlinear relativistic ionization phenomena induced by a strong laser radiation with elliptically polarization are considered. The starting point is the classical relativistic action for a free electron moving in the electromagnetic field created by a strong laser beam. The application of the relativistic action to the classical barrier-suppression ionization is briefly discussed. Further the relativistic version of the Landau-Dykhne formula is employed to consider the semiclassical sub-barrier ionization. Simple analytical expressions have been found for: (i) the rates of the strong-field nonlinear ionization including relativistic initial and final state effects; (ii) the most probable value of the components of the photoelectron final state momentum; (iii) the most probable direction of photoelectron emission and (iv) the distribution of the photoelectron momentum near its maximum value.Comment: 13 pages, 3 figures, to be published in Phys. Rev.