Gradient dynamics in reinforcement learning

Despite the success achieved by the analysis of supervised learning algorithms in the framework of statistical mechanics, reinforcement learning has remained largely untouched. Here we move towards closing the gap by analyzing the dynamics of the policy gradient algorithm. For a convex problem, we show that it obeys a drift-diffusion motion with coeffcients tuned by learning rate. Furthermore, we propose a mapping between a non-convex reinforcement learning problem and a disordered system. This mapping enables us to show how the learning rate acts as an effective temperature and thus is capable of smoothing rough landscapes, corroborating what is displayed by the drift-diffusive description and paving the way for physics-inspired algorithmic optimization based on annealing procedures in disordered systems.Comment: 15 pages, 6 figures. Submitted to Physical Review

Comparison of pure and combined search strategies for single and multiple targets

We address the generic problem of random search for a point-like target on a line. Using the measures of search reliability and efficiency to quantify the random search quality, we compare Brownian search with L\'evy search based on long-tailed jump length distributions. We then compare these results with a search process combined of two different long-tailed jump length distributions. Moreover, we study the case of multiple targets located by a L\'evy searcher.Comment: 16 pages, 12 figure

Conformations and dynamics of active star polymers

We study conformations and dynamics of active star polymers. The analysis shows that active star polymers stretching behaviour is quite different from that of active linear chains. The visual inspection of conformations and bond-bond correlations reveal a better coordination for the alignment and coordination of bonds for the star polymers than for the linear counterparts. The architecture substantially affects the chain extension transition at high values of active force. The scaling laws for the shape factor and the arm asphericity ratio established for the passive star polymers coincide with the passive case for active force values below the transition. For the values above the transition range the scaling of these quantities switches to different values.Comment: 14 pages, 17 figure

Interpretation of the vibrational spectra of glassy polymers using coarse-grained simulations

The structure and vibrational density of states (VDOS) of polymer glasses are investigated using numerical simulations based on the classical Kremer-Grest bead-spring model. We focus on the roles of chain length and bending stiffness, the latter being set by imposing three-body angular potentials along chain backbones. Upon increasing the chain length and bending stiffness, structural reorganisation leads to volumetric expansion of the material and build-up of internal stresses. The VDOS has two dominant bands: a low frequency one corresponding to inter- and intra-chain non-bonding interactions and a high frequency one corresponding principally to vibrations of bonded beads that constitute skeletal chain backbones. Upon increasing the steepness of the angular potential, vibrational modes associated with chain bending gradually move from the low-frequency to the high-frequency band. This redistribution of modes is reflected in a reduction of the so-called Boson peak upon increasing chain stiffness. Remarkably, the finer structure and the peaks of the high-frequency band, and their variations with stiffness, can, for short chains, be explained using an analytical solution derived for a model triatomic molecule. For longer chains, the qualitative evolution of the VDOS with chain stiffness is similar, although the distinct peaks observed for short chains become increasingly smoothed-out. Our findings can be used to guide a systematic approach to interpretation of Brillouin and Raman scattering spectra of glassy polymers in future work, with applications in polymer processing diagnostics.Comment: To appear in Macromolecule

Parameter-free predictions of the viscoelastic response of glassy polymers from non-affine lattice dynamics

We study the viscoelastic response of amorphous polymers using theory and simulations. By accounting for internal stresses and considering instantaneous normal modes (INMs) within athermal non-affine theory, we make parameter-free predictions of the dynamic viscoelastic moduli obtained in coarse-grained simulations of polymer glasses at non-zero temperatures. The theoretical results show very good correspondence with rheology data collected from molecular dynamics simulations over five orders of magnitude in frequency, with some instabilities that accumulate in the low-frequency part on approach to the glass transition. These results provide evidence that the mechanical glass transition itself is continuous and thus represents a crossover rather than a true phase transition. The relatively sharp drop of the low-frequency storage modulus across the glass transition temperature can be explained mechanistically within the proposed theory: the proliferation of low-eigenfrequency vibrational excitations (boson peak and nearly-zero energy excitations) is directly responsible for the rapid growth of a negative non-affine contribution to the storage modulus.Comment: 10 pages, 7 figure