Iterative Policy-Space Expansion in Reinforcement Learning

Humans and animals solve a difficult problem much more easily when they are presented with a sequence of problems that starts simple and slowly increases in difficulty. We explore this idea in the context of reinforcement learning. Rather than providing the agent with an externally provided curriculum of progressively more difficult tasks, the agent solves a single task utilizing a decreasingly constrained policy space. The algorithm we propose first learns to categorize features into positive and negative before gradually learning a more refined policy. Experimental results in Tetris demonstrate superior learning rate of our approach when compared to existing algorithms.Comment: Workshop on Biological and Artificial Reinforcement Learning at the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canad

Role of Iron on the Structure and Stability of Ni3.2Fe/Al2O3 during Dynamic CO2 Methanation for P2X Applications

An energy scenario, mainly based on renewables, requires efficient and flexible Power-to-X (P2X) storage technologies, including the methanation of CO₂. As active Ni⁰ surface sites of monometallic nickel-based catalysts are prone to surface oxidation under hydrogen-deficient conditions, we investigated iron as “protective” dopant. A combined operando X-ray absorption spectroscopy and X-ray diffraction setup with quantitative on-line product analysis was used to unravel the structure of Ni and Fe in an alloyed Ni-Fe/Al₂O₃ catalyst during dynamically driven methanation of CO₂. We observed that Fe protects Ni from oxidation and is itself more dynamic in the oxidation and reduction process. Hence, such “sacrificial” or“protective” dopants added in order to preserve the catalytic activity under dynamic reaction conditions may not only be of high relevance with respect to fine-tuning of catalysts for future industrial P2X applications but certainly also of general interest

Fractal Weyl law for chaotic microcavities: Fresnel's laws imply multifractal scattering

We demonstrate that the harmonic inversion technique is a powerful tool to analyze the spectral properties of optical microcavities. As an interesting example we study the statistical properties of complex frequencies of the fully chaotic microstadium. We show that the conjectured fractal Weyl law for open chaotic systems [W. T. Lu, S. Sridhar, and M. Zworski, Phys. Rev. Lett. 91, 154101 (2003)] is valid for dielectric microcavities only if the concept of the chaotic repeller is extended to a multifractal by incorporating Fresnel's laws.Comment: 8 pages, 12 figure

Evolution of spectral properties along the O(6)-U(5) transition in the interacting boson model. II. Classical trajectories

This article continues our previous study of level dynamics in the [O(6)-U(5)]$\supset$O(5) transition of the interacting boson model [nucl-th/0504016] using the semiclassical theory of spectral fluctuations. We find classical monodromy, related to a singular bundle of orbits with infinite period at energy E=0, and bifurcations of numerous periodic orbits for E>0. The spectrum of allowed ratios of periods associated with beta- and gamma-vibrations exhibits an abrupt change around zero energy. These findings explain anomalous bunching of quantum states in the E$\approx$0 region, which is responsible for the redistribution of levels between O(6) and U(5) multiplets.Comment: 11 pages, 7 figures; continuation of nucl-th/050401

A collagen-based layered chronic wound biofilm model for testing antimicrobial wound products

A new in vitro chronic wound biofilm model was recently published, which provided a layered scaffold simulating mammalian tissue composition on which topical wound care products could be tested. In this paper, we updated the model even further to mimic the dynamic influx of nutrients from below as is the case in a chronic wound. The modified in vitro model was created using collagen instead of agar as the main matrix component and contained both Staphylococcus aureus and Pseudomonas aeruginosa. The model was cast in transwell inserts and then placed in wound simulating media, which allowed for an exchange of nutrients and waste products across a filter. Three potential wound care products and chlorhexidine digluconate 2% solution as a positive control were used to evaluate the model. The tested products were composed of hydrogels made from completely biodegradable starch microspheres carrying different active compounds. The compounds were applied topically and left for 2–4 days. Profiles of oxygen concentration and pH were measured to assess the effect of treatments on bacterial activity. Confocal microscope images were obtained of the models to visualise the existence of microcolonies. Results showed that the modified in vitro model maintained a stable number of the two bacterial species over 6 days. In untreated models, steep oxygen gradients developed and pH increased to &gt;8.0. Hydrogels containing active compounds alleviated the high oxygen consumption and decreased pH drastically. Moreover, all three hydrogels reduced the colony forming units significantly and to a larger extent than the chlorhexidine control treatment. Overall, the modified model expressed several characteristics similar to in vivo chronic wounds.</p