346 research outputs found
Approximate Shielding of Atari Agents for Safe Exploration
Balancing exploration and conservatism in the constrained setting is an
important problem if we are to use reinforcement learning for meaningful tasks
in the real world. In this paper, we propose a principled algorithm for safe
exploration based on the concept of shielding. Previous approaches to shielding
assume access to a safety-relevant abstraction of the environment or a
high-fidelity simulator. Instead, our work is based on latent shielding -
another approach that leverages world models to verify policy roll-outs in the
latent space of a learned dynamics model. Our novel algorithm builds on this
previous work, using safety critics and other additional features to improve
the stability and farsightedness of the algorithm. We demonstrate the
effectiveness of our approach by running experiments on a small set of Atari
games with state dependent safety labels. We present preliminary results that
show our approximate shielding algorithm effectively reduces the rate of safety
violations, and in some cases improves the speed of convergence and quality of
the final agent.Comment: Accepted for presentation at the ALA workshop as part of AAMAS 202
Approximate Model-Based Shielding for Safe Reinforcement Learning
Reinforcement learning (RL) has shown great potential for solving complex
tasks in a variety of domains. However, applying RL to safety-critical systems
in the real-world is not easy as many algorithms are sample-inefficient and
maximising the standard RL objective comes with no guarantees on worst-case
performance. In this paper we propose approximate model-based shielding (AMBS),
a principled look-ahead shielding algorithm for verifying the performance of
learned RL policies w.r.t. a set of given safety constraints. Our algorithm
differs from other shielding approaches in that it does not require prior
knowledge of the safety-relevant dynamics of the system. We provide a strong
theoretical justification for AMBS and demonstrate superior performance to
other safety-aware approaches on a set of Atari games with state-dependent
safety-labels.Comment: Accepted at ECAI 2023 (main technical track
Enabling 3D magnetic circuits by the additive manufacturing of soft magnetic material
Additive manufacturing has been revolutionary in enabling complex structural components such as lattice structures and topologically optimised parts, however the utilisation of functional materials such as soft magnetic materials is only just being realised. 3D magnetic flux pathways in electrical machines have been illusive due to the high eddy current losses caused by thick cross-sections, and the inability to process electrical steel laminations into 3D structures. By processing soft magnetic materials with additive manufacturing, geometry can be tailored to avoid large bulk cross sections and reduce eddy currents whilst maintaining a 3D flux pathway, enabling the creation of new electrical machine architectures in the pursuit of higher power density and efficiency, which may enable the decarbonisation of the transport sectors including commercial aircraft.
This thesis demonstrates the processing of high silicon electrical steel (Fe-6.5 wt%Si) using laser powder bed fusion and characterises the magnetic properties of this material. The importance of surface roughness on the magnetic susceptibility is investigated, showing that contours may be used to improve the as-built surface finish, but post-processing methods such as polishing are required to obtain the best magnetic properties. The samples in this study exhibited a weak crystallographic texture and the orientation of the samples in the build chamber displayed little impact on the magnetic susceptibility.
Two methods are used in this study to reduce eddy currents and enable components with 3D magnetic flux pathways to be manufactured. The first is by designing thin-walled cross-sections, which use air as an insulating medium to reduce the thick cross section of the material. This method is demonstrated in lab experiments showing a reduction of the eddy current loss coefficient to 0.0005 using a novel hexagonal cross section. A Hilbert pattern was implemented into an axial flux electrical machine, demonstrating loss performance comparable to thick electrical steel laminations below 500 Hz, increasing torque density by 13% by achieving a reduction in volume of magnetic material of 33%. The second method uses process control to create stochastic cracking within the material, demonstrating excellent loss behaviour of 2.2 W/kg (50 Hz, 1T) with stacking factors >97%. The mechanical integrity was confirmed to be adequate for implementation into the axial flux machine tested with a UTS of 25 MPa when embedded with epoxy resin. These methods can be implemented into electrical machines enabling the creation of new architectures, with the hope to increase power density and efficiency.
This is the first time that additively manufactured soft magnetic material has been characterised in an electrical machine, overcoming the issues of large cross sections. Although the soft magnetic material has not displayed loss behaviour as good as electrical steel laminations, it does enable 3D magnetic circuits within electrical machines which may be exploited to improve the performance of the machine. Further optimisation of the stochastic cracking method of eddy current management by aligning the cracks with the flux direction will yield further improvements, and may compete with the thinnest laminations. Due to the current cost and limitations of additive manufacturing, this technology is only likely to be implemented into the highest value electrical machines, such as those in top end automotive and aerospace, where benefits in performance are of upmost importance. This development in processing of soft magnetic material is the missing piece to enable fully additively manufactured motors, which could revolutionise electrical machine architecture
The inverse-Compton ghost HDF 130 and the giant radio galaxy 6C 0905+3955: matching an analytic model for double radio source evolution
We present new GMRT observations of HDF 130, an inverse-Compton (IC) ghost of
a giant radio source that is no longer being powered by jets. We compare the
properties of HDF 130 with the new and important constraint of the upper limit
of the radio flux density at 240 MHz to an analytic model. We learn what values
of physical parameters in the model for the dynamics and evolution of the radio
luminosity and X-ray luminosity (due to IC scattering of the cosmic microwave
background (CMB)) of a Fanaroff-Riley II (FR II) source are able to describe a
source with features (lobe length, axial ratio, X-ray luminosity, photon index
and upper limit of radio luminosity) similar to the observations. HDF 130 is
found to agree with the interpretation that it is an IC ghost of a powerful
double-lobed radio source, and we are observing it at least a few Myr after jet
activity (which lasted 5--100 Myr) has ceased. The minimum Lorentz factor of
injected particles into the lobes from the hotspot is preferred to be
for the model to describe the observed quantities well,
assuming that the magnetic energy density, electron energy density, and lobe
pressure at time of injection into the lobe are linked by constant factors
according to a minimum energy argument, so that the minimum Lorentz factor is
constrained by the lobe pressure. We also apply the model to match the features
of 6C 0905+3955, a classical double FR II galaxy thought to have a low-energy
cutoff of in the hotspot due to a lack of hotspot
inverse-Compton X-ray emission. The models suggest that the low-energy cutoff
in the hotspots of 6C 0905+3955 is , just slightly above
the particles required for X-ray emission.Comment: 9 pages, 3 figure
Novel Cold-Adapted Lipase from Marine Plankton, Salpa thompsoni
Please click Download on the upper right corner to see the full description
Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow
We present Trieste, an open-source Python package for Bayesian optimization
and active learning benefiting from the scalability and efficiency of
TensorFlow. Our library enables the plug-and-play of popular TensorFlow-based
models within sequential decision-making loops, e.g. Gaussian processes from
GPflow or GPflux, or neural networks from Keras. This modular mindset is
central to the package and extends to our acquisition functions and the
internal dynamics of the decision-making loop, both of which can be tailored
and extended by researchers or engineers when tackling custom use cases.
Trieste is a research-friendly and production-ready toolkit backed by a
comprehensive test suite, extensive documentation, and available at
https://github.com/secondmind-labs/trieste
- …