Safe Model-Free Reinforcement Learning using Disturbance-Observer-Based Control Barrier Functions

Safe reinforcement learning (RL) with assured satisfaction of hard state constraints during training has recently received a lot of attention. Safety filters, e.g., based on control barrier functions (CBFs), provide a promising way for safe RL via modifying the unsafe actions of an RL agent on the fly. Existing safety filter-based approaches typically involve learning of uncertain dynamics and quantifying the learned model error, which leads to conservative filters before a large amount of data is collected to learn a good model, thereby preventing efficient exploration. This paper presents a method for safe and efficient model-free RL using disturbance observers (DOBs) and control barrier functions (CBFs). Unlike most existing safe RL methods that deal with hard state constraints, our method does not involve model learning, and leverages DOBs to accurately estimate the pointwise value of the uncertainty, which is then incorporated into a robust CBF condition to generate safe actions. The DOB-based CBF can be used as a safety filter with any model-free RL algorithms by minimally modifying the actions of an RL agent whenever necessary to ensure safety throughout the learning process. Simulation results on a unicycle and a 2D quadrotor demonstrate that the proposed method outperforms a state-of-the-art safe RL algorithm using CBFs and Gaussian processes-based model learning, in terms of safety violation rate, and sample and computational efficiency

Guaranteed Nonlinear Tracking in the Presence of DNN-Learned Dynamics With Contraction Metrics and Disturbance Estimation

This paper presents an approach to trajectory-centric learning control based on contraction metrics and disturbance estimation for nonlinear systems subject to matched uncertainties. The proposed approach allows for the use of deep neural networks to learn uncertain dynamics while still providing guarantees of transient tracking performance throughout the learning phase. Within the proposed approach, a disturbance estimation law is adopted to estimate the pointwise value of the uncertainty, with pre-computable estimation error bounds (EEBs). The learned dynamics, the estimated disturbances, and the EEBs are then incorporated in a robust Riemannian energy condition to compute the control law that guarantees exponential convergence of actual trajectories to desired ones throughout the learning phase, even when the learned model is poor. On the other hand, with improved accuracy, the learned model can be incorporated into a high-level planner to plan better trajectories with improved performance, e.g., lower energy consumption and shorter travel time. The proposed framework is validated on a planar quadrotor navigation example.Comment: Shorter version submitted to CDC 202

Convex Synthesis of Control Barrier Functions Under Input Constraints

This letter presents a systematic method based on the sum of square (SOS) optimization to synthesize control barrier functions (CBFs) for nonlinear polynomial systems subject to input constraints. The approach consists of two design steps. In the first step, using a linear-like representation of the nonlinear dynamics, an SOS optimization problem is formulated to search for an initial CBF and controller jointly. In the second step, an iterative optimization procedure involving the solution of a series of SOS problems is proposed to alternatively update the CBF and the controller to increase the invariant set defined by the CBF. The efficacy of the proposed approach is validated using numerical examples.Peer reviewe

Giant magneto-birefringence effect and tuneable colouration of 2D crystals' suspensions

One of the long sought-after goals in manipulation of light through light-matter interactions is the realization of magnetic-field-tuneable colouration, so-called magneto-chromatic effect, which holds great promise for optical, biochemical and medical applications due to its contactless and non-invasive nature. This goal can be achieved by magnetic-field controlled birefringence, where colours are produced by the interference between phase-retarded components of transmitted polarised light. Thus far birefringence-tuneable coloration has been demonstrated using electric field, material chirality and mechanical strain but magnetic field control remained elusive due to either weak magneto-optical response of transparent media or low transmittance to visible light of magnetically responsive media, such as ferrofluids. Here we demonstrate magnetically tuneable colouration of aqueous suspensions of two-dimensional cobalt-doped titanium oxide which exhibit an anomalously large magneto-birefringence effect. The colour of the suspensions can be tuned over more than two wavelength cycles in the visible range by moderate magnetic fields below 0.8 T. We show that such giant magneto-chromatic response is due to particularly large phase retardation (>3 pi) of the polarised light, which in its turn is a combined result of a large Cotton-Mouton coefficient (three orders of magnitude larger than for known liquid crystals), relatively high saturation birefringence (delta n = 2 x 10^-4) and high transparency of our suspensions to visible light. The work opens a new avenue to achieve tuneable colouration through engineered magnetic birefringence and can readily be extended to other magnetic 2D nanocrystals. The demonstrated effect can be used in a variety of magneto-optical applications, including magnetic field sensors, wavelength-tuneable optical filters and see-through printing.Comment: 10 pages, 4 figures. Nature Communications, 2020, Accepte

Probing the Electroweak Phase Transition with Exotic Higgs Decays

An essential goal of the Higgs physics program at the LHC and beyond is to explore the nature of the Higgs potential and shed light on the mechanism of electroweak symmetry breaking. An important class of models alter the thermal history of electroweak symmetry breaking from the predictions of the Standard Model (SM). This paper reviews the existence of a region of parameter space where a strong first-order electroweak phase transition is compatible with exotic decays of the SM-like Higgs boson. A dedicated search for exotic Higgs decays can actively explore this framework at the Large Hadron Collider (LHC), while future exotic Higgs decay searches at the high-luminosity LHC and future Higgs factories will be vital to conclusively probe the scenario

DDC-PIM: Efficient Algorithm/Architecture Co-design for Doubling Data Capacity of SRAM-based Processing-In-Memory

Processing-in-memory (PIM), as a novel computing paradigm, provides significant performance benefits from the aspect of effective data movement reduction. SRAM-based PIM has been demonstrated as one of the most promising candidates due to its endurance and compatibility. However, the integration density of SRAM-based PIM is much lower than other non-volatile memory-based ones, due to its inherent 6T structure for storing a single bit. Within comparable area constraints, SRAM-based PIM exhibits notably lower capacity. Thus, aiming to unleash its capacity potential, we propose DDC-PIM, an efficient algorithm/architecture co-design methodology that effectively doubles the equivalent data capacity. At the algorithmic level, we propose a filter-wise complementary correlation (FCC) algorithm to obtain a bitwise complementary pair. At the architecture level, we exploit the intrinsic cross-coupled structure of 6T SRAM to store the bitwise complementary pair in their complementary states ($Q/\overline{Q}$), thereby maximizing the data capacity of each SRAM cell. The dual-broadcast input structure and reconfigurable unit support both depthwise and pointwise convolution, adhering to the requirements of various neural networks. Evaluation results show that DDC-PIM yields about $2.84\times$ speedup on MobileNetV2 and $2.69\times$ on EfficientNet-B0 with negligible accuracy loss compared with PIM baseline implementation. Compared with state-of-the-art SRAM-based PIM macros, DDC-PIM achieves up to $8.41\times$ and $2.75\times$ improvement in weight density and area efficiency, respectively.Comment: 14 pages, to be published in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD

Giant magneto-birefringence effect and tuneable colouration of 2D crystal suspensions

From Springer Nature via Jisc Publications RouterHistory: received 2020-02-13, accepted 2020-07-03, registration 2020-07-10, pub-electronic 2020-07-24, online 2020-07-24, collection 2020-12Publication status: PublishedAbstract: One of the long-sought-after goals in light manipulation is tuning of transmitted interference colours. Previous approaches toward this goal include material chirality, strain and electric-field controls. Alternatively, colour control by magnetic field offers contactless, non-invasive and energy-free advantages but has remained elusive due to feeble magneto-birefringence in conventional transparent media. Here we demonstrate an anomalously large magneto-birefringence effect in transparent suspensions of magnetic two-dimensional crystals, which arises from a combination of a large Cotton-Mouton coefficient and relatively high magnetic saturation birefringence. The effect is orders of magnitude stronger than those previously demonstrated for transparent materials. The transmitted colours of the suspension can be continuously tuned over two-wavelength cycles by moderate magnetic fields below 0.8 T. The work opens a new avenue to tune transmitted colours, and can be further extended to other systems with artificially engineered magnetic birefringence

Recognition of Abnormal-Laying Hens Based on Fast Continuous Wavelet and Deep Learning Using Hyperspectral Images

The egg production of laying hens is crucial to breeding enterprises in the laying hen breeding industry. However, there is currently no systematic or accurate method to identify low-egg-production-laying hens in commercial farms, and the majority of these hens are identified by breeders based on their experience. In order to address this issue, we propose a method that is widely applicable and highly precise. First, breeders themselves separate low-egg-production-laying hens and normal-laying hens. Then, under a halogen lamp, hyperspectral images of the two different types of hens are captured via hyperspectral imaging equipment. The vertex component analysis (VCA) algorithm is used to extract the cockscomb end member spectrum to obtain the cockscomb spectral feature curves of low-egg-production-laying hens and normal ones. Next, fast continuous wavelet transform (FCWT) is employed to analyze the data of the feature curves in order to obtain the two-dimensional spectral feature image dataset. Finally, referring to the two-dimensional spectral image dataset of the low-egg-production-laying hens and normal ones, we developed a deep learning model based on a convolutional neural network (CNN). When we tested the model’s accuracy by using the prepared dataset, we found that it was 0.975 percent accurate. This outcome demonstrates our identification method, which combines hyperspectral imaging technology, an FCWT data analysis method, and a CNN deep learning model, and is highly effective and precise in laying-hen breeding plants. Furthermore, the attempt to use FCWT for the analysis and processing of hyperspectral data will have a significant impact on the research and application of hyperspectral technology in other fields due to its high efficiency and resolution characteristics for data signal analysis and processing