Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL

Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets without interacting with the environment. However, the performance of existing offline RL algorithms heavily depends on the scale and state-action space coverage of datasets. Real-world data collection is often expensive and uncontrollable, leading to small and narrowly covered datasets and posing significant challenges for practical deployments of offline RL. In this paper, we provide a new insight that leveraging the fundamental symmetry of system dynamics can substantially enhance offline RL performance under small datasets. Specifically, we propose a Time-reversal symmetry (T-symmetry) enforced Dynamics Model (TDM), which establishes consistency between a pair of forward and reverse latent dynamics. TDM provides both well-behaved representations for small datasets and a new reliability measure for OOD samples based on compliance with the T-symmetry. These can be readily used to construct a new offline RL algorithm (TSRL) with less conservative policy constraints and a reliable latent space data augmentation procedure. Based on extensive experiments, we find TSRL achieves great performance on small benchmark datasets with as few as 1% of the original samples, which significantly outperforms the recent offline RL algorithms in terms of data efficiency and generalizability.Comment: The first two authors contributed equall

Hypoglycemic and Antioxidant Effects of Camellia nitidissima Flower on Type 2 Diabetic Mice

Objective: To comprehensively evaluate the hypoglycemic and antioxidant effects of Camellia nitidissima flower. Methods: The chemical constituents in the aqueous extract of C. nitidissima flower (CFA) were identified by high performance liquid chromatography-mass spectrometry (HPLC-MS). Next, a mouse model of type 2 diabetes was established, and the diabetic mice were randomly divided into five groups: model, positive control (acarbose at 20 mg/kg mb), low-, medium- and high-dose CFA (200, 400 and 800 mg/kg mb, respectively). After five weeks of intragastric intervention, general growth characteristics, serum glucose, fasting insulin (FINS) and lipid levels, oxidative stress in pancreas and liver tissues, tissue morphological changes and cell apoptosis were analyzed. Results: CFA had a high content of polyphenols and polysaccharides. The half-maximal inhibitory concentration (IC50) values for scavenging of 1,1-diphenyl-2-picrylhydrazyl (DPPH) radical and α-glucosidase inhibition were (24.14 ± 0.64) and (69.99 ± 1.97) μg/mL, respectively. Seven compounds were identified from CFA. In addition, CFA could effectively improve the ‘three more and one less’ symptoms of diabetic mice, significantly reduce the levels of fasting blood glucose (FBG), postprandial blood glucose (PBG), total cholesterol (TC), triglyceride (TG), low-density lipoprotein cholesterol (LDL-C) and malondialdehyde (MDA), increase the levels of insulin and high-density lipoprotein cholesterol (HDL-C), improve the activities of total superoxide dismutase (SOD) and glutathione peroxidase (GSH-Px), and alleviate pathological damage in pancreas and liver tissues. Conclusion: CFA has significant hypoglycemic and antioxidant effects on type 2 diabetic mice

Ultra-broadband near-field Josephson microwave microscopy

Advanced microwave technologies constitute the foundation of a wide range of modern sciences, including quantum computing, microwave photonics, spintronics, etc. To facilitate the design of chip-based microwave devices, there is an increasing demand for state-of-the-art microscopic techniques capable of characterizing the near-field microwave distribution and performance. In this work, we integrate Josephson junctions onto a nano-sized quartz tip, forming a highly sensitive microwave mixer on-tip. This allows us to conduct spectroscopic imaging of near-field microwave distributions with high spatial resolution. Leveraging its microwave-sensitive characteristics, our Josephson microscope achieves a broad detecting bandwidth of up to 200 GHz with remarkable frequency and intensity sensitivities. Our work emphasizes the benefits of utilizing the Josephson microscope as a real-time, non-destructive technique to advance integrated microwave electronics

Properties of the brightest globular cluster in M 81 based on multicolour observations

Context. Researching the properties of the brightest globular cluster (referred to as GC1) in M 81 can provide a fossil record of the earliest stages of galaxy formation and evolution. The Beijing–Arizona–Taiwan–Connecticut (BATC) Multicolour Sky Survey has carried out deep exposures of M 81. Aims. We derive the magnitudes in intermediate-band filters of the BATC system for GC1 and determine its age, mass, and structural parameters. Methods. GC1 was observed by BATC using 14 intermediate-band filters covering a wavelength range of 4000–10 000 Å. Based on photometric data in BATC and Two Micron All Sky Survey near-infrared JHKs filters, we constructed an extensive spectral energy distribution of GC1, spanning the wavelength range from 4000 to 20 000 Å. By comparing multicolour photometry with theoretical single stellar population synthesis models, we derived the age and mass of GC1. In addition, we obtained ellipticities, position angles, and surface brightness profiles for GC1 based on the images of deep observations with the Advanced Camera for Surveys on the Hubble Space Telescope. GC1 is better fitted by the Wilson model than by the King and Sérsic models in the F606W filter, and it is better fitted by the Sérsic model than by the King and Wilson models in the F814W filter. The ‘best-fit’ half-light radius of GC1 obtained here is 5.59 pc, which is larger than the majority of normal globular clusters (GCs) of the same luminosity. Results. The age and mass of GC1 estimated here are 13.0 ± 2.90 Gyr and 1.06 − 1.48 × 107 M⊙, respectively. The Rh versus MV diagram shows that GC1 occupies the same area as extended star clusters. Therefore, we suggest that GC1 is more likely an accreted former nuclear star cluster than a classical GC similar to most of those in the Milky Way

Estimating ages and metallicities of M31 star clusters from LAMOST DR6

Context. Determining the metallicities and ages of M31 clusters is fundamental to the study of the formation and evolution of M31 itself. The Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) has carried out a systematic spectroscopic campaign of clusters and candidates in M31. Aims. We constructed a catalogue of 346 M31 clusters observed by LAMOST. By combining the information of the LAMOST spectra and the multi-band photometry, we developed a new algorithm to estimate the metallicities and ages of these clusters. Methods. We distinguish young clusters from old using random forest classifiers based on a empirical training data set selected from the literature. Ages of young clusters are derived from the spectral energy distribution fits of their multi-band photometric measurements. Their metallicities are estimated by fitting their observed spectral principal components extracted from the LAMOST spectra with those from the young metal-rich single stellar population (SSP) models. For old clusters we built non-parameter random forest models between the spectral principal components and/or multi-band colours and the parameters of the clusters based on a training data set constructed from the SSP models. The ages and metallicities of the old clusters are then estimated by fitting their observed spectral principal components extracted from the LAMOST spectra and multi-band colours from the photometric measurements with the resultant random forest models. Results. We derived parameters of 53 young and 293 old clusters in our catalogue. Our resultant parameters are in good agreement with those from the literature. The ages of ∼30 catalogued clusters and metallicities of ∼40 sources are derived for the first time