Guided Cooperation in Hierarchical Reinforcement Learning via Model-based Rollout

Goal-conditioned hierarchical reinforcement learning (HRL) presents a promising approach for enabling effective exploration in complex long-horizon reinforcement learning (RL) tasks via temporal abstraction. Yet, most goal-conditioned HRL algorithms focused on the subgoal discovery, regardless of inter-level coupling. In essence, for hierarchical systems, the increased inter-level communication and coordination can induce more stable and robust policy improvement. Here, we present a goal-conditioned HRL framework with Guided Cooperation via Model-based Rollout (GCMR), which estimates forward dynamics to promote inter-level cooperation. The GCMR alleviates the state-transition error within off-policy correction through a model-based rollout, further improving the sample efficiency. Meanwhile, to avoid being disrupted by these corrected but possibly unseen or faraway goals, lower-level Q-function gradients are constrained using a gradient penalty with a model-inferred upper bound, leading to a more stable behavioral policy. Besides, we propose a one-step rollout-based planning to further facilitate inter-level cooperation, where the higher-level Q-function is used to guide the lower-level policy by estimating the value of future states so that global task information is transmitted downwards to avoid local pitfalls. Experimental results demonstrate that incorporating the proposed GCMR framework with ACLG, a disentangled variant of HIGL, yields more stable and robust policy improvement than baselines and substantially outperforms previous state-of-the-art (SOTA) HRL algorithms in both hard-exploration problems and robotic control

Optimization Method Based On Optimal Control

In this paper, we focus on a method based on optimal control to address the optimization problem. The objective is to find the optimal solution that minimizes the objective function. We transform the optimization problem into optimal control by designing an appropriate cost function. Using Pontryagin's Maximum Principle and the associated forward-backward difference equations (FBDEs), we derive the iterative update gain for the optimization. The steady system state can be considered as the solution to the optimization problem. Finally, we discuss the compelling characteristics of our method and further demonstrate its high precision, low oscillation, and applicability for finding different local minima of non-convex functions through several simulation examples

Graphene Oxide on the Microstructure and Mechanical Properties of Cement Based Composite Material

To investigate the mixing amount of graphene oxide and water cement ratio on the microstructure and mechanical properties of graphene oxide reinforced cement based composite material, graphene oxide suspension was developed using improved Hummers method, and the structure, size and morphology of graphene oxide were represented using Fourier transform infrared spectroscopy (FTIR), X-ray diffraction (XRD) and AFM. The results demonstrated that the bending and compressive strength of graphene oxide reinforced cement based composite material improved firstly and then declined with the increase of the mixing amount of graphene oxide, and moreover the improvement of the bending strength was obvious than that of the compressive strength. When the content of graphene oxide was 0.03%, the bending strength reached the maximum, 13.73 MPa. Under a high water cement ratio, the addition of graphene oxide was more effective in enhancing the strength of cement mortar. The representation of the microstructure of cement based composite material with scanning electron microscope (SEM) suggested that graphene oxide could optimize the microstructure of cement hydration products, improve the pore structure of set cement, reduce the volume of micropore in set cement, and increase the compactness of set cement, i.e. apparently strengthen the toughening effect of set cement. The research achievements are useful to improve the mechanical properties of cement based composite materials

Graphene Oxide on the Microstructure and Mechanical Properties of Cement Based Composite Material

To investigate the mixing amount of graphene oxide and water cement ratio on the microstructure and mechanical properties of graphene oxide reinforced cement based composite material, graphene oxide suspension was developed using improved Hummers method, and the structure, size and morphology of graphene oxide were represented using Fourier transform infrared spectroscopy (FTIR), X-ray diffraction (XRD) and AFM. The results demonstrated that the bending and compressive strength of graphene oxide reinforced cement based composite material improved firstly and then declined with the increase of the mixing amount of graphene oxide, and moreover the improvement of the bending strength was obvious than that of the compressive strength. When the content of graphene oxide was 0.03%, the bending strength reached the maximum, 13.73 MPa. Under a high water cement ratio, the addition of graphene oxide was more effective in enhancing the strength of cement mortar. The representation of the microstructure of cement based composite material with scanning electron microscope (SEM) suggested that graphene oxide could optimize the microstructure of cement hydration products, improve the pore structure of set cement, reduce the volume of micropore in set cement, and increase the compactness of set cement, i.e. apparently strengthen the toughening effect of set cement. The research achievements are useful to improve the mechanical properties of cement based composite materials

Study on the Effect of Regional Water Pollution—Take Huaxi River in Chongqing as an Example

Water pollution management plays a crucial role in China’s ecological environment development. It has evolved from being solely the responsibility of the government to a collaborative effort involving multiple entities. This paper presents findings from a field survey conducted around the collaborative capacity and effectiveness of wastewater treatment in Huaxi River, Chongqing. The study collected 427 valid questionnaires and employed SPSS26.0 software and AMOS24.0 software, utilizing structural equation modelling and regression analysis to verify the relationship between the variables. The results highlight that synergy mechanism acts as a mediating variable between synergy capability and synergy governance effect, underscoring the role of mechanism in the relationship between capability and governance effect. The conclusion emphasizes the importance of enhancing synergistic capacity and synergistic mechanism to effectively promote synergistic governance effect in the water pollution management of Huaxi River in Chongqing. This can be achieved by improving the abilities of multiple stakeholders in managing water pollution, enhancing cooperation among parties, and encouraging participation of social organizations, the public, and enterprises in the management process to achieve sustainable development of ecological civilization

Utterance Augmentation for Speaker Recognition

The speaker recognition problem is to automatically recognize a person from their voice. The training of a speaker recognition model typically requires a very large training corpus, e.g., multiple voice samples from a very large number of individuals. In the diverse domains of application of speaker recognition, it is often impractical to obtain a training corpus of the requisite size. This disclosure describes techniques that augment utterances, e.g., by cutting, splitting, shuffling, etc., such that the need for collections of raw voice samples from individuals is substantially reduced. In effect, the original model works better on the augmented utterances on the target domain

The effects of litter input and increased precipitation on soil microbial communities in a temperate grassland

Global warming has contributed to shifts in precipitation patterns and increased plant productivity, resulting in a significant increase in litter input into the soils. The enhanced litter input, combined with higher levels of precipitation, may potentially affect soil microbial communities. This study aims to investigate the effects of litter input and increased precipitation on soil microbial biomass, community structure, and diversity in a temperate meadow steppe in northeastern China. Different levels of litter input (0%, +30%, +60%) and increased precipitation (0%, +15%, +30%) were applied over a three-year period (2015–2017). The results showed that litter input significantly increased the biomass of bacteria and fungi without altering their diversity, as well as the ratio of bacterial to fungal biomass. Increased precipitation did not have a notable effect on the biomass and diversity of bacteria and fungi, but it did increase the fungal-to-bacterial biomass ratio. However, when litter input and increased precipitation interacted, bacterial diversity significantly increased while the fungal-to-bacterial biomass ratio remained unchanged. These findings indicate that the projected increases in litter and precipitation would have a substantial impact on soil microbial communities. In energy-and water-limited temperate grasslands, the additional litter inputs and increased precipitation contribute to enhanced nutrient and water availability, which in turn promotes microbial growth and leads to shifts in community structure and diversity

Effects of pectin on molecular structural changes in starch during digestion

Starch digestion rate is strongly related to metabolic diseases such as obesity and diabetes. Starchy foods always contain non-starch components, which can affect starch digestibility. Mixtures of ungelatinized corn starch with a common non-starch component, pectin, were used to investigate pectin's effect on starch digestibility rate and evolution of starch molecular structure during digestion using amyloglucosidase and pancreatin. The whole-molecule size distribution and the chain-length distribution of chains were measured by size-exclusion chromatography and fluorophore-assisted carbohydrate electrophoresis. Digestion profiles and changes in molecular size distributions of whole and debranched digesta during digestion show that addition of pectin significantly decreased starch digestion rates. While pectin did not change the amylose/amylopectin ratio during most of the digestion, it decreased the digestion rate of short amylopectin chains compared to long ones. UV–visible spectral data suggested that a major contributor to this digestion rate change is from substantial pectin/amyloglucosidase interaction. This suggests an approach to designing nutritionally more beneficial starch-based foods by taking account of interactions between pectin and digestive enzymes

A missense mutation in Pitx2 leads to early-onset glaucoma via NRF2-YAP1 axis.

Glaucoma is a leading cause of blindness, affecting 70 million people worldwide. Owing to the similarity in anatomy and physiology between human and mouse eyes and the ability to genetically manipulate mice, mouse models are an invaluable resource for studying mechanisms underlying disease phenotypes and for developing therapeutic strategies. Here, we report the discovery of a new mouse model of early-onset glaucoma that bears a transversion substitution c. G344T, which results in a missense mutation, p. R115L in PITX2. The mutation causes an elevation in intraocular pressure (IOP) and progressive death of retinal ganglion cells (RGC). These ocular phenotypes recapitulate features of pathologies observed in human glaucoma. Increased oxidative stress was evident in the inner retina. We demonstrate that the mutant PITX2 protein was not capable of binding to Nuclear factor-like 2 (NRF2), which regulates Pitx2 expression and nuclear localization, and to YAP1, which is necessary for co-initiation of transcription of downstream targets. PITX2-mediated transcription of several antioxidant genes were also impaired. Treatment with N-Acetyl-L-cysteine exerted a profound neuroprotective effect on glaucoma-associated neuropathies, presumably through inhibition of oxidative stress. Our study demonstrates that a disruption of PITX2 leads to glaucoma optic pathogenesis and provides a novel early-onset glaucoma model that will enable elucidation of mechanisms underlying the disease as well as to serve as a resource to test new therapeutic strategies