Enhancing the Hierarchical Environment Design via Generative Trajectory Modeling

Unsupervised Environment Design (UED) is a paradigm for automatically generating a curriculum of training environments, enabling agents trained in these environments to develop general capabilities, i.e., achieving good zero-shot transfer performance. However, existing UED approaches focus primarily on the random generation of environments for open-ended agent training. This is impractical in scenarios with limited resources, such as the constraints on the number of generated environments. In this paper, we introduce a hierarchical MDP framework for environment design under resource constraints. It consists of an upper-level RL teacher agent that generates suitable training environments for a lower-level student agent. The RL teacher can leverage previously discovered environment structures and generate environments at the frontier of the student's capabilities by observing the student policy's representation. Moreover, to reduce the time-consuming collection of experiences for the upper-level teacher, we utilize recent advances in generative modeling to synthesize a trajectory dataset to train the teacher agent. Our proposed method significantly reduces the resource-intensive interactions between agents and environments and empirical experiments across various domains demonstrate the effectiveness of our approach

Diversity Induced Environment Design via Self-Play

Recent work on designing an appropriate distribution of environments has shown promise for training effective generally capable agents. Its success is partly because of a form of adaptive curriculum learning that generates environment instances (or levels) at the frontier of the agent's capabilities. However, such an environment design framework often struggles to find effective levels in challenging design spaces and requires costly interactions with the environment. In this paper, we aim to introduce diversity in the Unsupervised Environment Design (UED) framework. Specifically, we propose a task-agnostic method to identify observed/hidden states that are representative of a given level. The outcome of this method is then utilized to characterize the diversity between two levels, which as we show can be crucial to effective performance. In addition, to improve sampling efficiency, we incorporate the self-play technique that allows the environment generator to automatically generate environments that are of great benefit to the training agent. Quantitatively, our approach, Diversity-induced Environment Design via Self-Play (DivSP), shows compelling performance over existing methods

Influence of Temperature on Hydrolysis Acidification of Food Waste

AbstractFor two-phase anaerobic digestion process of food waste, degree of hydrolysis and products by acidification during hydrolysis and acidification phase directly affect the performance of methanogenesis phase. Temperature has great impact on hydrolysis and acidification of food waste. This paper monitored the dynamic change of biogas production, biogas composition, pH, soluble chemical oxygen demand (SCOD) and volatile fatty acids (VFAs) during hydrolysis and acidification stage so as to investigate specific influence of temperature on food waste. With the same inoculum and 9 days’ fermentation, three different temperatures (35, 55 and 70°C) were taken into consideration. The results showed that cumulative gas production was 4860mL at 70°C, which was 129.79% and 37.87% higher than that at 35 and 55°C. Besides, hydrogen content at 70°C was 45.34%, which was the highest among the three temperatures. Hydrolysis rate was proportional to the increase of temperature. Meanwhile, total VFAs yield and composition widely differed at three different temperatures. The hydrolysis and acidification products at 35°C were mainly ethanol and acetic acids and the highest concentrations of ethanol at 35°C were 3.28 and 3.65 times of that at 55 and 70°C, but more acetic, isobutyric and butyric acids were generated at 55 and 70°C. Among three temperatures, 70°C had the highest acetic acids concentration while 55°C had the highest isobutyric and butyric acids concentration

Microscopic pore characteristics of coal seam and the controlling effect of sedimentary environment on pore structure in No.8 coal seam of the Ordos Basin

To accurately evaluate the distribution and potential of deep coalbed methane resources, it is critical to identify the coal sedimentary and pore distribution characteristics of deep coal reservoirs. Therefore, based on the analysis of coal macerals, nuclear magnetic resonance porosity, and electron microscopy, the characteristics of coal macerals and pore distribution in the Yulin area were identified, as well as the relationship between the coal sedimentary and pore structure characteristics of No.8 coal was evaluated. The results indicated that: ① The organic macerals of No.8 coal seam in the study area were predominantly composed of vitrinite, indicating a predominance of woody plants during the early stages of coal formation. As the degree of water cover increases, it gradually transitions into a mixture of woody and herbaceous plants. ② During the Benxi period, No. 8 coal evolved from an early wetland forest swamp to a later open water swamp with stagnant swamp water, creating a stable environment conducive to organic matter enrichment and coal seam stability. ③ The proportion of macropores in No.8 coal seam was negatively correlated with the structure preservation index (TPI), while it was positively correlated with the gel index (GI). ④ Matrix vitrinite developed dense or sparse banded pores, mass vitrinite developed few pores, and the structure vitrinite developed original tissue pores, mostly filled with minerals, while the homogeneous vitrinite did not develop pores. In conclusion, pore distribution in Yulin's coal seams was controlled by sedimentation and macerals. The pore structure of coal seams in the open water swamp sedimentary presented three peaks, with micropores and macropores developing best, which can simultaneously enrich free gas and adsorbed gas. The pore structure of coal seams in the wetland forest swamp sedimentary presented two peaks, with micropores developing well and macropores developing poorly, mainly enriching adsorbed gas

Improving Biomethane Production and Mass Bioconversion of Corn Stover Anaerobic Digestion by Adding NaOH Pretreatment and Trace Elements

This research applied sodium hydroxide (NaOH) pretreatment and trace elements to improve biomethane production when using corn stover for anaerobic digestion. Full-factor experimental tests identified the best combination of trace elements with the NaOH pretreatment, indicating that the best combination was with 1.0, 0.4, and 0.4 mg⋅L −1 ⋅d −1 of elements Fe, Co, and Ni, respectively. The cumulative biomethane production adding NaOH pretreatment and trace elements was 11,367 mL; total solid bioconversion rate was 55.7%, which was 41.8%-62.2% higher than with NaOH-pretreatment alone and 22.2%-56.3% higher than with untreated corn stover. The best combination was obtained 5-9 days shorter than T 90 and maintained good system operation stability. Only a fraction of the trace elements in the best combination was present in the resulting solution; more than 85% of the total amounts added were transferred into the solid fraction. Adding 0.897 g of Fe, 0.389 g of Co, and 0.349 g of Ni satisfied anaerobic digestion needs and enhanced biological activity at the beginning of the operation. The results showed that NaOH pretreatment and adding trace elements improve corn stover biodegradability and enhance biomethane production

Suggestions on the development strategy of shale gas in China

AbstractFrom the aspects of shale gas resource condition, main exploration and development progress, important breakthrough in key technologies and equipment, this paper systematically summarized and analyzed current situation of shale gas development in China and pointed out five big challenges such as misunderstandings, lower implementation degree and higher economic uncertainty of shale gas resource, and still no breakthrough in exploration and development core technologies and equipment for shale gas buried depth more than 3500 m, higher cost and other non-technical factors that restrict the development pace. Aiming at the above challenges, we put forward five suggestions to promote the shale gas development in China: (1) Make strategies and set goals according to our national conditions and exploration and development stages. That is, make sure to realize shale gas annual production of 20 × 109 m3, and strives to reach 30 × 109 m3. (2) Attach importance to the research of accumulation and enrichment geological theory and exploration & development key engineering technologies for lower production and lower pressure marine shale gas reservoir, and at the same time orderly promote the construction of non-marine shale gas exploration & development demonstration areas. (3) The government should introduce further policies and set special innovation funds to support the companies to carry out research and development of related technologies and equipment, especially to strengthen the research and development of technology, equipment and process for shale gas bellow 3500 m in order to achieve breakthrough in deep shale gas. (4) Continue to promote the geological theory, innovation in technology and management, and strengthen cost control on drilling, fracturing and the whole process in order to realize efficient, economic and scale development of China's shale gas. (5) Reform the mining rights management system, establish information platform of shale gas exploration and development data, and correctly guide the non-oil and gas companies to participate in shale gas exploration and development

The coupling coordination between health service supply and regional economy in China: spatio-temporal evolution and convergence

BackgroundThe coordination of health service supply and regional economy is an integral path to promote China’s prosperity.MethodsBased on the coupling mechanism of health service supply and regional economy, we sampled the data from 30 provinces in China from 2009 to 2021 in this study and constructed the evaluation index system. Additionally, we calculated the coupling coordination degree (HED) of the two through the coupling coordination degree model. We further used the kernel density estimation, Moran’s I index, and spatial β convergence model to assess the dynamic evolution trends, spatial aggregation effect, and spatial convergence characteristics of coupling coordination.Conclusion(1) HED in China showed a rising trend during the study period but with large regional differences, forming a gradient distribution pattern of “high in the east and low in the west.” (2) The results of Kernel density estimation show that HED has formed a gradient differentiation phenomenon within each region in China. (3) HED has modeled spatial clustering characteristics during the study period, with high-value clusters mainly appearing in the eastern region and low-value clusters appearing in the northwestern region. (4) There are absolute β-convergence and conditional β-convergence trends in HED in China and the three major regions during the study period, but there is an obvious regional heterogeneity in the control factors. The research provides a reference for accurately implementing policies according to different levels of health service supply and economic development, in addition to narrowing the regional differences of the coupling coordination between the regional economy and health service supply