Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration

Safe Reinforcement Learning (RL) aims to find a policy that achieves high rewards while satisfying cost constraints. When learning from scratch, safe RL agents tend to be overly conservative, which impedes exploration and restrains the overall performance. In many realistic tasks, e.g. autonomous driving, large-scale expert demonstration data are available. We argue that extracting expert policy from offline data to guide online exploration is a promising solution to mitigate the conserveness issue. Large-capacity models, e.g. decision transformers (DT), have been proven to be competent in offline policy learning. However, data collected in real-world scenarios rarely contain dangerous cases (e.g., collisions), which makes it prohibitive for the policies to learn safety concepts. Besides, these bulk policy networks cannot meet the computation speed requirements at inference time on real-world tasks such as autonomous driving. To this end, we propose Guided Online Distillation (GOLD), an offline-to-online safe RL framework. GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms. Experiments in both benchmark safe RL tasks and real-world driving tasks based on the Waymo Open Motion Dataset (WOMD) demonstrate that GOLD can successfully distill lightweight policies and solve decision-making problems in challenging safety-critical scenarios

Analysis of internal flow characteristics and entropy generation of low head bulb tubular pump

To study the internal flow characteristics and energy characteristics of a large bulb perfusion pump. Based on the CFX software of the ANSYS platform, the steady calculation of the three-dimensional model of the pump device is carried out. The numerical simulation results obtained by SST k-ω and RNG k-ε turbulence models are compared with the experimental results. Finally, SST k-ω is selected for subsequent calculation. With the help of the flow line diagram and turbulent kinetic energy table of the whole flow channel of the pump device, the flow components of the pump device under different working conditions are analyzed, and the pressure and velocity distribution at the impeller and guide vane are analyzed by pressure cloud diagram and velocity cloud diagram. It is found that there are three high-pressure areas in the impeller and guide vane section, and the high-pressure regions are mainly distributed in the middle of the impeller channel. As the head decreases, the pressure at the impeller and guide vane positions decreases gradually, and the flow rate increases. Based on the entropy production principle, the wall entropy production and the distribution of mainstream entropy production at the impeller and guide vane parts are analyzed

Impermeable inorganic “walls” sandwiching perovskite layer toward inverted and indoor photovoltaic devices

Interfaces between the perovskite active layer and the charge-transport layers (CTLs) play a critical role in both efficiency and stability of halide-perovskite photovoltaics. One of the major concerns is that surface defects of perovskite could cause detrimental nonradiative recombination and material degradation. In this work, we addressed this challenging problem by inserting ultrathin alkali-fluoride (AF) films between the tri-cation lead-iodide perovskite layer and both CTLs. This bilateral inorganic “walls” strategy makes use of both physical-blocking and chemical-anchoring functionalities of the continuous, uniform and compact AF framework: on the one hand, the uniformly distributed alkali-iodine coordination at the perovskite-AF interfaces effectively suppresses the formation of iodine-vacancy defects at the surfaces, thus reducing the trap-assisted recombination at the perovskite-CTL interfaces and therewith the open-voltage loss; on the other hand, the impermeable AF buffer layers effectively prevent the bidirectional ion migration at the perovskite-CTLs interfaces even under harsh working conditions. As a result, a power-conversion efficiency (PCE) of 22.02% (certified efficiency 20.4%) with low open-voltage deficit (<0.4 V) was achieved for the low-temperature processed inverted planar perovskite solar cells. Exceptional operational stability (500 h, ISOS-L-2) and thermal stability (1000 h, ISOS-D-2) were obtained. Meanwhile, a 35.7% PCE was obtained under dim-light source (1000 lux white LED light) with the optimized device, which is among the best records in perovskite indoor photovoltaics

Finishing the euchromatic sequence of the human genome

The sequence of the human genome encodes the genetic instructions for human physiology, as well as rich information about human evolution. In 2001, the International Human Genome Sequencing Consortium reported a draft sequence of the euchromatic portion of the human genome. Since then, the international collaboration has worked to convert this draft into a genome sequence with high accuracy and nearly complete coverage. Here, we report the result of this finishing process. The current genome sequence (Build 35) contains 2.85 billion nucleotides interrupted by only 341 gaps. It covers ∼99% of the euchromatic genome and is accurate to an error rate of ∼1 event per 100,000 bases. Many of the remaining euchromatic gaps are associated with segmental duplications and will require focused work with new methods. The near-complete sequence, the first for a vertebrate, greatly improves the precision of biological analyses of the human genome including studies of gene number, birth and death. Notably, the human enome seems to encode only 20,000-25,000 protein-coding genes. The genome sequence reported here should serve as a firm foundation for biomedical research in the decades ahead

Unveiling NUSAP1 as a common gene signature linking chronic HBV infection and HBV-related HCC

Abstract Background Hepatitis B virus (HBV) is a significant contributor to the development of hepatocellular carcinoma (HCC). Chronic HBV infection (CHB) facilitates disease progression through various mechanisms. However, the specific factor responsible for the progression of HBV infection to HCC remains unresolved. This study aims to identify the hub gene linking CHB and HBV-related HCC through bioinformatic analysis and experimental verification. Methods Differentially expressed genes (DEGs) were identified in datasets encompassing CHB and HBV-HCC patients from the GEO database. Enriched pathways were derived from GO and KEGG analysis. Hub genes were screened by protein–protein interaction (PPI) analysis and different modules in Cytoscape software. The significance of the selected hub gene in prognosis was further assessed in validated datasets. The effects of hub genes on cell growth and apoptosis were further determined in functional experiments. Results The study revealed upregulation of NUSAP1 in CHBs and HBV-HCCs. High expression of NUSAP1 served as an independent predictor for poor prognosis of liver cancers. Functional experiments demonstrated that NUSAP1 promotes cell growth, influences cell cycle process, and protects cells from apoptosis in HepG2.2.15 cells. Conclusion NUSAP1 serves as a poor prognostic indicator for liver cancers, and potentially plays a crucial role in HBV-HCC progression by promoting proliferation and inhibiting apoptosis

Occurrence of Grapevine Leafroll-Associated Virus-3 (GLRaV-3), Complete Nucleotide Sequence and Cultivar Susceptibility to a GLRaV-3 Isolate from Shaanxi Province of China

Grapevine (Vitis spp.) is globally one of the most economically important fruit crops. China is the largest grapevine-growing country of the world and Shaanxi province is one of the major grapevine-growing provinces in the country. A survey of GLRaV-3 found it widespread, with 57–100% infection frequencies, in both wine and table grapevine cultivars of three grapevine-growing regions of Shaanxi province. The virus infection frequencies varied with cultivars and regions. In order to obtain the full genomic length of a new GLRaV-3 isolate, GLRaV-3-Sau (accession number MK988555), was sequenced. This isolate has a genome of 18026 nucleotides, and 14 open reading frames (ORFs). The full-genome of the isolate GLRaV-3-Sau shared 85.88% nucleotide identity to GLRaV-3-LN, another isolate found in China. Coat protein (CP) genes of GLRaV-3 isolates were identical (99%) to the Vitis vinifera isolate (accession number HQ185608.1) from the USA. Immunohistochemistry for virus localization found that distribution patterns were similar in red-berried cultivar ‘Cabernet Sauvignon’ and white-berried cultivar ‘Chardonnay’, and GLRaV-3 is restricted in phloem tissue of vascular bundles. Virus transmission by micrografting found virus transmission efficiency was higher in ‘Chardonnay’ and ‘Thompson Seedless’ than in ‘Hunan-1’, indicating that ‘Hunan-1’ was less sensitive to GLRaV-3. As far as we know, these are the most comprehensive comparisons on the genome and CP genes of GLRaV-3 worldwide and the first to have found that the grapevine ‘Hunan-1’ is less susceptible to GLRaV-3

Stress Characteristics Analysis of Vertical Bi-Directional Flow Channel Axial Pump Blades Based on Fluid–Structure Coupling

The RANS equation and the RNG k-ε turbulence model were used in the three-dimensional non-constant numerical simulations of the full flow path of a vertical axial-flow pump which was carried out by applying CFX software. The velocity characteristics of the flow field and the pressure distribution of the impeller under different operating conditions were analyzed and verified by external characteristic tests. The fluid–structure interaction research was conducted for the stress distribution and deformation features of different surfaces of the blade under different working conditions. The result shows that where stress is most concentrated is at the point of the root of the blade near the hub. The low-pressure zone on the suction surface is mainly distributed near the rim, and the low-pressure area on the pressure side is mainly distributed near the hub

Stress Characteristics Analysis of Vertical Bi-Directional Flow Channel Axial Pump Blades Based on Fluid&ndash;Structure Coupling

The RANS equation and the RNG k-&epsilon; turbulence model were used in the three-dimensional non-constant numerical simulations of the full flow path of a vertical axial-flow pump which was carried out by applying CFX software. The velocity characteristics of the flow field and the pressure distribution of the impeller under different operating conditions were analyzed and verified by external characteristic tests. The fluid&ndash;structure interaction research was conducted for the stress distribution and deformation features of different surfaces of the blade under different working conditions. The result shows that where stress is most concentrated is at the point of the root of the blade near the hub. The low-pressure zone on the suction surface is mainly distributed near the rim, and the low-pressure area on the pressure side is mainly distributed near the hub

Content Variation and Potential Runoff Loss Risk of Nutrients in Surface Water of Saline-Alkali Paddy in Response to the Application of Different Nitrogen Fertilizer Types

As the saline-alkali paddy area continues to grow, the nutrient (e.g., nitrogen (N) and phosphorus (P)) runoff loss is becoming more serious in the world. The N-fertilizer application affects the nutrient runoff loss risk in paddy. Selecting suitable fertilizer types to reduce nutrient loss is beneficial to agricultural sustainability. However, the effects of N-fertilizer application in saline-alkali paddy are not clear. This study measured the N and P concentration of surface water in saline-alkali paddy, using various N—fertilizer treatments (i.e., urea (U), urea with urease—nitrification inhibitors (UI), organic–inorganic compound fertilizer (OCF), carbon—based slow—release fertilizer (CSF), and no N fertilization (CK)). Based on the structural equation model, both phosphate (PO43−-P) and total−P (TP) concentrations had a positive influence on total-N (TN) concentration regardless of N−fertilizer types applied. Potential risks of ammonia—N (NH4+—N) and nitrate—N (NO3−—N) runoff losses were reduced in UI treatment, but the TN and TP losses were increased. At the panicle-initiation fertilizer stage, the NO3−−N, TN, and TP concentrations in CSF and OCF treatments were lower than U. The CSF application can control the TP runoff loss risk during the rice-growing season. UI should not be suggested for the control of nutrient runoff loss in saline-alkali paddy