Feasible Policy Iteration

Safe reinforcement learning (RL) aims to solve an optimal control problem under safety constraints. Existing $\textit{direct}$ safe RL methods use the original constraint throughout the learning process. They either lack theoretical guarantees of the policy during iteration or suffer from infeasibility problems. To address this issue, we propose an $\textit{indirect}$ safe RL method called feasible policy iteration (FPI) that iteratively uses the feasible region of the last policy to constrain the current policy. The feasible region is represented by a feasibility function called constraint decay function (CDF). The core of FPI is a region-wise policy update rule called feasible policy improvement, which maximizes the return under the constraint of the CDF inside the feasible region and minimizes the CDF outside the feasible region. This update rule is always feasible and ensures that the feasible region monotonically expands and the state-value function monotonically increases inside the feasible region. Using the feasible Bellman equation, we prove that FPI converges to the maximum feasible region and the optimal state-value function. Experiments on classic control tasks and Safety Gym show that our algorithms achieve lower constraint violations and comparable or higher performance than the baselines

The Research Progress of Oil Sand Separation Technology in China

From 2007 to 2008, Research Institute of Petroleum Exploration & Development, Langfang Branch launched oil sand resource exploration and the study of hot water separation technology in Fengcheng area, Northwest of Junggar Basin, and the recoverable oil-sand oil resource is 54.98 million tons with the oil content in 7.1-10%, which is distributed in Cretaceous and Jurassic with the thickness of 80-140 meters, the cover depth of oil sand is 50-90 meters. Combining with the characteristics of the oil sand in this area and based on the research of hot water separation mechanism in oil sand, the hot water separation reagent for the oil sand in this area has been successfully developed, and its separation rate reaches 90%, provided that the concentrations of the agent is 4% and the separation temperature is 85 °C. Based on series of study, the construction of testing site, which is capable of processing 10,000 tons oil sand in this area, is completed, and the on-site separation tests of oil sand are launched with the recovery rate of 90% in normal operation, and the hot water separation technology and equipment research & development are successful.Key words: Oil sand; Hot water separation technology; Separation reagent; Test

Effects of Cutting Intensity on Soil Physical and Chemical Properties in a Mixed Natural Forest in Southeastern China

The mixed Chinese fir (Cunninghamia lanceolata (Lamb.) Hook.), Masson’s pine (Pinus massoniana Lamb.), and hardwood forest is a major forest type in China and of national and international importance in terms of its provision of both timber and ecosystem services. However, over-harvesting has threatened its long-term productivity and sustainability. We examined the impacts of timber harvesting intensity on soil physical and chemical properties 10 and 15 years after cutting using the research plots established with a randomized block design. We considered five treatments, including clear cutting and low (13.0% removal of growing stock volume), medium (29.1%), high (45.8%), and extra-high (67.1) intensities of selective cutting with non-cutting as the control. The impact on overall soil properties derived from principal component analysis showed increasing with a rise in cutting intensity, and the most critical impact was on soil nutrients, P and K in particular. Soil nutrient loss associated with timber harvesting even at a low cutting intensity could lead to nutrient deficits in this forest although most of the soil physical properties could be recovered under the low and medium intensities of cutting. These results indicate that clear cutting and the selective cutting of extra-high and high intensities should be avoided in this type of forest in the region

A Comparative Study of Systolic and Diastolic Mechanical Synchrony in Canine, Primate, and Healthy and Failing Human Hearts.

Aim: Mechanical dyssynchrony (MD) is associated with heart failure (HF) and may be prognostically important in cardiac resynchronization therapy (CRT). Yet, little is known about its patterns in healthy or diseased hearts. We here investigate and compare systolic and diastolic MD in both right (RV) and left ventricles (LV) of canine, primate and healthy and failing human hearts. Methods and Results: RV and LV mechanical function were examined by pulse-wave Doppler in 15 beagle dogs, 59 rhesus monkeys, 100 healthy human subjects and 39 heart failure (HF) patients. This measured RV and LV pre-ejection periods (RVPEP and LVPEP) and diastolic opening times (Q-TVE and Q-MVE). The occurrence of right (RVMDs) and left ventricular systolic mechanical delay (LVMDs) was assessed by comparing RVPEP and LVPEP values. That of right (RVMDd) and left ventricular diastolic mechanical delay (LVMDd) was assessed from the corresponding diastolic opening times (Q-TVE and Q-MVE). These situations were quantified by values of interventricular systolic (IVMDs) and diastolic mechanical delays (IVMDd), represented as positive if the relevant RV mechanical events preceded those in the LV. Healthy hearts in all species examined showed greater LV than RV delay times and therefore positive IVMDs and IVMDd. In contrast a greater proportion of the HF patients showed both markedly increased IVMDs and negative IVMDd, with diastolic mechanical asynchrony negatively correlated with LVEF. Conclusion: The present IVMDs and IVMDd findings have potential clinical implications particularly for personalized setting of parameter values in CRT in individual patients to achieve effective treatment of HF

Insight of novel biomarkers for papillary thyroid carcinoma through multiomics

IntroductionThe overdiagnosing of papillary thyroid carcinoma (PTC) in China necessitates the development of an evidence-based diagnosis and prognosis strategy in line with precision medicine. A landscape of PTC in Chinese cohorts is needed to provide comprehensiveness.Methods6 paired PTC samples were employed for whole-exome sequencing, RNA sequencing, and data-dependent acquisition mass spectrum analysis. Weighted gene co-expression network analysis and protein-protein interactions networks were used to screen for hub genes. Moreover, we verified the hub genes' diagnostic and prognostic potential using online databases. Logistic regression was employed to construct a diagnostic model, and we evaluated its efficacy and specificity based on TCGA-THCA and GEO datasets.ResultsThe basic multiomics landscape of PTC among local patients were drawn. The similarities and differences were compared between the Chinese cohort and TCGA-THCA cohorts, including the identification of PNPLA5 as a driver gene in addition to BRAF mutation. Besides, we found 572 differentially expressed genes and 79 differentially expressed proteins. Through integrative analysis, we identified 17 hub genes for prognosis and diagnosis of PTC. Four of these genes, ABR, AHNAK2, GPX1, and TPO, were used to construct a diagnostic model with high accuracy, explicitly targeting PTC (AUC=0.969/0.959 in training/test sets).DiscussionMultiomics analysis of the Chinese cohort demonstrated significant distinctions compared to TCGA-THCA cohorts, highlighting the unique genetic characteristics of Chinese individuals with PTC. The novel biomarkers, holding potential for diagnosis and prognosis of PTC, were identified. Furthermore, these biomarkers provide a valuable tool for precise medicine, especially for immunotherapeutic or nanomedicine based cancer therapy

RA-MAP, molecular immunological landscapes in early rheumatoid arthritis and healthy vaccine recipients

Rheumatoid arthritis (RA) is a chronic inflammatory disorder with poorly defined aetiology characterised by synovial inflammation with variable disease severity and drug responsiveness. To investigate the peripheral blood immune cell landscape of early, drug naive RA, we performed comprehensive clinical and molecular profiling of 267 RA patients and 52 healthy vaccine recipients for up to 18 months to establish a high quality sample biobank including plasma, serum, peripheral blood cells, urine, genomic DNA, RNA from whole blood, lymphocyte and monocyte subsets. We have performed extensive multi-omic immune phenotyping, including genomic, metabolomic, proteomic, transcriptomic and autoantibody profiling. We anticipate that these detailed clinical and molecular data will serve as a fundamental resource offering insights into immune-mediated disease pathogenesis, progression and therapeutic response, ultimately contributing to the development and application of targeted therapies for RA.</p

A Study on the Preparation of Microbial and Nonstarch Polysaccharide Enzyme Synergistic Fermented Maize Cob Feed and Its Feeding Efficiency in Finishing Pigs

1000 g maize cob mixed material was synergistically fermented by adding 2.5% composite probiotics and 0.06-0.08% NSP (nonstarch polysaccharide) enzyme to prepare fermented feed, and its effectiveness as feed for fattening pigs was investigated. The results showed that the appearance, texture, and nutrient quality of maize cobs significantly improved after fermentation, the total number of bacteria was 4.5×1010 CFU/g, and the protein content was 7.1%. Compared to the control group, the pigs in the 6% fermented maize cob feed experimental group showed significantly increased daily feed intake, daily weight gain, and nutrient digestion rate (p<0.05) and reduced feed conversion ratio (p<0.05). Most indicators including slaughter performance and meat quality significantly improved. In addition, beneficial bacteria including Lactobacillus in the intestines of the finishing pigs significantly increased, and pathogenic bacteria including Escherichia coli in the intestines and feces were found to be significantly reduced (p<0.05). The intestinal crypt depth, VH/CD ratio, and ileal mucosal immunity of the finishing pigs also significantly improved (p<0.05). The cytokine content and gene expression of sIgA, IL-8, and TNF-α were found to be significantly increased (p<0.05). It could be concluded that the addition of 6% fermented maize cob feed to the diets of finishing pigs could promote their growth, improve their production performance and slaughter performance meat quality, and enhance their intestinal microecological balance and immunity

Identification and Evolution of TGF-β Signaling Pathway Members in Twenty-Four Animal Species and Expression in Tilapia

Transforming growth factor β (TGF-β) signaling controls diverse cellular processes during embryogenesis as well as in mature tissues of multicellular animals. Here we carried out a comprehensive analysis of TGF-β pathway members in 24 representative animal species. The appearance of the TGF-β pathway was intrinsically linked to the emergence of metazoan. The total number of TGF-β ligands, receptors, and smads changed slightly in all invertebrates and jawless vertebrates analyzed. In contrast, expansion of the pathway members, especially ligands, was observed in jawed vertebrates most likely due to the second round of whole genome duplication (2R) and additional rounds in teleosts. Duplications of TGFB2, TGFBR2, ACVR1, SMAD4 and SMAD6, which were resulted from 2R, were first isolated. Type II receptors may be originated from the ACVR2-like ancestor. Interestingly, AMHR2 was not identified in Chimaeriformes and Cypriniformes even though they had the ligand AMH. Based on transcriptome data, TGF-β ligands exhibited a tissue-specific expression especially in the heart and gonads. However, most receptors and smads were expressed in multiple tissues indicating they were shared by different ligands. Spatial and temporal expression profiles of 8 genes in gonads of different developmental stages provided a fundamental clue for understanding their important roles in sex determination and reproduction. Taken together, our findings provided a global insight into the phylogeny and expression patterns of the TGF-β pathway genes, and hence contribute to the greater understanding of their biological roles in the organism especially in teleosts