Careful at Estimation and Bold at Exploration

Exploration strategies in continuous action space are often heuristic due to the infinite actions, and these kinds of methods cannot derive a general conclusion. In prior work, it has been shown that policy-based exploration is beneficial for continuous action space in deterministic policy reinforcement learning(DPRL). However, policy-based exploration in DPRL has two prominent issues: aimless exploration and policy divergence, and the policy gradient for exploration is only sometimes helpful due to inaccurate estimation. Based on the double-Q function framework, we introduce a novel exploration strategy to mitigate these issues, separate from the policy gradient. We first propose the greedy Q softmax update schema for Q value update. The expected Q value is derived by weighted summing the conservative Q value over actions, and the weight is the corresponding greedy Q value. Greedy Q takes the maximum value of the two Q functions, and conservative Q takes the minimum value of the two different Q functions. For practicality, this theoretical basis is then extended to allow us to combine action exploration with the Q value update, except for the premise that we have a surrogate policy that behaves like this exploration policy. In practice, we construct such an exploration policy with a few sampled actions, and to meet the premise, we learn such a surrogate policy by minimizing the KL divergence between the target policy and the exploration policy constructed by the conservative Q. We evaluate our method on the Mujoco benchmark and demonstrate superior performance compared to previous state-of-the-art methods across various environments, particularly in the most complex Humanoid environment.Comment: 20 page

Could in-network caching benefit information-centric networking

ABSTRACT Information-Centric Networking is (ICN) [1] gaining increasingly concerns, as an important direction of the Future Internet Architecture research. Although In-network caching is considered as one of the most significant properties of ICN, the cache policy for ICN is still little explored. In this paper, we formulate the in-network caching problem of ICN into Mixed-Integer Linear Programming problem. We also propose a novel cache policy named LB (Least Benefit) with taking into account the distance factor and a new forwarding scheme with shallow flooding (FSF for short) to improve the performance further,. Our simulation results show that with in-networking caching, the average hops of the ICN network can be reduced significantly by nearly 50% with simple cache policy like LFU and with some simple improvement such as LB and FSF the average hop can be reduced further

Prevalence of Anemia and its Associated Factors among Chinese 9-, 12-, and 14-Year-Old Children: Results from 2014 Chinese National Survey on Students Constitution and Health

Background: Anemia has been one of the main nutritional challenges around the world. Not enough attention has been paid to this issue in children and adolescents in China. In this study, we aimed to estimate the prevalence of anemia among 9-, 12-, and 14-year old Chinese children and investigate the associated factors of anemia. Methods: Data come from a cross-sectional survey conducted in 26 provinces and 4 municipalities in mainland China. A total of 48,537 children aged 9, 12, and 14 years old were included in data analyses. Anthropometric measurements were conducted to obtain information about height and weight. Capillary blood was collected from the fingertip, and hemoglobin concentration was tested by HemoCue201+. Information about sleep duration, daily consumption of eggs, milk, and breakfast were obtained from a self-administrated questionnaire. The mixed-effects logistic regression model was applied to estimate the association between selected variables and risk of anemia. Results: A total of 8.4% of participants were identified as being anemic; and the prevalence was higher in girls and rural children. Mixed-effects logistic regression analysis showed that children who were overweight, obese, and consumed eggs and milk every day had a lower risk of anemia. Spermarche, overweight/obesity, and having milk every day were associated with lower risk of anemia in boys, while menarche was found to be a risk factor and eating eggs every day to be a protective factor of anemia in girls. Conclusions: Anemia among 9-, 12-, and 14-year-old children is still high. Intervention programs of adding egg and milk into school daily diet might contribute to reducing anemia in Chinese school aged children, especially for those living in rural areas or girls with menarche

Increase and Spatial Variation in Soil Infiltration Rates Associated with Fibrous and Tap Tree Roots

Trees play important roles in urban stormwater management; through the loosening of soils by root growth, they increase infiltration and reduce runoff, helping to mitigate flooding and recharge groundwater. Malus baccata with fibrous roots and Sophora japonica with tap roots were studied experimentally to assess their enhancement of soil infiltration. A blank test without a tree was conducted for comparison. Steady-state soil infiltration rates at the bottom of test tanks were measured as 0.28 m/d, 0.33 m/d, and 0.61 m/d for the blank test, M. baccata, and S. japonica, respectively. This represents a 19% increase in the infiltration rate by planting M. baccata and a 118% increase by planting S. japonica. A larger increase in the infiltration rate by S. japonica is consistent with the effects of deeper and more vertical roots that help loosen deeper soils. Spatial variations in soil infiltration rates were also measured. Infiltration rates for M. baccata (1.06 m/d and 0.62 m/d) were larger than those for S. japonica (0.91 m/d and 0.51 m/d) at the same depths (0.35 m and 0.70 m); this is consistent with the expected effects of the shallower and more lateral roots of M. baccata. This study furthers our understanding of the roles of trees in watersheds and urban environments

Phytohormone requirements of pollination drop secretion in Ginkgo biloba ovules

Secretion of the pollination drop (PD) plays a crucial role in the sexual reproduction of gymnosperms. However, the molecular mechanism regulating PD secretion has not been reported. In our study, we successfully constructed cDNA libraries in ovules between PD secretion and PD withdrawal stages, and identified 2975 differentially expressed genes (DEGs). These genes were involved in biosynthesis of secondary metabolites, fatty acid biosynthesis, fatty acid metabolism and hormone-related pathways. We especially identified DEGs related to jasmonic acid (JA) metabolism and signal transduction, including Gb_39942 (SDP1), Gb_03931 (OPR2), and Gb_18696 (4CLL4), which are all upregulated at the stage of PD secretion in Ginkgo biloba. Gb_17431 (TIF3B) and Gb_17432 (TIF9), which encode the negative regulator JAZ protein, are downregulated during this period. We also found that KO1 (Gb_05787 and Gb_38576) and GA20ox1 (Gb_16591), which are related to gibberellin (GA) synthesis, had increased expression, whereas Gb_17754, encoding the GA receptor GID1, is downregulated at the PD secretion. From these results, we predicted that JA and GA may play important roles in the regulation of PD secretion in G. biloba ovules.The accepted manuscript in pdf format is listed with the files at the bottom of this page. The presentation of the authors' names and (or) special characters in the title of the manuscript may differ slightly between what is listed on this page and what is listed in the pdf file of the accepted manuscript; that in the pdf file of the accepted manuscript is what was submitted by the author

Gene Expression Profiles and Flavonoid Accumulation during Salt Stress in Ginkgo biloba Seedlings

Ginkgo biloba is an economically valuable tree, as a variety of flavonoid compounds are produced by the leaves of its seedlings. Although soil salinity is a serious threat to agricultural productivity worldwide, the effect of salt stress on G. biloba seedlings remains unclear. In this study, we found that under high NaCl concentrations (200 and 300 mmol/L), seedling growth was inhibited and the water content, chlorophyll, and peroxidase (POD) enzyme activity were significantly decreased in the leaves, whereas the soluble protein and proline levels increased significantly. However, at low NaCl concentrations (50 and 100 mmol/L), the seedlings grew normally because of the regulation of catalase (CAT) and POD enzyme activities. To elucidate the molecular mechanisms behind G. biloba salt tolerance, we examined the transcriptome of G. biloba seedlings treated with 100 mmol/L NaCl. Twelve differentially expressed genes (DEGs) were found to be involved in ion osmotic potential signal transduction and amplification, including two ABA signaling genes, five CDPK/CIPK genes, and five mitogen-activated protein kinase (MAPK) signaling genes. We also found that NAC transcription factors may be involved in the salt stress response; these included positive regulators (Gb_12203, Gb_27819, Gb_37720, and Gb_41540) and negative regulators (Gb_32549, Gb_35048, and Gb_37444). Importantly, treatment with 100 mmol/L NaCl can significantly improve flavonoid and flavonol glycoside biosynthesis. Simultaneously, the expression of flavonoid biosynthesis-related genes, including PAL (Gb_10949, Gb_21115) and FLS (Gb_00285, Gb_14024, and Gb_14029), was significantly upregulated. Based on these results, we reveal that G. biloba seedlings can tolerate low-level soil salinity stress through the regulation of different kinds of genes and transcriptome factors, especially flavonoid biosynthesis, which is improved to respond to environmental stress

Physiological, Transcriptomic, and Metabolic Responses of Ginkgo biloba L. to Drought, Salt, and Heat Stresses

Ginkgo biloba L. is highly adaptable and resistant to a range of abiotic stressors, allowing its growth in various environments. However, it is unclear how G. biloba responds to common environmental stresses. We explored the physiological, transcriptomic, and metabolic responses of G. biloba to short-term drought, salt, and heat stresses. Proline, H2O2, and ABA contents, along with CAT activity, increased under all three types of stress. SOD activity increased under salt and heat stresses, while soluble protein and IAA contents decreased under drought and salt stresses. With respect to metabolites, D-glyceric acid increased in response to drought and salt stresses, whereas isomaltose 1, oxalamide, and threonine 2 increased under drought. Piceatannol 2,4-hydroxybutyrate and 1,3-diaminopropane increased under salt stress, whereas 4-aminobutyric acid 1 and galactonic acid increased in response to heat stress. Genes regulating nitrogen assimilation were upregulated only under drought, while the GRAS gene was upregulated under all three types of stressors. ARF genes were downregulated under heat stress, whereas genes encoding HSF and SPL were upregulated. Additionally, we predicted that miR156, miR160, miR172, and their target genes participate in stress responses. Our study provides valuable data for studying the multilevel response to drought, salinity, and heat in G. biloba

Genome-Wide Identification of Circular RNAs in Arabidopsis thaliana

Circular RNAs (circRNAs) are a family of transcripts with covalently closed circular structures and still largely unknown functions. Large numbers of circRNAs have been found in various biological processes in humans and animals, but fewer circRNAs have been identified in plants. We performed a genome-wide analysis of circRNAs in Arabidopsis thaliana via deep sequencing. We constructed 14 strand-specific libraries from 13 samples of plants from four developmental stages, four stress treatments, and five organs and a mixed sample across the lifespan. In total, we identified 5861 circRNAs, including 1275 novel ones, using the strict threshold of at least two unique back-spliced supporting reads. The circRNAs were non-randomly distributed in all chromosomes; most were exonic. Sequence similarity analysis of circRNAs between A. thaliana and four other species showed that some circRNAs are conserved in plants. Functional annotation indicated that many parent genes of circRNAs are involved in many fundamental processes including plant development, reproduction, and response to stimulus. In addition, a small proportion of circRNAs was shown to be potential targets of miRNAs, indicating that the circRNAs could interact with miRNAs to regulate gene expression. qRT-PCR analysis revealed that circRNAs displayed diverse expression patterns at different growth stages. Our results provide an important resource for continuing circRNA research in A. thaliana, and should enhance our understanding of circRNAs in plants

Comparison of Stomatal Structure and Distribution between Ovules and Leaves in Ginkgo biloba

Stomata are plant epidermal structures that play essential roles in photosynthesis, respiration, and transpiration. Although stomata on plant leaves have been extensively studied, their structure and distribution on other organs remain poorly understood. The living fossil , Ginkgo biloba, has naked ovules that are thought to be primitive reproductive structures in ancient seed plants. Therefore, we hypothesized that there are some distinct stoma features in G. biloba ovules that have not been reported. In this study, we investigated the morphological development of stomata on ovules and leaves of Ginkgo biloba using scanning electron microscopy, then examined the anatomical characteristics of the general stalk and petiole using semi-thin sectioning. We found that stomata were distributed on the epidermis of the whole ovule, except near the micropyle; these stomata persisted until harvest, indicating that ovules perform gross photosynthesis to an extent similar to the photosynthesis observed in leaves, which is beneficial to ovule development. Ovule and leaf stomata share similar orientation, composition, and development; however, their distribution and subsidiary cell morphology significantly differ. The morphology of the general stalk was similar to the morphology of the petiole, but xylem cell development was minimal, and no sclerenchyma cells were present beneath the epidermis; these findings suggested that the general stalk is biomechanically weaker than the petiole. Overall, these results suggest that despite their differences, G. biloba ovules and leaves share many morphological and anatomical similarities in terms of stomatal architecture and stalk anatomy. These findings will help to elucidate the leaf origins of flowers in ancient plants