Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction

Value function is the central notion of Reinforcement Learning (RL). Value estimation, especially with function approximation, can be challenging since it involves the stochasticity of environmental dynamics and reward signals that can be sparse and delayed in some cases. A typical model-free RL algorithm usually estimates the values of a policy by Temporal Difference (TD) or Monte Carlo (MC) algorithms directly from rewards, without explicitly taking dynamics into consideration. In this paper, we propose Value Decomposition with Future Prediction (VDFP), providing an explicit two-step understanding of the value estimation process: 1) first foresee the latent future, 2) and then evaluate it. We analytically decompose the value function into a latent future dynamics part and a policy-independent trajectory return part, inducing a way to model latent dynamics and returns separately in value estimation. Further, we derive a practical deep RL algorithm, consisting of a convolutional model to learn compact trajectory representation from past experiences, a conditional variational auto-encoder to predict the latent future dynamics and a convex return model that evaluates trajectory representation. In experiments, we empirically demonstrate the effectiveness of our approach for both off-policy and on-policy RL in several OpenAI Gym continuous control tasks as well as a few challenging variants with delayed reward.Comment: Accepted paper on AAAI 2021. arXiv admin note: text overlap with arXiv:1905.1110

Genome-wide identification and analysis of the invertase gene family in tobacco (Nicotiana tabacum) reveals NtNINV10 participating the sugar metabolism

Sucrose (Suc) is directly associated with plant growth and development as well as tolerance to various stresses. Invertase (INV) enzymes played important role in sucrose metabolism by irreversibly catalyzing Suc degradation. However, genome-wide identification and function of individual members of the INV gene family in Nicotiana tabacum have not been conducted. In this report, 36 non-redundant NtINV family members were identified in Nicotiana tabacum including 20 alkaline/neutral INV genes (NtNINV1-20), 4 vacuolar INV genes (NtVINV1-4), and 12 cell wall INV isoforms (NtCWINV1-12). A comprehensive analysis based on the biochemical characteristics, the exon-intron structures, the chromosomal location and the evolutionary analysis revealed the conservation and the divergence of NtINVs. For the evolution of the NtINV gene, fragment duplication and purification selection were major factors. Besides, our analysis revealed that NtINV could be regulated by miRNAs and cis-regulatory elements of transcription factors associated with multiple stress responses. In addition, 3D structure analysis has provided evidence for the differentiation between the NINV and VINV. The expression patterns in diverse tissues and under various stresses were investigated, and qRT-PCR experiments were conducted to confirm the expression patterns. Results revealed that changes in NtNINV10 expression level were induced by leaf development, drought and salinity stresses. Further examination revealed that the NtNINV10-GFP fusion protein was located in the cell membrane. Furthermore, inhibition of the expression of NtNINV10 gene decreased the glucose and fructose in tobacco leaves. Overall, we have identified possible NtINV genes functioned in leaf development and tolerance to environmental stresses in tobacco. These findings provide a better understanding of the NtINV gene family and establish the basis for future research

Tactile Sensing, Skill Learning, and Robotic Dexterous Manipulation

Tactile Sensing, Skill Learning and Robotic Dexterous Manipulation focuses on cross-disciplinary lines of research and groundbreaking research ideas in three research lines: tactile sensing, skill learning and dexterous control. The book introduces recent work about human dexterous skill representation and learning, along with discussions of tactile sensing and its applications on unknown objects’ property recognition and reconstruction. Sections also introduce the adaptive control schema and its learning by imitation and exploration. Other chapters describe the fundamental part of relevant research, paying attention to the connection among different fields and showing the state-of-the-art in related branches. The book summarizes the different approaches and discusses the pros and cons of each. Chapters not only describe the research but also include basic knowledge that can help readers understand the proposed work, making it an excellent resource for researchers and professionals who work in the robotics industry, haptics and in machine learning

Simulation of Radial Growth of Mongolian Pine (<i>Pinus sylvestris</i> var. <i>mongolica</i>) and Dahurian Larch (<i>Larix gmelinii</i>) Using the Vaganov–Shashkin Model in the Greater Khingan Range, Northeast China

It is necessary to understand the radial growth responses of trees in the Greater Khingan Range to climatic factors to predict changes in forest ecosystems under climate change scenarios. We sampled Mongolian pine (Pinus sylvestris var. mongolica) and Dahurian larch (Larix gmelinii (Rupr.) Kuzen) at four locations at elevations of 900–1100 m in the Greater Khingan Range to establish a tree-ring chronology. The Vaganov–Shashkin (VS) model was used to describe the relationship between tree radial growth and the dominant limiting climatic factors with a focus on physiological processes. The results showed that the VS model accurately reflected the effects of various climatic factors on the growth of Mongolian pine and Dahurian larch. The simulated and measured tree-ring widths index (RWI, the same as below) were consistent. The physiological parameters affecting tree growth differed for the two tree species in the study area. Mongolian pine required higher temperatures and less soil moisture for growth than Dahurian larch. The growth rings of the two tree species are more consistent across the elevation gradient. Higher-elevation trees had an “intensive strategy” with shorter growing periods and high growth rates, whereas low-elevation tree species had a “broad strategy” with lower maximum growth rates for longer periods. The start and cessation date of tree growth strongly affected the RWI of Mongolian pine and high-elevation Dahurian larch, but no significant effect on the RWI of low-elevation Dahurian larch. Differences in the limiting climatic factors were observed between Mongolian pine and Dahurian larch. Mongolian pine shows some similarity between high and low elevations, subject to the common limitations of temperature and soil moisture during the growing season for both, with a greater proportion of the lower elevations being limited by soil moisture. Dahurian larch was influenced by the growing season temperatures and May–August soil moisture at higher elevations and by the growing season soil moisture at lower elevations. This study provides a scientific basis for the management and conservation of forest ecosystems in the Greater Khingan Range

Spectroscopic properties of near-stoichiometric In:Er:LiNbO3 crystals

Natural Science Foundation of Hebei Province, China [F2009000465]The congruent and near-stoichiometric In(2 mol%):Er(1 mol%):LiNbO3 crystals have been grown by the Czochralski method from melts having compositions of 48.6 and 58 mol% Li2O, respectively. The OH- absorption spectra were characterized to investigate the structure defects of the crystals. The appearance of the 3508 cm(-1) absorption peak manifests that the threshold concentration of In3+ is below or near 2 mol% in In:Er:LiNbO3 crystal with melt [Li]/[Nb] ratio of 1.38. The [Li]/[Nb] ratios in the crystals are estimated from the shift of ultraviolet absorption edge. In addition, the influence of [Li]/[Nb] ratio on the Judd-Ofelt intensity parameter (Omega) over dot(lambda) is analyzed using the Judd-Ofelt theory. The upconversion fluorescence spectra excited by an 800 nm femtosecond laser were measured at room temperature. The results show that three emission peaks centered at 535, 551 and 673 nm exist in both crystals but an additional 400 nm peak appears in the In:Er:LiNbO3 crystal with [Li][Nb] ratio of 1.38 in melt. The 400 nm emission peak is verified experimentally to be the second harmonic generation (SHG) of 800 nm light. (C) 2012 Elsevier B.V. All rights reserved

Spatial and Temporal Variation in Primary Forest Growth in the Northern Daxing’an Mountains Based on Tree-Ring and NDVI Data

We used tree-ring width data of Larix gmelinii and Pinus sylvestris var. mongolica from the northern region of the Daxing’an Mountains, China; normalized difference vegetation index (NDVI) data; and microtopographic information (elevation, slope direction, slope gradient, and topographic location index) to assess spatiotemporal dynamics in the growth of the boreal forest and topographic patterns of forest decline under the background of climate warming. Forest growth trends were determined based on tree growth decline indicators and NDVI time series trends, and topographic patterns of forest decline were analyzed using the C5.0 decision tree model. More climatic information was present in the radial growth of the trees at higher elevations, and P. sylvestris var. mongolica was influenced strongly by climatic factors of the previous year. Since 1759, tree radial growth trends in the study area have experienced two recessions during 1878–1893 and 1935–1943, which were characterized by persistent narrow whorls of tree rings of below-average growth. Changes in NDVI and tree-ring information were similar, and they together indicate a high risk of declining forest growth in the northern Daxing’an Mountains after 2010, especially at higher elevations. The NDVI time series showed that the high temperatures in 2003 negatively affected forest growth in the study area, which was confirmed by the tree-ring data. The decision tree terrain model results had an accuracy of 0.861, and elevation was the most important terrain factor affecting forest decline. The relative importance of elevation, topographic position index, aspect, and slope was 58.41%, 17.70%, 16.81%, and 7.08%, respectively. Classification rule-based decision tree models can be used to quantify the effects of terrain factors on tree growth. This research methodology can aid the management of regional forestry resources and the conservation of forest resources under the background of climate change, which increases the risk of forest decline

Bird Object Detection: Dataset Construction, Model Performance Evaluation, and Model Lightweighting

The application of object detection technology has a positive auxiliary role in advancing the intelligence of bird recognition and enhancing the convenience of bird field surveys. However, challenges arise due to the absence of dedicated bird datasets and evaluation benchmarks. To address this, we have not only constructed the largest known bird object detection dataset, but also compared the performances of eight mainstream detection models on bird object detection tasks and proposed feasible approaches for model lightweighting in bird object detection. Our constructed bird detection dataset of GBDD1433-2023, includes 1433 globally common bird species and 148,000 manually annotated bird images. Based on this dataset, two-stage detection models like Faster R-CNN and Cascade R-CNN demonstrated superior performances, achieving a Mean Average Precision (mAP) of 73.7% compared to one-stage models. In addition, compared to one-stage object detection models, two-stage object detection models have a stronger robustness to variations in foreground image scaling and background interference in bird images. On bird counting tasks, the accuracy ranged between 60.8% to 77.2% for up to five birds in an image, but this decreased sharply beyond that count, suggesting limitations of object detection models in multi-bird counting tasks. Finally, we proposed an adaptive localization distillation method for one-stage lightweight object detection models that are suitable for offline deployment, which improved the performance of the relevant models. Overall, our work furnishes an enriched dataset and practice guidelines for selecting suitable bird detection models