A Sarsa( λ

Solving reinforcement learning problems in continuous space with function approximation is currently a research hotspot of machine learning. When dealing with the continuous space problems, the classic Q-iteration algorithms based on lookup table or function approximation converge slowly and are difficult to derive a continuous policy. To overcome the above weaknesses, we propose an algorithm named DFR-Sarsa(λ) based on double-layer fuzzy reasoning and prove its convergence. In this algorithm, the first reasoning layer uses fuzzy sets of state to compute continuous actions; the second reasoning layer uses fuzzy sets of action to compute the components of Q-value. Then, these two fuzzy layers are combined to compute the Q-value function of continuous action space. Besides, this algorithm utilizes the membership degrees of activation rules in the two fuzzy reasoning layers to update the eligibility traces. Applying DFR-Sarsa(λ) to the Mountain Car and Cart-pole Balancing problems, experimental results show that the algorithm not only can be used to get a continuous action policy, but also has a better convergence performance

Efficient Actor-Critic Algorithm with Hierarchical Model Learning and Planning

To improve the convergence rate and the sample efficiency, two efficient learning methods AC-HMLP and RAC-HMLP (AC-HMLP with l2-regularization) are proposed by combining actor-critic algorithm with hierarchical model learning and planning. The hierarchical models consisting of the local and the global models, which are learned at the same time during learning of the value function and the policy, are approximated by local linear regression (LLR) and linear function approximation (LFA), respectively. Both the local model and the global model are applied to generate samples for planning; the former is used only if the state-prediction error does not surpass the threshold at each time step, while the latter is utilized at the end of each episode. The purpose of taking both models is to improve the sample efficiency and accelerate the convergence rate of the whole algorithm through fully utilizing the local and global information. Experimentally, AC-HMLP and RAC-HMLP are compared with three representative algorithms on two Reinforcement Learning (RL) benchmark problems. The results demonstrate that they perform best in terms of convergence rate and sample efficiency

Identification of membrane protein types via deep residual hypergraph neural network

A membrane protein's functions are significantly associated with its type, so it is crucial to identify the types of membrane proteins. Conventional computational methods for identifying the species of membrane proteins tend to ignore two issues: High-order correlation among membrane proteins and the scenarios of multi-modal representations of membrane proteins, which leads to information loss. To tackle those two issues, we proposed a deep residual hypergraph neural network (DRHGNN), which enhances the hypergraph neural network (HGNN) with initial residual and identity mapping in this paper. We carried out extensive experiments on four benchmark datasets of membrane proteins. In the meantime, we compared the DRHGNN with recently developed advanced methods. Experimental results showed the better performance of DRHGNN on the membrane protein classification task on four datasets. Experiments also showed that DRHGNN can handle the over-smoothing issue with the increase of the number of model layers compared with HGNN. The code is available at https://github.com/yunfighting/Identification-of-Membrane-Protein-Types-via-deep-residual-hypergraph-neural-network

A method for measuring banana pseudo-stem phenotypic parameters based on handheld mobile LiDAR and IMU fusion

Diameter and height are crucial morphological parameters of banana pseudo-stems, serving as indicators of the plant’s growth status. Currently, in densely cultivated banana plantations, there is a lack of applicable research methods for the scalable measurement of phenotypic parameters such as diameter and height of banana pseudo-stems. This paper introduces a handheld mobile LiDAR and Inertial Measurement Unit (IMU)-fused laser scanning system designed for measuring phenotypic parameters of banana pseudo-stems within banana orchards. To address the challenges posed by dense canopy cover in banana orchards, a distance-weighted feature extraction method is proposed. This method, coupled with Lidar-IMU integration, constructs a three-dimensional point cloud map of the banana plantation area. To overcome difficulties in segmenting individual banana plants in complex environments, a combined segmentation approach is proposed, involving Euclidean clustering, Kmeans clustering, and threshold segmentation. A sliding window recognition method is presented to determine the connection points between pseudo-stems and leaves, mitigating issues caused by crown closure and heavy leaf overlap. Experimental results in banana orchards demonstrate that, compared with manual measurements, the mean absolute errors and relative errors for banana pseudo-stem diameter and height are 0.2127 cm (4.06%) and 3.52 cm (1.91%), respectively. These findings indicate that the proposed method is suitable for scalable measurements of banana pseudo-stem diameter and height in complex, obscured environments, providing a rapid and accurate inter-orchard measurement approach for banana plantation managers

The genome of broomcorn millet

Broomcorn millet (Panicum miliaceum L.) is the most water-efficient cereal and one of the earliest domesticated plants. Here we report its high-quality, chromosome-scale genome assembly using a combination of short-read sequencing, single-molecule real-time sequencing, Hi-C, and a high-density genetic map. Phylogenetic analyses reveal two sets of homologous chromosomes that may have merged ~5.6 million years ago, both of which exhibit strong synteny with other grass species. Broomcorn millet contains 55,930 proteincoding genes and 339 microRNA genes. We find Paniceae-specific expansion in several subfamilies of the BTB (broad complex/tramtrack/bric-a-brac) subunit of ubiquitin E3 ligases, suggesting enhanced regulation of protein dynamics may have contributed to the evolution of broomcorn millet. In addition, we identify the coexistence of all three C4 subtypes of carbon fixation candidate genes. The genome sequence is a valuable resource for breeders and will provide the foundation for studying the exceptional stress tolerance as well as C4 biology

Unveiling the additive-assisted oriented growth of perovskite crystallite for high performance light-emitting diodes.

Solution-processed metal halide perovskites have been recognized as one of the most promising semiconductors, with applications in light-emitting diodes (LEDs), solar cells and lasers. Various additives have been widely used in perovskite precursor solutions, aiming to improve the formed perovskite film quality through passivating defects and controlling the crystallinity. The additive's role of defect passivation has been intensively investigated, while a deep understanding of how additives influence the crystallization process of perovskites is lacking. Here, we reveal a general additive-assisted crystal formation pathway for FAPbI3 perovskite with vertical orientation, by tracking the chemical interaction in the precursor solution and crystallographic evolution during the film formation process. The resulting understanding motivates us to use a new additive with multi-functional groups, 2-(2-(2-Aminoethoxy)ethoxy)acetic acid, which can facilitate the orientated growth of perovskite and passivate defects, leading to perovskite layer with high crystallinity and low defect density and thereby record-high performance NIR perovskite LEDs (~800 nm emission peak, a peak external quantum efficiency of 22.2% with enhanced stability)

Building Energy Consumption Prediction Using a Deep-Forest-Based DQN Method

When deep reinforcement learning (DRL) methods are applied in energy consumption prediction, performance is usually improved at the cost of the increasing computation time. Specifically, the deep deterministic policy gradient (DDPG) method can achieve higher prediction accuracy than deep Q-network (DQN), but it requires more computing resources and computation time. In this paper, we proposed a deep-forest-based DQN (DF–DQN) method, which can obtain higher prediction accuracy than DDPG and take less computation time than DQN. Firstly, the original action space is replaced with the shrunken action space to efficiently find the optimal action. Secondly, deep forest (DF) is introduced to map the shrunken action space to a single sub-action space. This process can determine the specific meaning of each action in the shrunken action space to ensure the convergence of DF–DQN. Thirdly, state class probabilities obtained by DF are employed to construct new states by considering the probabilistic process of shrinking the original action space. The experimental results show that the DF–DQN method with 15 state classes outperforms other methods and takes less computation time than DRL methods. MAE, MAPE, and RMSE are decreased by 5.5%, 7.3%, and 8.9% respectively, and R2 is increased by 0.3% compared to the DDPG method

Habitat Niche-Fitness and Radix Yield Prediction Models for Angelica sinensis Cultivated in the Alpine Area of the Southeastern Region of Gansu Province, China

Dried root of Angelica sinensis has been used for thousands of years in traditional Chinese medicinal prescriptions. Researches on better knowledge of appropriate habitats for cultivation of this species are required to encourage the potential ecological sustainable industry. From 2001 to 2004, transplanting trials on the regulation of fertilizers and planting density were conducted for collection of habitat factor data at four sites of four counties in the southeastern region of Gansu Province, China. Introducing the niche theory into the research, habitat niche-fitness (HNF) is defined as the degree of similarity of an actual habitat state to the optimum habitat. A new model of HNF is constructed to evaluate the adaptive extent of A. sinensis. The results showed that the model of HNF notably outperforms the proportional similarity index and the geometric parallelism formula both in mathematical justification and biological principle testing. With HNF as a surrogate for composite environmental factors, a radix yield model was constructed. Evaluation of the present model by the sampled subplots specified for data validation proved that the model could be well used for predicted of radix yield across a wide-spread area. The radix yield prediction model and its uses are recommended within the limitations of the data used in the study area. Beyond this range, validation of the radix yield prediction model will be necessary