EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data

Sample efficiency remains a crucial challenge in applying Reinforcement Learning (RL) to real-world tasks. While recent algorithms have made significant strides in improving sample efficiency, none have achieved consistently superior performance across diverse domains. In this paper, we introduce EfficientZero V2, a general framework designed for sample-efficient RL algorithms. We have expanded the performance of EfficientZero to multiple domains, encompassing both continuous and discrete actions, as well as visual and low-dimensional inputs. With a series of improvements we propose, EfficientZero V2 outperforms the current state-of-the-art (SOTA) by a significant margin in diverse tasks under the limited data setting. EfficientZero V2 exhibits a notable advancement over the prevailing general algorithm, DreamerV3, achieving superior outcomes in 50 of 66 evaluated tasks across diverse benchmarks, such as Atari 100k, Proprio Control, and Vision Control.Comment: 21 pages,10 figure

Exploring the Diversity of Music Experiences for Deaf and Hard of Hearing People

Sensory substitution or enhancement techniques have been proposed to enable deaf or hard of hearing (DHH) people to listen to and even compose music. However, little is known about how such techniques enhance DHH people's music experience. Since deafness is a spectrum -- as are DHH people's preferences and perceptions of music -- a more situated understanding of their interaction with music is needed. To understand the music experience of this population, we conducted social media analyses, both qualitatively and quantitatively, in the deaf and hard of hearing Reddit communities. Our content analysis revealed that DHH people leveraged sign language and visual/haptic cues to feel the music and preferred familiar, non-lyrical, instrument-heavy, and loud music. In addition, hearing aids were not customized for music, and the visual/haptic techniques developed were not widely adopted by DHH people, leading to their suboptimal music experiences. The DHH community embodied mutual support among music lovers, evidenced by active information sharing and Q&A around music and hearing loss. We reflect on design justice for DHH people's music experience and propose practical design implications to create a more accessible music experience for them

PET Tracer Conversion among Brain PET via Variable Augmented Invertible Network

Positron emission tomography (PET) serves as an essential tool for diagnosis of encephalopathy and brain science research. However, it suffers from the limited choice of tracers. Nowadays, with the wide application of PET imaging in neuropsychiatric treatment, 6-18F-fluoro-3, 4-dihydroxy-L-phenylalanine (DOPA) has been found to be more effective than 18F-labeled fluorine-2-deoxyglucose (FDG) in the field. Nevertheless, due to the complexity of its preparation and other limitations, DOPA is far less widely used than FDG. To address this issue, a tracer conversion invertible neural network (TC-INN) for image projection is developed to map FDG images to DOPA images through deep learning. More diagnostic information is obtained by generating PET images from FDG to DOPA. Specifically, the proposed TC-INN consists of two separate phases, one for training traceable data, the other for rebuilding new data. The reference DOPA PET image is used as a learning target for the corresponding network during the training process of tracer conversion. Meanwhile, the invertible network iteratively estimates the resultant DOPA PET data and compares it to the reference DOPA PET data. Notably, the reversible model employs variable enhancement technique to achieve better power generation. Moreover, image registration needs to be performed before training due to the angular deviation of the acquired FDG and DOPA data information. Experimental results exhibited excellent generation capability in mapping between FDG and DOPA, suggesting that PET tracer conversion has great potential in the case of limited tracer applications

Real-time scheduling of renewable power systems through planning-based reinforcement learning

The growing renewable energy sources have posed significant challenges to traditional power scheduling. It is difficult for operators to obtain accurate day-ahead forecasts of renewable generation, thereby requiring the future scheduling system to make real-time scheduling decisions aligning with ultra-short-term forecasts. Restricted by the computation speed, traditional optimization-based methods can not solve this problem. Recent developments in reinforcement learning (RL) have demonstrated the potential to solve this challenge. However, the existing RL methods are inadequate in terms of constraint complexity, algorithm performance, and environment fidelity. We are the first to propose a systematic solution based on the state-of-the-art reinforcement learning algorithm and the real power grid environment. The proposed approach enables planning and finer time resolution adjustments of power generators, including unit commitment and economic dispatch, thus increasing the grid's ability to admit more renewable energy. The well-trained scheduling agent significantly reduces renewable curtailment and load shedding, which are issues arising from traditional scheduling's reliance on inaccurate day-ahead forecasts. High-frequency control decisions exploit the existing units' flexibility, reducing the power grid's dependence on hardware transformations and saving investment and operating costs, as demonstrated in experimental results. This research exhibits the potential of reinforcement learning in promoting low-carbon and intelligent power systems and represents a solid step toward sustainable electricity generation.Comment: 12 pages, 7 figure

Testing the Hypothesis of Multiple Origins of Holoparasitism in Orobanchaceae: Phylogenetic Evidence from the Last Two Unplaced Holoparasitic Genera, Gleadovia and Phacellanthus

Orobanchaceae is the largest family among the parasitic angiosperms. It comprises non-parasites, hemi- and holoparasites, making this family an ideal test case for studying the evolution of parasitism. Previous phylogenetic analyses showed that holoparasitism had arisen at least three times from the hemiparasitic taxa in Orobanchaceae. Until now, however, not all known genera of Orobanchaceae were investigated in detail. Among them, the unknown phylogenetic positions of the holoparasites Gleadovia and Phacellanthus are the key to testing how many times holoparasitism evolved. Here, we provide clear evidence for the first time that they are members of the tribe Orobancheae, using sequence data from multiple loci (nuclear genes ITS, PHYA, PHYB, and plastid genes rps2, matK). Gleadovia is an independent lineage whereas Phacellanthus should be merged into genus Orobanche section Orobanche. Our results unambiguously support the hypothesis that there are only three origins of holoparasitism in Orobanchaceae. Divergence dating reveals for the first time that the three origins of holoparasitism were not synchronous. Our findings suggest that holoparasitism can persist in specific clades for a long time and holoparasitism may evolve independently as an adaptation to certain hosts

Impatiens yingjingensis (Balsaminaceae), a new species from Sichuan, China

This study describes Impatiens yingjingensis X.Q. Song, B.N. Song & Biao Yang, sp. nov., a new species collected from the Yingjing area of the Giant Panda National Park. This new species is distributed at an altitude of 1400–2100 m, with a plant height of 30–130 cm. The flowers are purple-red or light purple red, with 3–9 flowers on each inflorescence and the dorsal auricle of the lateral united petals is thread-like and about 2 cm long, differing significantly from other species of Impatiens. Furthermore, molecular data, as well as micro-morphological evidence under SEM (of pollens), also support the establishment of the new species

Optimization of ultrasonic-assisted extraction of polysaccharides and triterpenoids from the medicinal mushroom Ganoderma lucidum and evaluation of their in vitro antioxidant capacities.

Ganoderma lucidum (Fr.) Krast, commonly known as "Lingzhi" in Chinese, is a medicinal mushroom that is rich in biologically active substances. Polysaccharides and triterpenoids are the two major components responsible for the bioactivity of this fungus. In the present study, the ultrasonic-assisted co-extraction (UACE) of polysaccharides and triterpenoids from G. lucidum was optimized using response surface methodology with a desirability function, with the equal importance for the two components. Following single factor experiments, the optimal conditions were determine as ultrasonic power of 210 W, extraction temperature of 80C, ratio of liquid to solid of 50 mL/g, and 100 min extraction time, using aqueous ethanol (50%, v/v) as the extracting solvent. Under the optimal conditions, the extraction yields of polysaccharides and triterpenoids reached 0.63% and 0.38%, respectively. On the basis of the scavenging capacity of 2,2-diphenyl-1-picrylhydrazyl and evaluation of reducing power, the antioxidant capacities of the polysaccharides obtained by optimal UACE process were higher than those of polysaccharides extracted using traditional hot water extraction, whereas the triterpenoid-rich extracts showed antioxidant activities similar to those obtained using the ethanol maceration method. The present study is the first report on the simultaneous extraction of polysaccharides and triterpenoids from G. lucidum. The developed UACE process could be useful in preparation of a polysaccharide- and triterpenoid-rich ingredient that holds great promise for application in the Ganoderma industry

The complete chloroplast genome of Fraxinus hupehensis and phylogenic analysis of Lamiales

Fraxinus hupehensis is a national rare and Endangered tree in the Oleaceae family, that has high commercial value owing to its slow growth, interlaced roots, intricate tree shape, and easy to shape. Here, we determined the complete chloroplast (cp) genome sequence for F. hupehensis using genome skimming sequencing. The cp genome was 155,698 bp and consisted of a large single copy (LSC) region (86,498 bp), a small single copy (SSC) region (17,803 bp) and two inverted repeats (IRs) (25,694 bp). It encodes 131 genes, including 88 protein-coding genes, 8 rRNAs and 35 tRNAs. Phylogenetic analysis indicates that F. hupehensis was relatively closely related to F. chinensis compared to other species in the Oleaceae family

Generating the Isocurve Representation for Configuration Space of Mechanisms

This paper proposes an approach for generating a set of isocurves as a representation for configuration space (CS) of a mechanism. An isocurve here is a curve in CS with some parameters fixed. Compared with conventional methods like box approximations, sampling points, and boundary tessellations, the isocurve-based representation has some advantages in space parameterization and data management. This approach directly formulates the joint loop equations in the form of kinematic matrices, which does not need any extra conversions and solves the equations with an isocurve-tracing method applying ODE solvers. Since the isocurves are connected in certain orders with the guide isocurves in a lower-dimensional space, tracing all the isocurves only needs one initial solution point for an isolated solution component. In addition, the proposed approach includes an interference-handling step, which trims off the collision portions of the isocurves by checking their feasibility according to the previously defined half-space constraints, and a measure for identifying the forward direction at singular points where the first-order derivatives vanish. The approach is implemented through programming and the results for a few examples show its effectiveness

Mathematical modelling of the effects of statins on the growth of necrotic core in atherosclerotic plaque

A large necrotic core increases the risk of atherosclerotic plaque instability. Statins can delay the growth of necrotic core in plaques, but the kinetic mechanism of statins in slowing down the necrotic core has not yet been addressed in detail. In this paper, a mathematical model is governed by a system of advection-diffusion-reaction equations coupling of the porous nature of vessel wall is established and applied to illustrate the plaque growth with lipid-rich necrotic core (LRNC) with and without statins using finite element method. We study the influence of LRNC plaque growth for different drug concentrations at different time intervals. The results showed that the drug use at different time points has a significant impact on the treatment efficacy. Compared with short-term, low-dose treatment, early statin treatment with high dose showed more pronounced effects on reducing the low-density lipoprotein (LDL) cholesterol, decreasing the volume of necrotic core, changing the characteristics of plaques, and improving the plaque stability. The model is validated by comparing with the clinical data, and may be used to predict the progression of LRNC plaque and the effects of statin therapy