RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalization

Visual Reinforcement Learning (Visual RL), coupled with high-dimensional observations, has consistently confronted the long-standing challenge of generalization. Despite the focus on algorithms aimed at resolving visual generalization problems, we argue that the devil is in the existing benchmarks as they are restricted to isolated tasks and generalization categories, undermining a comprehensive evaluation of agents' visual generalization capabilities. To bridge this gap, we introduce RL-ViGen: a novel Reinforcement Learning Benchmark for Visual Generalization, which contains diverse tasks and a wide spectrum of generalization types, thereby facilitating the derivation of more reliable conclusions. Furthermore, RL-ViGen incorporates the latest generalization visual RL algorithms into a unified framework, under which the experiment results indicate that no single existing algorithm has prevailed universally across tasks. Our aspiration is that RL-ViGen will serve as a catalyst in this area, and lay a foundation for the future creation of universal visual generalization RL agents suitable for real-world scenarios. Access to our code and implemented algorithms is provided at https://gemcollector.github.io/RL-ViGen/

Estimation of affinities of ligands in mixtures via magnetic recovery of target-ligand complexes and chromatographic analyses: chemometrics and an experimental model

<p>Abstract</p> <p>Background</p> <p>The combinatorial library strategy of using multiple candidate ligands in mixtures as library members is ideal in terms of cost and efficiency, but needs special screening methods to estimate the affinities of candidate ligands in such mixtures. Herein, a new method to screen candidate ligands present in unknown molar quantities in mixtures was investigated.</p> <p>Results</p> <p>The proposed method involves preparing a processed-mixture-for-screening (PMFS) with each mixture sample and an exogenous reference ligand, initiating competitive binding among ligands from the PMFS to a target immobilized on magnetic particles, recovering target-ligand complexes in equilibrium by magnetic force, extracting and concentrating bound ligands, and analyzing ligands in the PMFS and the concentrated extract by chromatography. The relative affinity of each candidate ligand to its reference ligand is estimated <it>via </it>an approximation equation assuming (a) the candidate ligand and its reference ligand bind to the same site(s) on the target, (b) their chromatographic peak areas are over five times their intercepts of linear response but within their linear ranges, (c) their binding ratios are below 10%. These prerequisites are met by optimizing primarily the quantity of the target used and the PMFS composition ratio.</p> <p>The new method was tested using the competitive binding of biotin derivatives from mixtures to streptavidin immobilized on magnetic particles as a model. Each mixture sample containing a limited number of candidate biotin derivatives with moderate differences in their molar quantities were prepared <it>via </it>parallel-combinatorial-synthesis (PCS) without purification, or <it>via </it>the pooling of individual compounds. Some purified biotin derivatives were used as reference ligands. This method showed resistance to variations in chromatographic quantification sensitivity and concentration ratios; optimized conditions to validate the approximation equation could be applied to different mixture samples. Relative affinities of candidate biotin derivatives with unknown molar quantities in each mixture sample were consistent with those estimated by a homogenous method using their purified counterparts as samples.</p> <p>Conclusions</p> <p>This new method is robust and effective for each mixture possessing a limited number of candidate ligands whose molar quantities have moderate differences, and its integration with PCS has promise to routinely practice the mixture-based library strategy.</p

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization

Visual reinforcement learning (RL) has shown promise in continuous control tasks. Despite its progress, current algorithms are still unsatisfactory in virtually every aspect of the performance such as sample efficiency, asymptotic performance, and their robustness to the choice of random seeds. In this paper, we identify a major shortcoming in existing visual RL methods that is the agents often exhibit sustained inactivity during early training, thereby limiting their ability to explore effectively. Expanding upon this crucial observation, we additionally unveil a significant correlation between the agents' inclination towards motorically inactive exploration and the absence of neuronal activity within their policy networks. To quantify this inactivity, we adopt dormant ratio as a metric to measure inactivity in the RL agent's network. Empirically, we also recognize that the dormant ratio can act as a standalone indicator of an agent's activity level, regardless of the received reward signals. Leveraging the aforementioned insights, we introduce DrM, a method that uses three core mechanisms to guide agents' exploration-exploitation trade-offs by actively minimizing the dormant ratio. Experiments demonstrate that DrM achieves significant improvements in sample efficiency and asymptotic performance with no broken seeds (76 seeds in total) across three continuous control benchmark environments, including DeepMind Control Suite, MetaWorld, and Adroit. Most importantly, DrM is the first model-free algorithm that consistently solves tasks in both the Dog and Manipulator domains from the DeepMind Control Suite as well as three dexterous hand manipulation tasks without demonstrations in Adroit, all based on pixel observations

Epidemiologic characterization of 30 confirmed cases of human infection with avian influenza A(H7N9) virus in Hangzhou, China

BACKGROUND: We examined the clinical and epidemiological characteristics of 30 cases of human infection with avian influenza A(H7N9) virus in Hangzhou and investigated their external environments to provide evidence for contact tracing and disease prevention and control. METHODS: The cases confirmed from April 1 through May 1, 2013 were studied. Field epidemiologic surveys were conducted to collect the clinical and epidemiologic data. Case-related and environmental specimens were collected for etiologic detection. RESULTS: Thirty cases of human infection with avian influenza A(H7N9) virus were confirmed in Hangzhou from April 1 through May 1, 2013, including one pregnant woman and three deaths. The median age of the patients was 62 years (range: 38–86 years). Twenty-three of the patients were men (76.67%). The median duration between disease onset and occurrence of respiratory failure and confirmed diagnosis was 5 and 6 days, respectively. Maximum medical observation of 666 close contacts of the patients revealed no irregularity. Of 314 external environmental specimens, the overall positive detection rate of H7N9 nucleic acid was 28.98%. Eight districts of Hangzhou city had positive detections in the external environments, the highest rate being in Yuhang District (78.13%). Statistical analysis of the specimen collection locations indicates a significant difference between the case-linked locations and the non-case locations (χ( 2 ) = 16.563, p < 0.05) in terms of H7N9 viral nucleic acid detection rate. No epidemiologic link has been found among the 30 cases. CONCLUSIONS: Most of the infected were retired individuals aged 60 years or older. Men made the majority. The cases are sporadic at present, with no evidence of human-to-human transmission. Exposures to poultry and live poultry markets may be important sources of infection

Synthesis of monodisperse Iron oxide nanoparticles without surfactants

Monodisperse iron oxide nanoparticles could be successfully synthesized with two kinds of precipitants through a precipitation method. As-prepared nanoparticles in the size around 10 nm with regular spherical-like shape were achieved by adjusting pH values. NaOH and NH 3 ⋅H 2 O were used as two precipitants for comparison. The average size of nanoparticles with NH 3 ⋅H 2 O precipitant got smaller and represented better dispersibility, while nanoparticles with NaOH precipitant represented better magnetic property. This work provided a simple method without using any organic solvents, organic metal salts, or surfactants which could easily obtain monodisperse nanoparticles with tunable morphology

The impact of renal function on the prognostic value of N-terminal pro–B-type natriuretic peptide in patients with coronary artery disease

Background: The impact of renal function on the prognostic value of N-terminal pro–B-type natriureticpeptide (NT-proBNP) remains unclear in coronary artery disease (CAD). This study sought toinvestigate the value of using NT-proBNP level to predict prognoses of CAD patients with differentestimated glomerular filtration rates (eGFRs).Methods: A retrospective analysis was conducted from a single registered database. 2087 consecutivepatients with CAD confirmed by coronary angiography were enrolled. The primary endpoint was allcausemortality.Results: The mean follow-up time was 26.4 ± 11.9 months and death events occurred in 197 cases.The NT-proBNP levels increased with the deterioration of renal function, as well as the optimal cutoffvalues based on eGFR stratification to predict endpoint outcome (179.4 pg/mL, 1443.0 pg/mL,3478.0 pg/mL, for eGFR ≥ 90, 60–90 and &lt; 60 mL/min/1.73 m2, respectively). Compared with theroutine cut-off value or overall optimal one, stratified optimal ones had superior predictive ability forendpoint in each eGFR group (all with the highest Youden’s J statistics). And the prognostic value becameweaker as eGFR level decreased (eGFR ≥ 90 vs. 60–90 vs. &lt; 60 mL/min/1.73 m2, odds ratio [OR]7.7; 95% confidence interval [CI] 1.7–33.9 vs. OR 4.8; 95% CI 2.7–8.5 vs. OR 3.0; 95% CI 1.5–6.2).Conclusions: This study demonstrated that NT-proBNP exhibits different predictive values for prognosisfor CAD patients with different levels of renal function. Among the assessed values, the NT-proBNPcut-off value determined using renal function improve the accuracy of the prognosis prediction of CAD.Moreover, lower eGFR is associated with a higher NT-proBNP cut-off value for prognostic prediction