AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

Evaluating the general abilities of foundation models to tackle human-level tasks is a vital aspect of their development and application in the pursuit of Artificial General Intelligence (AGI). Traditional benchmarks, which rely on artificial datasets, may not accurately represent human-level capabilities. In this paper, we introduce AGIEval, a novel benchmark specifically designed to assess foundation model in the context of human-centric standardized exams, such as college entrance exams, law school admission tests, math competitions, and lawyer qualification tests. We evaluate several state-of-the-art foundation models, including GPT-4, ChatGPT, and Text-Davinci-003, using this benchmark. Impressively, GPT-4 surpasses average human performance on SAT, LSAT, and math competitions, attaining a 95% accuracy rate on the SAT Math test and a 92.5% accuracy on the English test of the Chinese national college entrance exam. This demonstrates the extraordinary performance of contemporary foundation models. In contrast, we also find that GPT-4 is less proficient in tasks that require complex reasoning or specific domain knowledge. Our comprehensive analyses of model capabilities (understanding, knowledge, reasoning, and calculation) reveal these models' strengths and limitations, providing valuable insights into future directions for enhancing their general capabilities. By concentrating on tasks pertinent to human cognition and decision-making, our benchmark delivers a more meaningful and robust evaluation of foundation models' performance in real-world scenarios. The data, code, and all model outputs are released in https://github.com/microsoft/AGIEval.Comment: 19 page

Influence of Gamma Radiation on the Damping Property of Magnetorheological Elastomer

Magnetorheological elastomer (MRE) is a kind of smart material, whose mechanical property can be controlled by the external magnetic field quickly and reversibly. The damping property of MRE is one of the most concerned properties when designing MRE based devices. In this work, the influence of gamma radiation on the damping property of MRE was investigated. Six different exposures of gamma radiation were applied to the MRE samples. The highest gamma radiation dose was up to 1 × 105 Gy(Si), which can cover most of the engineering application scenarios. The influence of gamma radiation on the damping-strain relation and the damping-magnetic-field relation were studied. The probable mechanisms were discussed in detail. It is found that the gamma radiation does not affect the variation trend of loss factor of MRE with increasing strain amplitude or magnetic flux density. But it affects the variation trend of the maximum change of strain-induced or magnetic-field-induced loss factor of MRE. Besides, with constant strain and constant magnetic flux density, the loss factor of MRE shows w-shape variation trend with increasing gamma radiation dose. It is considered to be resulted from the combined action of the intrinsic damping and the interfacial friction damping of MRE

Evaluation of Four Genetic Variants in Han Chinese Subjects with High Myopia

High myopia is one of the leading causes of blindness worldwide. However, the exact etiology of high myopia remains unraveled despite numerous attempts of elucidation. Previous genome-wide association study (GWAS) has revealed that four single nucleotide polymorphisms (SNPs), including rs2969180, rs1652333, rs9307551, and rs7837791, were associated with high myopia in Caucasians. The present study was conducted to investigate whether these genetic variants were associated with high myopia in Han Chinese. These four SNPs were genotyped by SNaPshot method in a Han Chinese cohort composed of 827 patients with high myopia and 988 healthy controls. Among the SNPs genotyped, only rs9307551 was found to be significantly associated with high myopia in this study. Carriers of rs9307551A allele, AA, and AC genotypes had an increased risk of high myopia (OR = 1.33, 95% CI 1.14–1.54; OR = 1.75, 95% CI 1.28–2.38; OR = 1.59, 95% CI 1.24–2.01, resp.). Interestingly, when split by gender, the association between rs9307551 and high myopia proved to be gender-specific with significance observed only in females but not males. These findings suggested that the SNP of rs9307551 showed a gender-specific association with high myopia in the Han Chinese population. In addition, LOC100506035, a lincRNA gene, might play a crucial role in the susceptibility to high myopia

Small Activating RNA Modulation of the G Protein‐Coupled Receptor for Cancer Treatment

International audienceG protein-coupled receptors (GPCRs) are the most common and important drug targets. However, >70% of GPCRs are undruggable or difficult to target using conventional chemical agonists/antagonists. Small nucleic acid molecules, which can sequence-specifically modulate any gene, offer a unique opportunity to effectively expand drug targets, especially those that are undruggable or difficult to address, such as GPCRs. Here, the authors report for the first time that small activating RNAs (saRNAs) effectively modulate a GPCR for cancer treatment. Specifically, saRNAs promoting the expression of Mas receptor (MAS1), a GPCR that counteracts the classical angiotensin II pathway in cancer cell proliferation and migration, are identified. These saRNAs, delivered by an amphiphilic dendrimer vector, enhance MAS1 expression, counteracting the angiotensin II/angiotensin II Receptor Type 1 axis, and leading to significant suppression of tumorigenesis and the inhibition of tumor progression of multiple cancers in tumor-xenografted mouse models and patient-derived tumor models. This study provides not only a new strategy for cancer therapy by targeting the renin-angiotensin system, but also a new avenue to modulate GPCR signaling by RNA activation