Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think -- Introducing AI Detectability Index

With the rise of prolific ChatGPT, the risk and consequences of AI-generated text has increased alarmingly. To address the inevitable question of ownership attribution for AI-generated artifacts, the US Copyright Office released a statement stating that 'If a work's traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it'. Furthermore, both the US and the EU governments have recently drafted their initial proposals regarding the regulatory framework for AI. Given this cynosural spotlight on generative AI, AI-generated text detection (AGTD) has emerged as a topic that has already received immediate attention in research, with some initial methods having been proposed, soon followed by emergence of techniques to bypass detection. This paper introduces the Counter Turing Test (CT^2), a benchmark consisting of techniques aiming to offer a comprehensive evaluation of the robustness of existing AGTD techniques. Our empirical findings unequivocally highlight the fragility of the proposed AGTD methods under scrutiny. Amidst the extensive deliberations on policy-making for regulating AI development, it is of utmost importance to assess the detectability of content generated by LLMs. Thus, to establish a quantifiable spectrum facilitating the evaluation and ranking of LLMs according to their detectability levels, we propose the AI Detectability Index (ADI). We conduct a thorough examination of 15 contemporary LLMs, empirically demonstrating that larger LLMs tend to have a higher ADI, indicating they are less detectable compared to smaller LLMs. We firmly believe that ADI holds significant value as a tool for the wider NLP community, with the potential to serve as a rubric in AI-related policy-making.Comment: EMNLP 2023 Mai

Proteomic and phosphoproteomic characterization of cardiovascular tissues after long term exposure to simulated space radiation

Introduction: It may take decades to develop cardiovascular dysfunction following exposure to high doses of ionizing radiation from medical therapy or from nuclear accidents. Since astronauts may be exposed continually to a complex space radiation environment unlike that experienced on Earth, it is unresolved whether there is a risk to cardiovascular health during long-term space exploration missions. Previously, we have described that mice exposed to a single dose of simplified Galactic Cosmic Ray (GCR 5-ion) develop cardiovascular dysfunction by 12 months post-radiation. Methods: To investigate the biological basis of this dysfunction, here we performed a quantitative mass spectrometry-based proteomics analysis of heart tissue (proteome and phosphoproteome) and plasma (proteome only) from these mice at 8 months post-radiation.Results: Differentially expressed proteins (DEPs) for irradiated versus sham irradiated samples (fold-change ≥1.2 and an adjusted p-value of ≤0.05) were identified for each proteomics data set. For the heart proteome, there were 87 significant DEPs (11 upregulated and 76 downregulated); for the heart phosphoproteome, there were 60 significant differentially phosphorylated peptides (17 upregulated and 43 downregulated); and for the plasma proteome, there was only one upregulated protein. A Gene Set Enrichment Analysis (GSEA) technique that assesses canonical pathways from BIOCARTA, KEGG, PID, REACTOME, and WikiPathways revealed significant perturbation in pathways in each data set. For the heart proteome, 166 pathways were significantly altered (36 upregulated and 130 downregulated); for the plasma proteome, there were 73 pathways significantly altered (25 upregulated and 48 downregulated); and for the phosphoproteome, there were 223 pathways significantly affected at 0.1 adjusted p-value cutoff. Pathways related to inflammation were the most highly perturbed in the heart and plasma. In line with sustained inflammation, neutrophil extracellular traps (NETs) were demonstrated to be increased in GCR 5-ion irradiated hearts at 12-month post irradiation. NETs play a fundamental role in combating bacterial pathogens, modulating inflammatory responses, inflicting damage on healthy tissues, and escalating vascular thrombosis. Discussion: These findings suggest that a single exposure to GCR5-ion results in long-lasting changes in the proteome and that these proteomic changes can potentiate acute and chronic health issues for astronauts, such as what we have previously described with late cardiac dysfunction in these mice

Impact of primary kidney disease on the effects of empagliflozin in patients with chronic kidney disease: secondary analyses of the EMPA-KIDNEY trial

Background: The EMPA KIDNEY trial showed that empagliflozin reduced the risk of the primary composite outcome of kidney disease progression or cardiovascular death in patients with chronic kidney disease mainly through slowing progression. We aimed to assess how effects of empagliflozin might differ by primary kidney disease across its broad population. Methods: EMPA-KIDNEY, a randomised, controlled, phase 3 trial, was conducted at 241 centres in eight countries (Canada, China, Germany, Italy, Japan, Malaysia, the UK, and the USA). Patients were eligible if their estimated glomerular filtration rate (eGFR) was 20 to less than 45 mL/min per 1·73 m2, or 45 to less than 90 mL/min per 1·73 m2 with a urinary albumin-to-creatinine ratio (uACR) of 200 mg/g or higher at screening. They were randomly assigned (1:1) to 10 mg oral empagliflozin once daily or matching placebo. Effects on kidney disease progression (defined as a sustained ≥40% eGFR decline from randomisation, end-stage kidney disease, a sustained eGFR below 10 mL/min per 1·73 m2, or death from kidney failure) were assessed using prespecified Cox models, and eGFR slope analyses used shared parameter models. Subgroup comparisons were performed by including relevant interaction terms in models. EMPA-KIDNEY is registered with ClinicalTrials.gov, NCT03594110. Findings: Between May 15, 2019, and April 16, 2021, 6609 participants were randomly assigned and followed up for a median of 2·0 years (IQR 1·5–2·4). Prespecified subgroupings by primary kidney disease included 2057 (31·1%) participants with diabetic kidney disease, 1669 (25·3%) with glomerular disease, 1445 (21·9%) with hypertensive or renovascular disease, and 1438 (21·8%) with other or unknown causes. Kidney disease progression occurred in 384 (11·6%) of 3304 patients in the empagliflozin group and 504 (15·2%) of 3305 patients in the placebo group (hazard ratio 0·71 [95% CI 0·62–0·81]), with no evidence that the relative effect size varied significantly by primary kidney disease (pheterogeneity=0·62). The between-group difference in chronic eGFR slopes (ie, from 2 months to final follow-up) was 1·37 mL/min per 1·73 m2 per year (95% CI 1·16–1·59), representing a 50% (42–58) reduction in the rate of chronic eGFR decline. This relative effect of empagliflozin on chronic eGFR slope was similar in analyses by different primary kidney diseases, including in explorations by type of glomerular disease and diabetes (p values for heterogeneity all >0·1). Interpretation: In a broad range of patients with chronic kidney disease at risk of progression, including a wide range of non-diabetic causes of chronic kidney disease, empagliflozin reduced risk of kidney disease progression. Relative effect sizes were broadly similar irrespective of the cause of primary kidney disease, suggesting that SGLT2 inhibitors should be part of a standard of care to minimise risk of kidney failure in chronic kidney disease. Funding: Boehringer Ingelheim, Eli Lilly, and UK Medical Research Council

Life cycle cost and economic assessment of biochar-based bioenergy production and biochar land application in Northwestern Ontario, Canada

Background Replacement of fossil fuel based energy with biochar-based bioenergy production can help reduce greenhouse gas emissions while mitigating the adverse impacts of climate change and global warming. However, the production of biochar-based bioenergy depends on a sustainable supply of biomass. Although, Northwestern Ontario has a rich and sustainable supply of woody biomass, a comprehensive life cycle cost and economic assessment of biochar-based bioenergy production technology has not been done so far in the region. Methods In this paper, we conducted a thorough life cycle cost assessment (LCCA) of biochar-based bioenergy production and its land application under four different scenarios: 1) biochar production with low feedstock availability; 2) biochar production with high feedstock availability; 3) biochar production with low feedstock availability and its land application; and 4) biochar production with high feedstock availability and its land application- using SimaPro®, EIOLCA® software and spreadsheet modeling. Based on the LCCA results, we further conducted an economic assessment for the break-even and viability of this technology over the project period. Results It was found that the economic viability of biochar-based bioenergy production system within the life cycle analysis system boundary based on study assumptions is directly dependent on costs of pyrolysis, feedstock processing (drying, grinding and pelletization) and collection on site and the value of total carbon offset provided by the system. Sensitivity analysis of transportation distance and different values of C offset showed that the system is profitable in case of high biomass availability within 200 km and when the cost of carbon sequestration exceeds CAD $60 per tonne of equivalent carbon (CO2e). Conclusions Biochar-based bioenergy system is economically viable when life cycle costs and environmental assumptions are accounted for. This study provides a medium scale slow-pyrolysis plant scenario and we recommend similar experiments with large-scale plants in order to implement the technology at industrial scale

Additional file 1: of Life cycle cost and economic assessment of biochar-based bioenergy production and biochar land application in Northwestern Ontario, Canada

Life Cycle Cost Assessment (LCCA). (XLSX 735 kb