An Overview of Catastrophic AI Risks

Rapid advancements in artificial intelligence (AI) have sparked growing concerns among experts, policymakers, and world leaders regarding the potential for increasingly advanced AI systems to pose catastrophic risks. Although numerous risks have been detailed separately, there is a pressing need for a systematic discussion and illustration of the potential dangers to better inform efforts to mitigate them. This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories: malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs; organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents; and rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans. For each category of risk, we describe specific hazards, present illustrative stories, envision ideal scenarios, and propose practical suggestions for mitigating these dangers. Our goal is to foster a comprehensive understanding of these risks and inspire collective and proactive efforts to ensure that AIs are developed and deployed in a safe manner. Ultimately, we hope this will allow us to realize the benefits of this powerful technology while minimizing the potential for catastrophic outcomes

Optimized media and workflow for the expansion of human pluripotent stem cells as aggregates in suspension cultures

3D suspension culture enables scale-up of human pluripotent stem cell (hPSC) manufacturing. However, media and methods optimized for 2D adherent cultures can lead to low volumetric productivity and laborious workflow in suspension cultures. To overcome these limitations we developed fed-batch media based on either mTeSRTM1 (BSA-containing) or TeSRTM-E8TM (animal component-free) for hPSC expansion as aggregates in suspension cultures. Fed-batch feeding protocols are more efficient and cost-effective than batch media changes because only exhausted components are replenished. Optimization studies were performed using human embryonic (H7 and H9), and human induced pluripotent (WLS-1C and STiPS-M001) stem cell lines. Suspension cultures were fed daily using either 50% medium exchanges of standard 2D media, or fed-batch optimized media and protocols. hPSC aggregate diameter must be kept below 350 μm to maintain cell viability and phenotype. With observed growth rates, aggregates required passaging every 3 or 4 days into clumps of 5-10 cells with Gentle Cell Dissociation Reagent. Clumps were re-seeded into fresh test medium plus 10 μM Y-27632. Passaging and feeding cycles were repeated for at least 5 passages. Optimization was performed by iteratively modifying the feed solution to maintain consistent nutrient levels and maximal growth rate while maintaining cell quality. Control and optimized fed-batch formulations demonstrated between 1.4 and 1.8-fold expansion per day, \u3e90% viability, Oct4 and TRA-1-60 expression \u3e90%, in vitro trilineage differentiation, and normal karyotype (n=8 independent cultures). Suspension culture optimized mTeSRTM-3D or TeSRTM-E8TM3D fed-batch media enables the cost-effective production of hPSCs as aggregates with efficient workflow and high cell quality

Optimized media and workflow for the expansion of human pluripotent stem cells as aggregates in suspension

3D suspension culture enables the efficient and cost-effective scale-up of human pluripotent stem cell (hPSCs) manufacturing. However, media optimized for 2D adherent cultures can lead to low volumetric productivity and inefficient workflow. To overcome these limitations we developed mTeSRTM3D, a defined medium based on mTeSRTM1, and novel protocols for fed-batch culture of hPSC aggregates. Human embryonic stem cell (hESC) lines (H1 or H9) or human induced pluripotent stem cell (hiPSC) lines (WLS-1C or STiPS-M001) that were previously maintained in 2D mTeSRTM1 culture were seeded into multiple suspension culture vessels containing mTeSRTM3D Seed Medium plus 10 μM Y-27632 ROCK inhibitor. 3D cultures were maintained using either daily 50% mTeSRTM1 medium exchanges (control) or using a fed-batch protocol whereby the culture medium was supplemented daily with mTeSRTM3D Feed Medium. After 3 or 4 days in suspension culture, aggregates were harvested, dissociated into small clumps with Gentle Cell Dissociation Reagent (GCDR) or single cell suspensions enzymatically, and re-seeded in mTeSRTM3D Seed Medium plus 10 μM Y-27632. Passaging and feeding cycles were repeated for at least 5 passages. 3D cultures were assessed for growth, viability, hPSC marker expression, in vitro differentiation potential, and karyotype. In addition, media was analyzed for molar glucose to lactate yield to characterize metabolism. By day 4, aggregates cultured in mTeSRTM3D typically grew to a mean diameter of 350 μm, with a 5-fold increase in cell number. Using mTeSRTM3D up to 109 cells can be produced from a single plate within 2-3 weeks representing a greater than 500-fold expansion. hPSC cultures maintained in mTeSRTM3D differentiated into all 3 germ layers with high efficiency. The average volumetric productivities were 0.7, 3.1 and 6.9 (x105) viable cells / mL in 2D, daily 50% media exchange, and mTeSRTM3D cultures, respectively. Using the GCDR clump passaging protocol, mTeSRTM3D cultured hPSCs retained normal karyotypes. Culture performance was evaluated in shaker bottles, spinner flasks and bioreactors. Performance in each culture system was comparable confirming straightforward scale-up and wide applicability. Typical growth rates were on the order of 1.5-fold expansion per day. Metabolic activity as assessed by the moles lactate produced to glucose consumed was 1.7, consistent with a primarily glycolytic metabolism. Image analysis was performed to estimate aggregate size during growth. Adaptation times for cells moving from 2D to 3D aggregate culture varied with different cell lines; typically one passage in 3D was required before consistent expansion passage over passage was obtained. Additionally, protocols were developed for use on a Hamilton® robotic platform for reproducible, matrix-free, high-throughput hPSC suspension culture at a small scale. mTeSRTM3D enables efficient scale-up and scale-down of hPSC cultures with greatly simplified workflow

Oxy-Combustion Burner and Integrated Pollutant Removal Research and Development Test Facility

A high flame temperature oxy-combustion test facility consisting of a 5 MWe equivalent test boiler facility and 20 KWe equivalent IPR® was constructed at the Hammond, Indiana manufacturing site. The test facility was operated natural gas and coal fuels and parametric studies were performed to determine the optimal performance conditions and generated the necessary technical data required to demonstrate the technologies are viable for technical and economic scale-up. Flame temperatures between 4930-6120F were achieved with high flame temperature oxy-natural gas combustion depending on whether additional recirculated flue gases are added to balance the heat transfer. For high flame temperature oxy-coal combustion, flame temperatures in excess of 4500F were achieved and demonstrated to be consistent with computational fluid dynamic modeling of the burner system. The project demonstrated feasibility and effectiveness of the Jupiter Oxygen high flame temperature oxy-combustion process with Integrated Pollutant Removal process for CCS and CCUS. With these technologies total parasitic power requirements for both oxygen production and carbon capture currently are in the range of 20% of the gross power output. The Jupiter Oxygen high flame temperature oxy-combustion process has been demonstrated at a Technology Readiness Level of 6 and is ready for commencement of a demonstration project

Oxy-Combustion Burner and Integrated Pollutant Removal Research and Development Test Facility

A high flame temperature oxy-combustion test facility consisting of a 5 MWe equivalent test boiler facility and 20 KWe equivalent IPR® was constructed at the Hammond, Indiana manufacturing site. The test facility was operated natural gas and coal fuels and parametric studies were performed to determine the optimal performance conditions and generated the necessary technical data required to demonstrate the technologies are viable for technical and economic scale-up. Flame temperatures between 4930-6120F were achieved with high flame temperature oxy-natural gas combustion depending on whether additional recirculated flue gases are added to balance the heat transfer. For high flame temperature oxy-coal combustion, flame temperatures in excess of 4500F were achieved and demonstrated to be consistent with computational fluid dynamic modeling of the burner system. The project demonstrated feasibility and effectiveness of the Jupiter Oxygen high flame temperature oxy-combustion process with Integrated Pollutant Removal process for CCS and CCUS. With these technologies total parasitic power requirements for both oxygen production and carbon capture currently are in the range of 20% of the gross power output. The Jupiter Oxygen high flame temperature oxy-combustion process has been demonstrated at a Technology Readiness Level of 6 and is ready for commencement of a demonstration project

Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark

Artificial agents have traditionally been trained to maximize reward, which may incentivize power-seeking and deception, analogous to how next-token prediction in language models (LMs) may incentivize toxicity. So do agents naturally learn to be Machiavellian? And how do we measure these behaviors in general-purpose models such as GPT-4? Towards answering these questions, we introduce MACHIAVELLI, a benchmark of 134 Choose-Your-Own-Adventure games containing over half a million rich, diverse scenarios that center on social decision-making. Scenario labeling is automated with LMs, which are more performant than human annotators. We mathematize dozens of harmful behaviors and use our annotations to evaluate agents' tendencies to be power-seeking, cause disutility, and commit ethical violations. We observe some tension between maximizing reward and behaving ethically. To improve this trade-off, we investigate LM-based methods to steer agents' towards less harmful behaviors. Our results show that agents can both act competently and morally, so concrete progress can currently be made in machine ethics--designing agents that are Pareto improvements in both safety and capabilities.Comment: ICML 2023 Oral; 31 pages, 5 figure

Automated Brainstem Segmentation Detects Differential Involvement in Atypical Parkinsonian Syndromes

OBJECTIVE: Brainstem segmentation has been useful in identifying potential imaging biomarkers for diagnosis and progression in atypical parkinsonian syndromes (APS). However, the majority of work has been performed using manual segmentation, which is time consuming for large cohorts. METHODS: We investigated brainstem involvement in APS using an automated method. We measured the volume of the medulla, pons, superior cerebellar peduncle (SCP) and midbrain from T1-weighted MRIs in 67 patients and 42 controls. Diagnoses were corticobasal syndrome (CBS, n = 14), multiple system atrophy (MSA, n = 16: 8 with parkinsonian syndrome, MSA-P; 8 with cerebellar syndrome, MSA-C), progressive supranuclear palsy with a Richardson’s syndrome (PSP-RS, n = 12), variant PSP (n = 18), and APS not otherwise specified (APS-NOS, n = 7). RESULTS: All brainstem regions were smaller in MSA-C (19–42% volume difference, p < 0.0005) and in both PSP groups (18–33%, p < 0.0005) than in controls. MSA-P showed lower volumes in all regions except the SCP (15–26%, p < 0.0005). The most affected region in MSA-C and MSA-P was the pons (42% and 26%, respectively), while the most affected regions in both the PSP-RS and variant PSP groups were the SCP (33% and 23%, respectively) and midbrain (26% and 24%, respectively). The brainstem was less affected in CBS, but nonetheless, the pons (14%, p < 0.0005), midbrain (14%, p < 0.0005) and medulla (10%, p = 0.001) were significantly smaller in CBS than in controls. The brainstem was unaffected in APS-NOS. CONCLUSION: Automated methods can accurately quantify the involvement of brainstem structures in APS. This will be important in future trials with large patient numbers where manual segmentation is unfeasible

Proximity extension assay testing reveals novel diagnostic biomarkers of atypical parkinsonian syndromes.

OBJECTIVE: The high degree of clinical overlap between atypical parkinsonian syndromes (APS) and Parkinson's disease (PD) makes diagnosis challenging. We aimed to identify novel diagnostic protein biomarkers of APS using multiplex proximity extension assay (PEA) testing. METHODS: Cerebrospinal fluid (CSF) samples from two independent cohorts, each consisting of APS and PD cases, and controls, were analysed for neurofilament light chain (NF-L) and Olink Neurology and Inflammation PEA biomarker panels. Whole-cohort comparisons of biomarker concentrations were made between APS (n=114), PD (n=37) and control (n=34) groups using logistic regression analyses that included gender, age and disease duration as covariates. RESULTS: APS versus controls analyses revealed 11 CSF markers with significantly different levels in cases and controls (p<0.002). Four of these markers also reached significance (p<0.05) in APS versus PD analyses. Disease-specific analyses revealed lower group levels of FGF-5, FGF-19 and SPOCK1 in multiple system atrophy compared with progressive supranuclear palsy and corticobasal syndrome. Receiver operating characteristic curve analyses suggested that the diagnostic accuracy of NF-L was superior to the significant PEA biomarkers in distinguishing APS, PD and controls. The biological processes regulated by the significant proteins include cell differentiation and immune cell migration. Delta and notch-like epidermal growth factor-related receptor (DNER) had the strongest effect size in APS versus controls and APS versus PD analyses. DNER is highly expressed in substantia nigra and is an activator of the NOTCH1 pathway which has been implicated in the aetiology of other neurodegenerative disorders including Alzheimer's disease. CONCLUSIONS: PEA testing has identified potential novel diagnostic biomarkers of APS

The tyrosine phosphatase CD148 is an essential positive regulator of platelet activation and thrombosis

Platelets play a fundamental role in hemostasis and thrombosis. They are also involved in pathologic conditions resulting from blocked blood vessels, including myocardial infarction and ischemic stroke. Platelet adhesion, activation, and aggregation at sites of vascular injury are regulated by a diverse repertoire of tyrosine kinase–linked and G protein–coupled receptors. Src family kinases (SFKs) play a central role in initiating and propagating signaling from several platelet surface receptors; however, the underlying mechanism of how SFK activity is regulated in platelets remains unclear. CD148 is the only receptor-like protein tyrosine phosphatase identified in platelets to date. In the present study, we show that mutant mice lacking CD148 exhibited a bleeding tendency and defective arterial thrombosis. Basal SFK activity was found to be markedly reduced in CD148-deficient platelets, resulting in a global hyporesponsiveness to agonists that signal through SFKs, including collagen and fibrinogen. G protein–coupled receptor responses to thrombin and other agonists were also marginally reduced. These results highlight CD148 as a global regulator of platelet activation and a novel antithrombotic drug targe

Unfolding the recipes for conflict resolution during the new service development effort

The management of conflicts that emerge during new service development (NSD) has escaped the attention of scholars. Yet differing conflict management styles (CMS) of team members and dynamics within the team create a complex managerial challenge. Additionally, the broader literature on conflict resolution shows contradictory findings preventing a clear roadmap for practitioner use when such conflicts emerge. This study draws on complexity theory and employs fuzzy set Qualitative Comparative Analysis, drawing on data from 543 members of 116 NSD projects, to unravel conflict resolution recipes. The results reveal, in detail, the variety of causal patterns that explain the linkages between individual CMS, the dynamics of the team and two critical conflict characteristics; conflict intensity and frequency. Implications for theory and practice are identified and discussed