Stable Bias: Analyzing Societal Representations in Diffusion Models

As machine learning-enabled Text-to-Image (TTI) systems are becoming increasingly prevalent and seeing growing adoption as commercial services, characterizing the social biases they exhibit is a necessary first step to lowering their risk of discriminatory outcomes. This evaluation, however, is made more difficult by the synthetic nature of these systems' outputs: common definitions of diversity are grounded in social categories of people living in the world, whereas the artificial depictions of fictive humans created by these systems have no inherent gender or ethnicity. To address this need, we propose a new method for exploring the social biases in TTI systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts, and comparing it to the variation engendered by spanning different professions. This allows us to (1) identify specific bias trends, (2) provide targeted scores to directly compare models in terms of diversity and representation, and (3) jointly model interdependent social variables to support a multidimensional analysis. We leverage this method to analyze images generated by 3 popular TTI systems (Dall-E 2, Stable Diffusion v 1.4 and 2) and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents. We also release the datasets and low-code interactive bias exploration platforms developed for this work, as well as the necessary tools to similarly evaluate additional TTI systems.Comment: Accepted to NeurIPS Datasets and Benchmarks 2023 (spotlight

Towards Openness Beyond Open Access: User Journeys through 3 Open AI Collaboratives

Open Artificial Intelligence (Open source AI) collaboratives offer alternative pathways for how AI can be developed beyond well-resourced technology companies and who can be a part of the process. To understand how and why they work and what additionality they bring to the landscape, we focus on three such communities, each focused on a different kind of activity around AI: building models (BigScience workshop), tools and ways of working (The Turing Way), and ecosystems (Mozilla Festival's Building Trustworthy AI Working Group). First, we document the community structures that facilitate these distributed, volunteer-led teams, comparing the collaboration styles that drive each group towards their specific goals. Through interviews with community leaders, we map user journeys for how members discover, join, contribute, and participate. Ultimately, this paper aims to highlight the diversity of AI work and workers that have come forth through these collaborations and how they offer a broader practice of openness to the AI space.Comment: Presented at the 2022 NeurIPS Workshop on Broadening Research Collaborations in M

BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model

The BigScience Workshop was a value-driven initiative that spanned one and half years of interdisciplinary research and culminated in the creation of ROOTS, a 1.6TB multilingual dataset that was used to train BLOOM, one of the largest multilingual language models to date. In addition to the technical outcomes and artifacts, the workshop fostered multidisciplinary collaborations around large models, datasets, and their analysis. This in turn led to a wide range of research publications spanning topics from ethics to law, data governance, modeling choices and distributed training. This paper focuses on the collaborative research aspects of BigScience and takes a step back to look at the challenges of large-scale participatory research, with respect to participant diversity and the tasks required to successfully carry out such a project. Our main goal is to share the lessons we learned from this experience, what we could have done better and what we did well. We show how the impact of such a social approach to scientific research goes well beyond the technical artifacts that were the basis of its inception.Comment: Presented at the 2022 NeurIPS Workshop on Broadening Research Collaborations in M

GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration

Noticing the urgent need to provide tools for fast and user-friendly qualitative analysis of large-scale textual corpora of the modern NLP, we propose to turn to the mature and well-tested methods from the domain of Information Retrieval (IR) - a research field with a long history of tackling TB-scale document collections. We discuss how Pyserini - a widely used toolkit for reproducible IR research can be integrated with the Hugging Face ecosystem of open-source AI libraries and artifacts. We leverage the existing functionalities of both platforms while proposing novel features further facilitating their integration. Our goal is to give NLP researchers tools that will allow them to develop retrieval-based instrumentation for their data analytics needs with ease and agility. We include a Jupyter Notebook-based walk through the core interoperability features, available on GitHub at https://github.com/huggingface/gaia. We then demonstrate how the ideas we present can be operationalized to create a powerful tool for qualitative data analysis in NLP. We present GAIA Search - a search engine built following previously laid out principles, giving access to four popular large-scale text collections. GAIA serves a dual purpose of illustrating the potential of methodologies we discuss but also as a standalone qualitative analysis tool that can be leveraged by NLP researchers aiming to understand datasets prior to using them in training. GAIA is hosted live on Hugging Face Spaces - https://huggingface.co/spaces/spacerini/gaia

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License

The effect of local chain stiffness on the mechanism of crystal nucleation in an oligomer melt

While the process by which a polymer crystal nucleates from the melt has been extensively studied via molecular simulation, differences in polymer models and simulated crystallization conditions have led to contradictory results. We make steps to resolve this controversy by computing low-temperature phase diagrams of oligomer melts using Wang Landau Monte Carlo simulations. Two qualitatively different crystallization mechanisms are possible depending on the local bending stiffness potential. Polymers with a discrete bending potential crystallize via a single-step mechanism, whereas polymers with a continuous bending potential can crystallize via a two-step mechanism that includes an intermediate nematic phase. Other model differences can be quantitatively accounted for using an effective volume fraction and a temperature scaled by the bending stiffness. These results suggest that at least two universality classes of nucleation exist for melts and that local chain stiffness is a key determining factor in the mechanism of nucleation

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings. The BigScience workshop, a 1-year international and multidisciplinary initiative, was formed with the goal of researching and training large language models as a values-driven undertaking, putting issues of ethics, harm, and governance in the foreground. This paper documents the data creation and curation efforts undertaken by BigScience to assemble the Responsible Open-science Open-collaboration Text Sources (ROOTS) corpus, a 1.6TB dataset spanning 59 languages that was used to train the 176-billion-parameter BigScience Large Open-science Open-access Multilingual (BLOOM) language model. We further release a large initial subset of the corpus and analyses thereof, and hope to empower large-scale monolingual and multilingual modeling projects with both the data and the processing tools, as well as stimulate research around this large multilingual corpus

Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0

International audienceIn this work, we explore whether the recently demonstrated zero-shot abilities of the T0 model extend to Named Entity Recognition for out-of-distribution languages and time periods. Using a historical newspaper corpus in 3 languages as test-bed, we use prompts to extract possible named entities. Our results show that a naive approach for prompt-based zero-shot multilingual Named Entity Recognition is error-prone, but highlights the potential of such an approach for historical languages lacking labeled datasets. Moreover, we also find that T0-like models can be probed to predict the publication date and language of a document, which could be very relevant for the study of historical texts

Ketamine, but Not the NMDAR Antagonist Lanicemine, Increases Prefrontal Global Connectivity in Depressed Patients

Background Identifying the neural correlates of ketamine treatment may facilitate and expedite the development of novel, robust, and safe rapid-acting antidepressants. Prefrontal cortex (PFC) global brain connectivity with global signal regression (GBCr) was recently identified as a putative biomarker of major depressive disorder. Accumulating evidence have repeatedly shown reduced PFC GBCr in major depressive disorder, an abnormality that appears to normalize following ketamine treatment. Methods Fifty-six unmedicated participants with major depressive disorder were randomized to intravenous placebo (normal saline; n = 18), ketamine (0.5 mg/kg; n = 19), or lanicemine (100 mg; n = 19). PFC GBCr was computed using time series from functional magnetic resonance imaging scans that were completed at baseline, during infusion, and at 24-h posttreatment. Results Compared to placebo, ketamine significantly increased average PFC GBCr during infusion ( p  = 0.01) and at 24-h posttreatment ( p  = 0.02). Lanicemine had no significant effects on GBCr during infusion ( p  = 0.45) and at 24-h posttreatment ( p  = 0.23) compared to placebo. Average delta PFC GBCr (during minus baseline) showed a pattern of positively predicting depression improvement in participants receiving ketamine ( r  = 0.44; p  = 0.06; d  = 1.0) or lanicemine ( r  = 0.55; p  = 0.01; d  = 1.3) but not those receiving placebo ( r  = −0.1; p  = 0.69; d  = 0.02). Follow-up vertex-wise analyses showed ketamine-induced GBCr increases in the dorsolateral, dorsomedial, and frontomedial PFC during infusion and in the dorsolateral and dorsomedial PFC at 24-h posttreatment ( corrected p  < 0.05). Exploratory vertex-wise analyses examining the relationship with depression improvement showed positive correlation with GBCr in the dorsal PFC during infusion and at 24-h posttreatment but negative correlation with GBCr in the ventral PFC during infusion ( uncorrected p  < 0.01). Conclusions In a randomized placebo-controlled approach, the results provide the first evidence in major depressive disorder of ketamine-induced increases in PFC GBCr during infusion and suggest that ketamine’s rapid-acting antidepressant properties are related to its acute effects on prefrontal connectivity. Overall, the study findings underscore the similarity and differences between ketamine and another N-methyl-D-aspartate receptor antagonist while proposing a pharmacoimaging paradigm for the optimization of novel rapid-acting antidepressants prior to testing in costly clinical trials