BLESS: Benchmarking Large Language Models on Sentence Simplification

We present BLESS, a comprehensive performance benchmark of the most recent state-of-the-art large language models (LLMs) on the task of text simplification (TS). We examine how well off-the-shelf LLMs can solve this challenging task, assessing a total of 44 models, differing in size, architecture, pre-training methods, and accessibility, on three test sets from different domains (Wikipedia, news, and medical) under a few-shot setting. Our analysis considers a suite of automatic metrics as well as a large-scale quantitative investigation into the types of common edit operations performed by the different models. Furthermore, we perform a manual qualitative analysis on a subset of model outputs to better gauge the quality of the generated simplifications. Our evaluation indicates that the best LLMs, despite not being trained on TS, perform comparably with state-of-the-art TS baselines. Additionally, we find that certain LLMs demonstrate a greater range and diversity of edit operations. Our performance benchmark will be available as a resource for the development of future TS methods and evaluation metrics.Comment: This paper has been accepted to EMNLP 2023 as a main long paper. 9 pages, 7 figure

Exploring people’s beliefs about the experience of time

Philosophical debates about the metaphysics of time typically revolve around two contrasting views of time. On the A-theory, time is something that itself undergoes change, as captured by the idea of the passage of time; on the B-theory, all there is to time is events standing in before/after or simultaneity relations to each other, and these temporal relations are unchanging. Philosophers typically regard the A-theory as being supported by our experience of time, and they take it that the B-theory clashes with how we experience time and therefore faces the burden of having to explain away that clash. In this paper, we investigate empirically whether these intuitions about the experience of time are shared by the general public. We asked directly for people’s subjective reports of their experience of time – in particular, whether they believe themselves to have a phenomenology as of time’s passing – and we probed their understanding of what time’s passage in fact is. We find that a majority of participants do share the aforementioned intuitions, but interestingly a minority do not

Dysplastic Bone Marrow Changes During Maintenance Therapy for Acute Leukemia

We describe the case of an 8-year-old girl with common precursor B-cell acute lymphoblastic leukemia who presented with severe pancytopenia during maintenance therapy with methotrexate and 6-mercaptopurine. The bone marrow smear showed moderate hypocellularity and trilinear dysplastic changes consistent with a diagnosis of drug toxicity, with no evidence of lymphoblasts. Flow cytometric immunophenotyping was negative for leukemic cells. Blood cell counts normalized after treatment with folinic acid. Maintenance therapy was gradually restarted and she remained well at follow-up visits. Myelotoxicity from methotrexate and 6-mercaptopurine may represent an unpredictable incident during an otherwise uneventful maintenance therapy, and may occur independently of other organ toxicities

Towards an account of intuitive time

Empirical research demonstrates that people hold many intuitive theories of the physical and biological worlds, but this research has yet to extend to the domain of time. However, in developing and motivating particular philosophical theories of time, many philosophers have made claims to the effect that adults hold an intuitive theory of time (‘common sense time’), and have suggested that a number of beliefs contribute to this theory: the Objective Now Assumption (there is an objective moment in time that is the present moment); the Past-Present-Future Difference Assumption (the past, the present, and the future are fundamentally different in nature); and the Dynamicity Assumption (time as something that undergoes constant change). We empirically examined the content of people’s beliefs about time, and whether these beliefs covary in a theory-like way. In Study 1 we drew on the three assumptions to present participants with a large number of statements about time, and explored their responses using exploratory factor analysis. We found that subsets of people’s beliefs about time covaried in interpretable ways, suggesting that they are associated with a number of latent belief constructs: Open Future, Mutable Past, Presentism, and Directionality.Study 2 successfully replicated this model in a new sample and, using latent profile analysis, found evidence of multiple and mutually incompatible intuitive theories of time. Three plausible belief profiles emerged. Approximately 60% of participants, falling into two distinct profiles, responded in a way that was broadly consistent with some aspects of common sense time. Both contained participants who endorsed Directionality and Open Future and rejected the mutability of the past; members of one profile demonstrated a noticeable degree of endorsement of Presentism, whereas members of the other rejected it. However, a significant minority of participants (approximately 40%) belonged to a profile whose members were more likely than were participants belonging to other profiles to reject aspects of common-sense time. This subpopulation appears to hold an intuitive theory of time that is somewhat more consistent with the view of the nature of time emerging from modern physics. On this view there is no objective present dividing past from future, and thus there are no fundamental differences between the past, present, and future, nor are there dynamic changes in time.While some philosophers might be read as developing theories of time based on the intuitive theories held by distinct subpopulations, we have provided evidence against the claim that there is only one such intuitive theory of time with which people typically operate. Given these differences between subpopulations, philosophers need to proceed with caution when invoking alleged intuitions about time, and might benefit from close collaboration with psychologists on how individual differences in beliefs about time can arise, the implications of such individual differences for the process of grasping scientific time, and the relation between beliefs about time and temporal biases.</p

Toward an Account of Intuitive Time

People hold intuitive theories of the physical world, such as theories of matter, energy, and motion, in the sense that they have a coherent conceptual structure supporting a network of beliefs about the domain. It is not yet clear whether people can also be said to hold a shared intuitive theory of time. Yet, philosophical debates about the metaphysical nature of time often revolve around the idea that people hold one or more ‘common sense’ assumptions about time: that there is an objective ‘now’; that the past, present, and future are fundamentally different in nature; and that time passes or flows. We empirically explored the question of whether people indeed share some or all of these assumptions by asking adults to what extent they agreed with a set of brief statements about time. Across two analyses, subsets of people’s beliefs about time were found consistently to covary in ways that suggested stable underlying conceptual dimensions related to aspects of the ‘common sense’ assumptions described by philosophers. However, distinct sub-sets of participants showed three mutually incompatible profiles of response, the most frequent of which did not closely match all of philosophers’ claims about common sense time. These exploratory studies provide a useful starting point in attempts to characterize intuitive theories of time

BLESS: Benchmarking Large Language Models on Sentence Simplification

We present BLESS, a comprehensive performance benchmark of the most recent state-of-the-art Large Language Models (LLMs) on the task of text simplification (TS). We examine how well off-the-shelf LLMs can solve this challenging task, assessing a total of 44 models, differing in size, architecture, pre-training methods, and accessibility, on three test sets from different domains (Wikipedia, news, and medical) under a few-shot setting. Our analysis considers a suite of automatic metrics, as well as a large-scale quantitative investigation into the types of common edit operations performed by the different models. Furthermore, we perform a manual qualitative analysis on a subset of model outputs to better gauge the quality of the generated simplifications. Our evaluation indicates that the best LLMs, despite not being trained on TS perform comparably with state-of-the-art TS baselines. Additionally, we find that certain LLMs demonstrate a greater range and diversity of edit operations. Our performance benchmark will be available as a resource for the development of future TS methods and evaluation metrics