"So, Tell Me What Users Want, What They Really, Really Want!"

Equating users' true needs and desires with behavioural measures of 'engagement' is problematic. However, good metrics of 'true preferences' are difficult to define, as cognitive biases make people's preferences change with context and exhibit inconsistencies over time. Yet, HCI research often glosses over the philosophical and theoretical depth of what it means to infer what users really want. In this paper, we present an alternative yet very real discussion of this issue, via a fictive dialogue between senior executives in a tech company aimed at helping people live the life they `really' want to live. How will the designers settle on a metric for their product to optimise

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Olaf Scholz was the ninth Chancellor of Germany", it will not automatically be able to answer the question, "Who was the ninth Chancellor of Germany?". Moreover, the likelihood of the correct answer ("Olaf Scholz") will not be higher than for a random name. Thus, models exhibit a basic failure of logical deduction and do not generalize a prevalent pattern in their training set (i.e. if "A is B'' occurs, "B is A" is more likely to occur). We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of 'Abyssal Melodies'" and showing that they fail to correctly answer "Who composed 'Abyssal Melodies?'". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation. We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. This shows a failure of logical deduction that we hypothesize is caused by the Reversal Curse. Code is available at https://github.com/lukasberglund/reversal_curse.Comment: 18 pages, 10 figure

Complement activation and increased anaphylatoxin receptor expression are associated with cortical grey matter lesions and the compartmentalised inflammatory response of multiple sclerosis

Background: The extent of cortical pathology is an important determinant of multiple sclerosis (MS) severity. Cortical demyelination and neurodegeneration are related to inflammation of the overlying leptomeninges, a more inflammatory CSF milieu and with parenchymal microglia and astroglia activation. These are all components of the compartmentalised inflammatory response. Compartmentalised inflammation is a feature of progressive MS, which is not targeted by disease modifying therapies. Complement is differentially expressed in the MS CSF and complement, and complement receptors, are associated with demyelination and neurodegeneration. Methods: To better understand if complement activation in the leptomeninges is associated with underlying cortical demyelination, inflammation, and microglial activation, we performed a neuropathological study of progressive MS (n = 22, 14 females), neuroinflammatory (n = 8), and non-neurological disease controls (n = 10). We then quantified the relative extent of demyelination, connective tissue inflammation, complement, and complement receptor positive microglia/macrophages. Results: Complement was elevated at the leptomeninges, subpial, and within and around vessels of the cortical grey matter. The extent of complement C1q immunoreactivity correlated with connective tissue infiltrates, whilst activation products C4d, Bb, and C3b associated with grey matter demyelination, and C3a receptor 1+ and C5a receptor 1+ microglia/macrophages closely apposed C3b labelled cells. The density of C3a receptor 1+ and C5a receptor 1+ cells was increased at the expanding edge of subpial and leukocortical lesions. C5a receptor 1+ cells expressed TNFα, iNOS and contained puncta immunoreactive for proteolipid protein, neurofilament and synaptophysin, suggesting their involvement in grey matter lesion expansion. Interpretation: The presence of products of complement activation at the brain surfaces, their association with the extent of underlying pathology and increased complement anaphylatoxin receptor positive microglia/macrophages at expanding cortical grey matter lesions, could represent a target to modify compartmentalised inflammation and cortical demyelination