138 research outputs found
Generating recommendations for entity-oriented exploratory search
We introduce the task of recommendation set generation for entity-oriented
exploratory search. Given an input search query which is open-ended or
under-specified, the task is to present the user with an easily-understandable
collection of query recommendations, with the goal of facilitating domain
exploration or clarifying user intent. Traditional query recommendation systems
select recommendations by identifying salient keywords in retrieved documents,
or by querying an existing taxonomy or knowledge base for related concepts. In
this work, we build a text-to-text model capable of generating a collection of
recommendations directly, using the language model as a "soft" knowledge base
capable of proposing new concepts not found in an existing taxonomy or set of
retrieved documents. We train the model to generate recommendation sets which
optimize a cost function designed to encourage comprehensiveness,
interestingness, and non-redundancy. In thorough evaluations performed by crowd
workers, we confirm the generalizability of our approach and the high quality
of the generated recommendations
The Effect of Moderation on Online Mental Health Conversations
Many people struggling with mental health issues are unable to access
adequate care due to high costs and a shortage of mental health professionals,
leading to a global mental health crisis. Online mental health communities can
help mitigate this crisis by offering a scalable, easily accessible alternative
to in-person sessions with therapists or support groups. However, people
seeking emotional or psychological support online may be especially vulnerable
to the kinds of antisocial behavior that sometimes occur in online discussions.
Moderation can improve online discourse quality, but we lack an understanding
of its effects on online mental health conversations. In this work, we
leveraged a natural experiment, occurring across 200,000 messages from 7,000
conversations hosted on a mental health mobile application, to evaluate the
effects of moderation on online mental health discussions. We found that
participation in group mental health discussions led to improvements in
psychological perspective, and that these improvements were larger in moderated
conversations. The presence of a moderator increased user engagement,
encouraged users to discuss negative emotions more candidly, and dramatically
reduced bad behavior among chat participants. Moderation also encouraged
stronger linguistic coordination, which is indicative of trust building. In
addition, moderators who remained active in conversations were especially
successful in keeping conversations on topic. Our findings suggest that
moderation can serve as a valuable tool to improve the efficacy and safety of
online mental health conversations. Based on these findings, we discuss
implications and trade-offs involved in designing effective online spaces for
mental health support.Comment: Accepted as a full paper at ICWSM 2021. 13 pages, 12 figures, 3
table
Language Models Hallucinate, but May Excel at Fact Verification
Recent progress in natural language processing (NLP) owes much to remarkable
advances in large language models (LLMs). Nevertheless, LLMs frequently
"hallucinate," resulting in non-factual outputs. Our carefully designed human
evaluation substantiates the serious hallucination issue, revealing that even
GPT-3.5 produces factual outputs less than 25% of the time. This underscores
the importance of fact verifiers in order to measure and incentivize progress.
Our systematic investigation affirms that LLMs can be repurposed as effective
fact verifiers with strong correlations with human judgments, at least in the
Wikipedia domain. Surprisingly, FLAN-T5-11B, the least factual generator in our
study, performs the best as a fact verifier, even outperforming more capable
LLMs like GPT3.5 and ChatGPT. Delving deeper, we analyze the reliance of these
LLMs on high-quality evidence, as well as their deficiencies in robustness and
generalization ability. Our study presents insights for developing trustworthy
generation models.Comment: 9 page
Estimating the Causal Effect of Early ArXiving on Paper Acceptance
What is the effect of releasing a preprint of a paper before it is submitted
for peer review? No randomized controlled trial has been conducted, so we turn
to observational data to answer this question. We use data from the ICLR
conference (2018--2022) and apply methods from causal inference to estimate the
effect of arXiving a paper before the reviewing period (early arXiving) on its
acceptance to the conference. Adjusting for confounders such as topic, authors,
and quality, we may estimate the causal effect. However, since quality is a
challenging construct to estimate, we use the negative outcome control method,
using paper citation count as a control variable to debias the quality
confounding effect. Our results suggest that early arXiving may have a small
effect on a paper's chances of acceptance. However, this effect (when existing)
does not differ significantly across different groups of authors, as grouped by
author citation count and institute rank. This suggests that early arXiving
does not provide an advantage to any particular group.Comment: Published at CLeaR 202
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Since the release of T\"ULU [Wang et al., 2023b], open resources for
instruction tuning have developed quickly, from better base models to new
finetuning techniques. We test and incorporate a number of these advances into
T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing
the understanding and best practices of adapting pretrained language models to
downstream tasks and user preferences. Concretely, we release: (1)
T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2)
T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU
2 models trained with direct preference optimization (DPO), including the
largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE
LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its
instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple
perspectives shows that the T\"ULU 2 suite achieves state-of-the-art
performance among open models and matches or exceeds the performance of
GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data,
training and evaluation code to facilitate future open efforts on adapting
large language models.Comment: technical report; fixed zephyr number
Perceived Barriers to Weight Management in Primary CareâPerspectives of Patients and Providers
BACKGROUND: Despite the consequences of overweight and obesity, effective weight management is not occurring in primary care. OBJECTIVE: To identify beliefs about obesity that act as barriers to weight management in primary care by surveying both patients and providers and comparing their responses. DESIGN: Anonymous, cross-sectional, self-administered survey of patients and providers of a Veteranâs Administration Primary Care Clinic, distributed at the clinic site. SUBJECTS: Forty-eight Internal Medicine providers and 488 patients. MEASUREMENTS: Beliefs, attitudes, and experiences with weight management as well as demographic characteristics were collected through a questionnaire. RESULTS: Providers and patients differed significantly on many beliefs about weight. Providers were more likely than patients to perceive that patients lack self-control to stay on a diet and that fattening food in society and lack of time for exercise were prime factors in weight gain. They also expressed more interest in helping patients with weight management than patients desiring this. Patients were more likely to state that weight problems should be managed on oneâs own, talking to a provider is not helpful, providers blame them for their weight problem, and that appointments contain sufficient time for weight discussion. CONCLUSION: Providers and patients emphasize different barriers to weight management. Providers need to be aware of the beliefs that their patients hold to improve weight management discussions and interventions in primary care
Recommended from our members
Evaluation of RNAi and CRISPR technologies by large-scale gene expression profiling in the Connectivity Map
The application of RNA interference (RNAi) to mammalian cells has provided the means to perform phenotypic screens to determine the functions of genes. Although RNAi has revolutionized loss-of-function genetic experiments, it has been difficult to systematically assess the prevalence and consequences of off-target effects. The Connectivity Map (CMAP) represents an unprecedented resource to study the gene expression consequences of expressing short hairpin RNAs (shRNAs). Analysis of signatures for over 13,000 shRNAs applied in 9 cell lines revealed that microRNA (miRNA)-like off-target effects of RNAi are far stronger and more pervasive than generally appreciated. We show that mitigating off-target effects is feasible in these datasets via computational methodologies to produce a consensus gene signature (CGS). In addition, we compared RNAi technology to clustered regularly interspaced short palindromic repeat (CRISPR)-based knockout by analysis of 373 single guide RNAs (sgRNAs) in 6 cells lines and show that the on-target efficacies are comparable, but CRISPR technology is far less susceptible to systematic off-target effects. These results will help guide the proper use and analysis of loss-of-function reagents for the determination of gene function
Do Health Beliefs and Behaviors Differ According to Severity of Obesity? A Qualitative Study of Australian Adults
Public responses to obesity have focused on providing standardized messages and supports to all obese individuals, but there is limited understanding of the impact of these messages on obese adults. This descriptive qualitative study using in-depth interviews and a thematic method of analysis, compares the health beliefs and behaviors of 141 Australian adults with mild to moderate (BMI 30â39.9) and severe (BMI â„ 40) obesity. Mildly obese individuals felt little need to change their health behaviors or to lose weight for health reasons. Most believed they could âlose weightâ if they needed to, distanced themselves from the word obesity, and stigmatized those âfatterâ than themselves. Severely obese individuals felt an urgent need to change their health behaviors, but felt powerless to do so. They blamed themselves for their weight, used stereotypical language to describe their health behaviors, and described being âat warâ with their bodies. Further research, particularly about the role of stigma and stereotyping, is needed to fully understand the impact of obesity messaging on the health beliefs, behaviors, and wellbeing of obese and severely obese adults
- âŠ