19 research outputs found
Estimating Numbers without Regression
Despite recent successes in language models, their ability to represent
numbers is insufficient. Humans conceptualize numbers based on their
magnitudes, effectively projecting them on a number line; whereas subword
tokenization fails to explicitly capture magnitude by splitting numbers into
arbitrary chunks. To alleviate this shortcoming, alternative approaches have
been proposed that modify numbers at various stages of the language modeling
pipeline. These methods change either the (1) notation in which numbers are
written (\eg scientific vs decimal), the (2) vocabulary used to represent
numbers or the entire (3) architecture of the underlying language model, to
directly regress to a desired number.
Previous work suggests that architectural change helps achieve
state-of-the-art on number estimation but we find an insightful ablation:
changing the model's vocabulary instead (\eg introduce a new token for numbers
in range 10-100) is a far better trade-off. In the context of masked number
prediction, a carefully designed tokenization scheme is both the simplest to
implement and sufficient, \ie with similar performance to the state-of-the-art
approach that requires making significant architectural changes. Finally, we
report similar trends on the downstream task of numerical fact estimation (for
Fermi Problems) and discuss reasons behind our findings.Comment: Workshop on Insights from Negative Results in NLP at EACL 202
Anthropomorphization of AI: Opportunities and Risks
Anthropomorphization is the tendency to attribute human-like traits to
non-human entities. It is prevalent in many social contexts -- children
anthropomorphize toys, adults do so with brands, and it is a literary device.
It is also a versatile tool in science, with behavioral psychology and
evolutionary biology meticulously documenting its consequences. With widespread
adoption of AI systems, and the push from stakeholders to make it human-like
through alignment techniques, human voice, and pictorial avatars, the tendency
for users to anthropomorphize it increases significantly. We take a dyadic
approach to understanding this phenomenon with large language models (LLMs) by
studying (1) the objective legal implications, as analyzed through the lens of
the recent blueprint of AI bill of rights and the (2) subtle psychological
aspects customization and anthropomorphization. We find that anthropomorphized
LLMs customized for different user bases violate multiple provisions in the
legislative blueprint. In addition, we point out that anthropomorphization of
LLMs affects the influence they can have on their users, thus having the
potential to fundamentally change the nature of human-AI interaction, with
potential for manipulation and negative influence. With LLMs being
hyper-personalized for vulnerable groups like children and patients among
others, our work is a timely and important contribution. We propose a
conservative strategy for the cautious use of anthropomorphization to improve
trustworthiness of AI systems
Toxicity in ChatGPT: Analyzing Persona-assigned Language Models
Large language models (LLMs) have shown incredible capabilities and
transcended the natural language processing (NLP) community, with adoption
throughout many services like healthcare, therapy, education, and customer
service. Since users include people with critical information needs like
students or patients engaging with chatbots, the safety of these systems is of
prime importance. Therefore, a clear understanding of the capabilities and
limitations of LLMs is necessary. To this end, we systematically evaluate
toxicity in over half a million generations of ChatGPT, a popular
dialogue-based LLM. We find that setting the system parameter of ChatGPT by
assigning it a persona, say that of the boxer Muhammad Ali, significantly
increases the toxicity of generations. Depending on the persona assigned to
ChatGPT, its toxicity can increase up to 6x, with outputs engaging in incorrect
stereotypes, harmful dialogue, and hurtful opinions. This may be potentially
defamatory to the persona and harmful to an unsuspecting user. Furthermore, we
find concerning patterns where specific entities (e.g., certain races) are
targeted more than others (3x more) irrespective of the assigned persona, that
reflect inherent discriminatory biases in the model. We hope that our findings
inspire the broader AI community to rethink the efficacy of current safety
guardrails and develop better techniques that lead to robust, safe, and
trustworthy AI systems
Distraction-free Embeddings for Robust VQA
The generation of effective latent representations and their subsequent
refinement to incorporate precise information is an essential prerequisite for
Vision-Language Understanding (VLU) tasks such as Video Question Answering
(VQA). However, most existing methods for VLU focus on sparsely sampling or
fine-graining the input information (e.g., sampling a sparse set of frames or
text tokens), or adding external knowledge. We present a novel "DRAX:
Distraction Removal and Attended Cross-Alignment" method to rid our cross-modal
representations of distractors in the latent space. We do not exclusively
confine the perception of any input information from various modalities but
instead use an attention-guided distraction removal method to increase focus on
task-relevant information in latent embeddings. DRAX also ensures semantic
alignment of embeddings during cross-modal fusions. We evaluate our approach on
a challenging benchmark (SUTD-TrafficQA dataset), testing the framework's
abilities for feature and event queries, temporal relation understanding,
forecasting, hypothesis, and causal analysis through extensive experiments
Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations
Offline reinforcement learning (RL) methods strike a balance between
exploration and exploitation by conservative value estimation -- penalizing
values of unseen states and actions. Model-free methods penalize values at all
unseen actions, while model-based methods are able to further exploit unseen
states via model rollouts. However, such methods are handicapped in their
ability to find unseen states far away from the available offline data due to
two factors -- (a) very short rollout horizons in models due to cascading model
errors, and (b) model rollouts originating solely from states observed in
offline data. We relax the second assumption and present a novel unseen state
augmentation strategy to allow exploitation of unseen states where the learned
model and value estimates generalize. Our strategy finds unseen states by
value-informed perturbations of seen states followed by filtering out states
with epistemic uncertainty estimates too high (high error) or too low (too
similar to seen data). We observe improved performance in several offline RL
tasks and find that our augmentation strategy consistently leads to overall
lower average dataset Q-value estimates i.e. more conservative Q-value
estimates than a baseline
ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance
Current Virtual Mental Health Assistants (VMHAs) provide counseling and suggestive care. They refrain from patient diagnostic assistance because of a lack of training on safety-constrained and specialized clinical process knowledge (Pro-Know). In this work, we define ProKnow as an ordered set of information that maps to evidence-based guidelines or categories of conceptual understanding to experts in a domain. We also introduce a new dataset of diagnostic conversations guided by safety constraints and ProKnow that healthcare professionals use (ProKnow-data). We develop a method for natural language question generation (NLG) that collects diagnostic information from the patient interactively (ProKnow-algo). We demonstrate the limitations of using state-of-the-art large-scale language models (LMs) on this dataset. ProKnow-algo models the process knowledge through explicitly modeling safety, knowledge capture, and explainability. LMs with ProKnow-algo generated 89% safer questions in the depression and anxiety domain. Further, without ProKnow-algo generations question did not adhere to clinical process knowledge in ProKnow-data. In comparison, ProKnow-algo-based generations yield a 96% reduction in averaged squared rank error. The Explainability of the generated question is assessed by computing similarity with concepts in depression and anxiety knowledge bases. Overall, irrespective of the type of LMs, ProKnow-algo achieved an averaged 82% improvement over simple pre-trained LMs on safety, explainability, and process-guided question generation. We qualitatively and quantitatively evaluate the efficacy of ProKnow-algo by introducing three new evaluation metrics for safety, explainability, and process knowledge-adherence. For reproducibility, we will make ProKnow-data and the code repository of ProKnow-algo publicly available upon acceptance
Personalized Content Recommendations Across Devices Using Discover Tab
A television device includes a media application that displays a user interface that enables a user to select and launch media content across a variety of streaming platforms. The user interface includes a discover tab that provides personalized recommendations across a plurality of streaming platforms. For example, a media platform may obtain signals relating to the userās entitlements (e.g., which service/application that the user has access to) and/or provider affinities (e.g., which service/application tends to user), and generate, using the signals, recommendations that are personalized to the user, where the recommendations include programs from multiple different streaming services. The recommendations are provided in the discover tab
CSTS: Conditional Semantic Textual Similarity
Semantic textual similarity (STS) has been a cornerstone task in NLP that
measures the degree of similarity between a pair of sentences, with
applications in information retrieval, question answering, and embedding
methods. However, it is an inherently ambiguous task, with the sentence
similarity depending on the specific aspect of interest. We resolve this
ambiguity by proposing a novel task called conditional STS (C-STS) which
measures similarity conditioned on an aspect elucidated in natural language
(hereon, condition). As an example, the similarity between the sentences "The
NBA player shoots a three-pointer." and "A man throws a tennis ball into the
air to serve." is higher for the condition "The motion of the ball." (both
upward) and lower for "The size of the ball." (one large and one small).
C-STS's advantages are two-fold: (1) it reduces the subjectivity and ambiguity
of STS, and (2) enables fine-grained similarity evaluation using diverse
conditions. C-STS contains almost 20,000 instances from diverse domains and we
evaluate several state-of-the-art models to demonstrate that even the most
performant fine-tuning and in-context learning models (GPT-4, Flan, SimCSE)
find it challenging, with Spearman correlation scores of <50. We encourage the
community to evaluate their models on C-STS to provide a more holistic view of
semantic similarity and natural language understanding