17 research outputs found
Multi-level Memory for Task Oriented Dialogs
Recent end-to-end task oriented dialog systems use memory architectures to
incorporate external knowledge in their dialogs. Current work makes simplifying
assumptions about the structure of the knowledge base, such as the use of
triples to represent knowledge, and combines dialog utterances (context) as
well as knowledge base (KB) results as part of the same memory. This causes an
explosion in the memory size, and makes the reasoning over memory harder. In
addition, such a memory design forces hierarchical properties of the data to be
fit into a triple structure of memory. This requires the memory reader to infer
relationships across otherwise connected attributes. In this paper we relax the
strong assumptions made by existing architectures and separate memories used
for modeling dialog context and KB results. Instead of using triples to store
KB results, we introduce a novel multi-level memory architecture consisting of
cells for each query and their corresponding results. The multi-level memory
first addresses queries, followed by results and finally each key-value pair
within a result. We conduct detailed experiments on three publicly available
task oriented dialog data sets and we find that our method conclusively
outperforms current state-of-the-art models. We report a 15-25% increase in
both entity F1 and BLEU scores.Comment: Accepted as full paper at NAACL 201
Behavioral Use Licensing for Responsible AI
Scientific research and development relies on the sharing of ideas and
artifacts. With the growing reliance on artificial intelligence (AI) for many
different applications, the sharing of code, data, and models is important to
ensure the ability to replicate methods and the democratization of scientific
knowledge. Many high-profile journals and conferences expect code to be
submitted and released with papers. Furthermore, developers often want to
release code and models to encourage development of technology that leverages
their frameworks and services. However, AI algorithms are becoming increasingly
powerful and generalized. Ultimately, the context in which an algorithm is
applied can be far removed from that which the developers had intended. A
number of organizations have expressed concerns about inappropriate or
irresponsible use of AI and have proposed AI ethical guidelines and responsible
AI initiatives. While such guidelines are useful and help shape policy, they
are not easily enforceable. Governments have taken note of the risks associated
with certain types of AI applications and have passed legislation. While these
are enforceable, they require prolonged scientific and political deliberation.
In this paper we advocate the use of licensing to enable legally enforceable
behavioral use conditions on software and data. We argue that licenses serve as
a useful tool for enforcement in situations where it is difficult or
time-consuming to legislate AI usage. Furthermore, by using such licenses, AI
developers provide a signal to the AI community, as well as governmental
bodies, that they are taking responsibility for their technologies and are
encouraging responsible use by downstream users
Prompting with Pseudo-Code Instructions
Prompting with natural language instructions has recently emerged as a
popular method of harnessing the capabilities of large language models. Given
the inherent ambiguity present in natural language, it is intuitive to consider
the possible advantages of prompting with less ambiguous prompt styles, such as
the use of pseudo-code.
In this paper we explore if prompting via pseudo-code instructions helps
improve the performance of pre-trained language models. We manually create a
dataset of pseudo-code prompts for 132 different tasks spanning classification,
QA and generative language tasks, sourced from the Super-NaturalInstructions
dataset. Using these prompts along with their counterparts in natural language,
we study their performance on two LLM families - BLOOM and CodeGen. Our
experiments show that using pseudo-code instructions leads to better results,
with an average increase (absolute) of 7-16 points in F1 scores for
classification tasks and an improvement (relative) of 12-38% in aggregate
ROUGE-L scores across all tasks. We include detailed ablation studies which
indicate that code comments, docstrings, and the structural clues encoded in
pseudo-code all contribute towards the improvement in performance.
To the best of our knowledge our work is the first to demonstrate how
pseudo-code prompts can be helpful in improving the performance of pre-trained
LMs.Comment: Published in EMNLP 2023 main trac
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks
based on a few demonstrations or natural language instructions. While these
capabilities have led to widespread adoption, most LLMs are developed by
resource-rich organizations and are frequently kept from the public. As a step
towards democratizing this powerful technology, we present BLOOM, a
176B-parameter open-access language model designed and built thanks to a
collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer
language model that was trained on the ROOTS corpus, a dataset comprising
hundreds of sources in 46 natural and 13 programming languages (59 in total).
We find that BLOOM achieves competitive performance on a wide variety of
benchmarks, with stronger results after undergoing multitask prompted
finetuning. To facilitate future research and applications using LLMs, we
publicly release our models and code under the Responsible AI License