1,597 research outputs found
SOUL: Towards Sentiment and Opinion Understanding of Language
Sentiment analysis is a well-established natural language processing task,
with sentiment polarity classification being one of its most popular and
representative tasks. However, despite the success of pre-trained language
models in this area, they often fall short of capturing the broader
complexities of sentiment analysis. To address this issue, we propose a new
task called Sentiment and Opinion Understanding of Language (SOUL). SOUL aims
to evaluate sentiment understanding through two subtasks: Review Comprehension
(RC) and Justification Generation (JG). RC seeks to validate statements that
focus on subjective information based on a review text, while JG requires
models to provide explanations for their sentiment predictions. To enable
comprehensive evaluation, we annotate a new dataset comprising 15,028
statements from 3,638 reviews. Experimental results indicate that SOUL is a
challenging task for both small and large language models, with a performance
gap of up to 27% when compared to human performance. Furthermore, evaluations
conducted with both human experts and GPT-4 highlight the limitations of the
small language model in generating reasoning-based justifications. These
findings underscore the challenging nature of the SOUL task for existing
models, emphasizing the need for further advancements in sentiment analysis to
address its complexities. The new dataset and code are available at
https://github.com/DAMO-NLP-SG/SOUL.Comment: EMNLP 2023 Main Conference, Short Pape
Multilingual Jailbreak Challenges in Large Language Models
While large language models (LLMs) exhibit remarkable capabilities across a
wide range of tasks, they pose potential safety concerns, such as the
``jailbreak'' problem, wherein malicious instructions can manipulate LLMs to
exhibit undesirable behavior. Although several preventive measures have been
developed to mitigate the potential risks associated with LLMs, they have
primarily focused on English data. In this study, we reveal the presence of
multilingual jailbreak challenges within LLMs and consider two potential risk
scenarios: unintentional and intentional. The unintentional scenario involves
users querying LLMs using non-English prompts and inadvertently bypassing the
safety mechanisms, while the intentional scenario concerns malicious users
combining malicious instructions with multilingual prompts to deliberately
attack LLMs. The experimental results reveal that in the unintentional
scenario, the rate of unsafe content increases as the availability of languages
decreases. Specifically, low-resource languages exhibit three times the
likelihood of encountering harmful content compared to high-resource languages,
with both ChatGPT and GPT-4. In the intentional scenario, multilingual prompts
can exacerbate the negative impact of malicious instructions, with
astonishingly high rates of unsafe output: 80.92\% for ChatGPT and 40.71\% for
GPT-4. To handle such a challenge in the multilingual context, we propose a
novel \textsc{Self-Defense} framework that automatically generates multilingual
training data for safety fine-tuning. Experimental results show that ChatGPT
fine-tuned with such data can achieve a substantial reduction in unsafe content
generation. Data is available at
https://github.com/DAMO-NLP-SG/multilingual-safety-for-LLMs. Warning: This
paper contains examples with potentially harmful content
FMRP regulates GABAA receptor channel activity to control signal integration in hippocampal granule cells
Fragile X syndrome, the most common inherited form of intellectual disability, is caused by loss of fragile X mental retardation protein (FMRP). GABAergic system dysfunction is one of the hallmarks of FXS, yet the underlying mechanisms remain poorly understood. Here, we report that FMRP interacts with GAB
Hyperexcitability of sensory neurons in Fragile X mouse model
Sensory hypersensitivity and somatosensory deficits represent the core symptoms of Fragile X syndrome (FXS). These alterations are believed to arise from changes in cortical sensory processing, while potential deficits in the function of peripheral sensory neurons residing in dorsal root ganglia remain unexplored. We found that peripheral sensory neurons exhibit pronounced hyperexcitability i
Sentiment Analysis in the Era of Large Language Models: A Reality Check
Sentiment analysis (SA) has been a long-standing research area in natural
language processing. It can offer rich insights into human sentiments and
opinions and has thus seen considerable interest from both academia and
industry. With the advent of large language models (LLMs) such as ChatGPT,
there is a great potential for their employment on SA problems. However, the
extent to which existing LLMs can be leveraged for different sentiment analysis
tasks remains unclear. This paper aims to provide a comprehensive investigation
into the capabilities of LLMs in performing various sentiment analysis tasks,
from conventional sentiment classification to aspect-based sentiment analysis
and multifaceted analysis of subjective texts. We evaluate performance across
13 tasks on 26 datasets and compare the results against small language models
(SLMs) trained on domain-specific datasets. Our study reveals that while LLMs
demonstrate satisfactory performance in simpler tasks, they lag behind in more
complex tasks requiring deeper understanding or structured sentiment
information. However, LLMs significantly outperform SLMs in few-shot learning
settings, suggesting their potential when annotation resources are limited. We
also highlight the limitations of current evaluation practices in assessing
LLMs' SA abilities and propose a novel benchmark, \textsc{SentiEval}, for a
more comprehensive and realistic evaluation. Data and code during our
investigations are available at
\url{https://github.com/DAMO-NLP-SG/LLM-Sentiment}
Voltage-independent SK-channel dysfunction causes neuronal hyperexcitability in the hippocampus of Fmr1 knock-out mice
Neuronal hyperexcitability is one of the major characteristics of fragile X syndrome (FXS), yet the molecular mechanisms of this critical dysfunction remain poorly understood. Here we report a major role of voltage-independent potassium (
- …