109 research outputs found
A broadband microwave Corbino spectrometer at He temperatures and high magnetic fields
We present the technical details of a broadband microwave spectrometer for
measuring the complex conductance of thin films covering the range from 50 MHz
up to 16 GHz in the temperature range 300 mK to 6 K and at applied magnetic
fields up to 8 Tesla. We measure the complex reflection from a sample
terminating a coaxial transmission line and calibrate the signals with three
standards with known reflection coefficients. Thermal isolation of the heat
load from the inner conductor is accomplished by including a section of NbTi
superconducting cable (transition temperature around 8 9 K) and hermetic
seal glass bead adapters. This enables us to stabilize the base temperature of
the sample stage at 300 mK. However, the inclusion of this superconducting
cable complicates the calibration procedure. We document the effects of the
superconducting cable on our calibration procedure and the effects of applied
magnetic fields and how we control the temperature with great repeatability for
each measurement. We have successfully extracted reliable data in this
frequency, temperature and field range for thin superconducting films and
highly resistive graphene samples
Reduction of Effective Terahertz Focal Spot Size By Means Of Nested Concentric Parabolic Reflectors
An ongoing limitation of terahertz spectroscopy is that the technique is
generally limited to the study of relatively large samples of order 4 mm across
due to the generally large size of the focal beam spot. We present a nested
concentric parabolic reflector design which can reduce the terahertz focal spot
size. This parabolic reflector design takes advantage of the feature that
reflected rays experience a relative time delay which is the same for all
paths. The increase in effective optical path for reflected light is equivalent
to the aperture diameter itself. We have shown that the light throughput of an
aperture of 2 mm can be increased by a factor 15 as compared to a regular
aperture of the same size at low frequencies. This technique can potentially be
used to reduce the focal spot size in terahertz spectroscopy and enable the
study of smaller samples
Multilingual Jailbreak Challenges in Large Language Models
While large language models (LLMs) exhibit remarkable capabilities across a
wide range of tasks, they pose potential safety concerns, such as the
``jailbreak'' problem, wherein malicious instructions can manipulate LLMs to
exhibit undesirable behavior. Although several preventive measures have been
developed to mitigate the potential risks associated with LLMs, they have
primarily focused on English data. In this study, we reveal the presence of
multilingual jailbreak challenges within LLMs and consider two potential risk
scenarios: unintentional and intentional. The unintentional scenario involves
users querying LLMs using non-English prompts and inadvertently bypassing the
safety mechanisms, while the intentional scenario concerns malicious users
combining malicious instructions with multilingual prompts to deliberately
attack LLMs. The experimental results reveal that in the unintentional
scenario, the rate of unsafe content increases as the availability of languages
decreases. Specifically, low-resource languages exhibit three times the
likelihood of encountering harmful content compared to high-resource languages,
with both ChatGPT and GPT-4. In the intentional scenario, multilingual prompts
can exacerbate the negative impact of malicious instructions, with
astonishingly high rates of unsafe output: 80.92\% for ChatGPT and 40.71\% for
GPT-4. To handle such a challenge in the multilingual context, we propose a
novel \textsc{Self-Defense} framework that automatically generates multilingual
training data for safety fine-tuning. Experimental results show that ChatGPT
fine-tuned with such data can achieve a substantial reduction in unsafe content
generation. Data is available at
https://github.com/DAMO-NLP-SG/multilingual-safety-for-LLMs. Warning: This
paper contains examples with potentially harmful content
SOUL: Towards Sentiment and Opinion Understanding of Language
Sentiment analysis is a well-established natural language processing task,
with sentiment polarity classification being one of its most popular and
representative tasks. However, despite the success of pre-trained language
models in this area, they often fall short of capturing the broader
complexities of sentiment analysis. To address this issue, we propose a new
task called Sentiment and Opinion Understanding of Language (SOUL). SOUL aims
to evaluate sentiment understanding through two subtasks: Review Comprehension
(RC) and Justification Generation (JG). RC seeks to validate statements that
focus on subjective information based on a review text, while JG requires
models to provide explanations for their sentiment predictions. To enable
comprehensive evaluation, we annotate a new dataset comprising 15,028
statements from 3,638 reviews. Experimental results indicate that SOUL is a
challenging task for both small and large language models, with a performance
gap of up to 27% when compared to human performance. Furthermore, evaluations
conducted with both human experts and GPT-4 highlight the limitations of the
small language model in generating reasoning-based justifications. These
findings underscore the challenging nature of the SOUL task for existing
models, emphasizing the need for further advancements in sentiment analysis to
address its complexities. The new dataset and code are available at
https://github.com/DAMO-NLP-SG/SOUL.Comment: EMNLP 2023 Main Conference, Short Pape
Sentiment Analysis in the Era of Large Language Models: A Reality Check
Sentiment analysis (SA) has been a long-standing research area in natural
language processing. It can offer rich insights into human sentiments and
opinions and has thus seen considerable interest from both academia and
industry. With the advent of large language models (LLMs) such as ChatGPT,
there is a great potential for their employment on SA problems. However, the
extent to which existing LLMs can be leveraged for different sentiment analysis
tasks remains unclear. This paper aims to provide a comprehensive investigation
into the capabilities of LLMs in performing various sentiment analysis tasks,
from conventional sentiment classification to aspect-based sentiment analysis
and multifaceted analysis of subjective texts. We evaluate performance across
13 tasks on 26 datasets and compare the results against small language models
(SLMs) trained on domain-specific datasets. Our study reveals that while LLMs
demonstrate satisfactory performance in simpler tasks, they lag behind in more
complex tasks requiring deeper understanding or structured sentiment
information. However, LLMs significantly outperform SLMs in few-shot learning
settings, suggesting their potential when annotation resources are limited. We
also highlight the limitations of current evaluation practices in assessing
LLMs' SA abilities and propose a novel benchmark, \textsc{SentiEval}, for a
more comprehensive and realistic evaluation. Data and code during our
investigations are available at
\url{https://github.com/DAMO-NLP-SG/LLM-Sentiment}
- …