A broadband microwave Corbino spectrometer at 3^3He temperatures and high magnetic fields

We present the technical details of a broadband microwave spectrometer for measuring the complex conductance of thin films covering the range from 50 MHz up to 16 GHz in the temperature range 300 mK to 6 K and at applied magnetic fields up to 8 Tesla. We measure the complex reflection from a sample terminating a coaxial transmission line and calibrate the signals with three standards with known reflection coefficients. Thermal isolation of the heat load from the inner conductor is accomplished by including a section of NbTi superconducting cable (transition temperature around 8 $-$ 9 K) and hermetic seal glass bead adapters. This enables us to stabilize the base temperature of the sample stage at 300 mK. However, the inclusion of this superconducting cable complicates the calibration procedure. We document the effects of the superconducting cable on our calibration procedure and the effects of applied magnetic fields and how we control the temperature with great repeatability for each measurement. We have successfully extracted reliable data in this frequency, temperature and field range for thin superconducting films and highly resistive graphene samples

Reduction of Effective Terahertz Focal Spot Size By Means Of Nested Concentric Parabolic Reflectors

An ongoing limitation of terahertz spectroscopy is that the technique is generally limited to the study of relatively large samples of order 4 mm across due to the generally large size of the focal beam spot. We present a nested concentric parabolic reflector design which can reduce the terahertz focal spot size. This parabolic reflector design takes advantage of the feature that reflected rays experience a relative time delay which is the same for all paths. The increase in effective optical path for reflected light is equivalent to the aperture diameter itself. We have shown that the light throughput of an aperture of 2 mm can be increased by a factor 15 as compared to a regular aperture of the same size at low frequencies. This technique can potentially be used to reduce the focal spot size in terahertz spectroscopy and enable the study of smaller samples

Multilingual Jailbreak Challenges in Large Language Models

While large language models (LLMs) exhibit remarkable capabilities across a wide range of tasks, they pose potential safety concerns, such as the ``jailbreak'' problem, wherein malicious instructions can manipulate LLMs to exhibit undesirable behavior. Although several preventive measures have been developed to mitigate the potential risks associated with LLMs, they have primarily focused on English data. In this study, we reveal the presence of multilingual jailbreak challenges within LLMs and consider two potential risk scenarios: unintentional and intentional. The unintentional scenario involves users querying LLMs using non-English prompts and inadvertently bypassing the safety mechanisms, while the intentional scenario concerns malicious users combining malicious instructions with multilingual prompts to deliberately attack LLMs. The experimental results reveal that in the unintentional scenario, the rate of unsafe content increases as the availability of languages decreases. Specifically, low-resource languages exhibit three times the likelihood of encountering harmful content compared to high-resource languages, with both ChatGPT and GPT-4. In the intentional scenario, multilingual prompts can exacerbate the negative impact of malicious instructions, with astonishingly high rates of unsafe output: 80.92\% for ChatGPT and 40.71\% for GPT-4. To handle such a challenge in the multilingual context, we propose a novel \textsc{Self-Defense} framework that automatically generates multilingual training data for safety fine-tuning. Experimental results show that ChatGPT fine-tuned with such data can achieve a substantial reduction in unsafe content generation. Data is available at https://github.com/DAMO-NLP-SG/multilingual-safety-for-LLMs. Warning: This paper contains examples with potentially harmful content

SOUL: Towards Sentiment and Opinion Understanding of Language

Sentiment analysis is a well-established natural language processing task, with sentiment polarity classification being one of its most popular and representative tasks. However, despite the success of pre-trained language models in this area, they often fall short of capturing the broader complexities of sentiment analysis. To address this issue, we propose a new task called Sentiment and Opinion Understanding of Language (SOUL). SOUL aims to evaluate sentiment understanding through two subtasks: Review Comprehension (RC) and Justification Generation (JG). RC seeks to validate statements that focus on subjective information based on a review text, while JG requires models to provide explanations for their sentiment predictions. To enable comprehensive evaluation, we annotate a new dataset comprising 15,028 statements from 3,638 reviews. Experimental results indicate that SOUL is a challenging task for both small and large language models, with a performance gap of up to 27% when compared to human performance. Furthermore, evaluations conducted with both human experts and GPT-4 highlight the limitations of the small language model in generating reasoning-based justifications. These findings underscore the challenging nature of the SOUL task for existing models, emphasizing the need for further advancements in sentiment analysis to address its complexities. The new dataset and code are available at https://github.com/DAMO-NLP-SG/SOUL.Comment: EMNLP 2023 Main Conference, Short Pape

Sentiment Analysis in the Era of Large Language Models: A Reality Check

Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring deeper understanding or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their potential when annotation resources are limited. We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, \textsc{SentiEval}, for a more comprehensive and realistic evaluation. Data and code during our investigations are available at \url{https://github.com/DAMO-NLP-SG/LLM-Sentiment}