Towards LLM-driven Dialogue State Tracking

Dialogue State Tracking (DST) is of paramount importance in ensuring accurate tracking of user goals and system actions within task-oriented dialogue systems. The emergence of large language models (LLMs) such as GPT3 and ChatGPT has sparked considerable interest in assessing their efficacy across diverse applications. In this study, we conduct an initial examination of ChatGPT's capabilities in DST. Our evaluation uncovers the exceptional performance of ChatGPT in this task, offering valuable insights to researchers regarding its capabilities and providing useful directions for designing and enhancing dialogue systems. Despite its impressive performance, ChatGPT has significant limitations including its closed-source nature, request restrictions, raising data privacy concerns, and lacking local deployment capabilities. To address these concerns, we present LDST, an LLM-driven DST framework based on smaller, open-source foundation models. By utilizing a novel domain-slot instruction tuning method, LDST achieves performance on par with ChatGPT. Comprehensive evaluations across three distinct experimental settings, we find that LDST exhibits remarkable performance improvements in both zero-shot and few-shot setting compared to previous SOTA methods. The source code is provided for reproducibility.Comment: Accepted at EMNLP 202

How Good Are Large Language Models at Out-of-Distribution Detection?

Out-of-distribution (OOD) detection plays a vital role in enhancing the reliability of machine learning (ML) models. The emergence of large language models (LLMs) has catalyzed a paradigm shift within the ML community, showcasing their exceptional capabilities across diverse natural language processing tasks. While existing research has probed OOD detection with smaller encoder-based Transformers like BERT and RoBERTa, the stark differences in scales, pre-training objectives, and inference paradigms call into question the applicability of these findings to LLMs. This paper embarks on a pioneering empirical investigation of OOD detection in the domain of LLMs, focusing on LLaMA series ranging from 7B to 65B in size. We thoroughly evaluate commonly-used OOD detectors, scrutinizing their performance in both zero-grad and fine-tuning scenarios. Notably, we alter previous discriminative in-distribution fine-tuning into generative fine-tuning, aligning the pre-training objective of LLMs with downstream tasks. Our findings unveil that a simple cosine distance OOD detector demonstrates superior efficacy, outperforming other OOD detectors. We provide an intriguing explanation for this phenomenon by highlighting the isotropic nature of the embedding spaces of LLMs, which distinctly contrasts with the anisotropic property observed in smaller BERT family models. The new insight enhances our understanding of how LLMs detect OOD data, thereby enhancing their adaptability and reliability in dynamic environments.Comment: Work in progres

Flattening Singular Values of Factorized Convolution for Medical Images

Convolutional neural networks (CNNs) have long been the paradigm of choice for robust medical image processing (MIP). Therefore, it is crucial to effectively and efficiently deploy CNNs on devices with different computing capabilities to support computer-aided diagnosis. Many methods employ factorized convolutional layers to alleviate the burden of limited computational resources at the expense of expressiveness. To this end, given weak medical image-driven CNN model optimization, a Singular value equalization generalizer-induced Factorized Convolution (SFConv) is proposed to improve the expressive power of factorized convolutions in MIP models. We first decompose the weight matrix of convolutional filters into two low-rank matrices to achieve model reduction. Then minimize the KL divergence between the two low-rank weight matrices and the uniform distribution, thereby reducing the number of singular value directions with significant variance. Extensive experiments on fundus and OCTA datasets demonstrate that our SFConv yields competitive expressiveness over vanilla convolutions while reducing complexity

Observation of the Anomalous Hall Effect in a Collinear Antiferromagnet

Time-reversal symmetry breaking is the basic physics concept underpinning many magnetic topological phenomena such as the anomalous Hall effect (AHE) and its quantized variant. The AHE has been primarily accompanied by a ferromagnetic dipole moment, which hinders the topological quantum states and limits data density in memory devices, or by a delicate noncollinear magnetic order with strong spin decoherence, both limiting their applicability. A potential breakthrough is the recent theoretical prediction of the AHE arising from collinear antiferromagnetism in an anisotropic crystal environment. This new mechanism does not require magnetic dipolar or noncollinear fields. However, it has not been experimentally observed to date. Here we demonstrate this unconventional mechanism by measuring the AHE in an epilayer of a rutile collinear antiferromagnet RuO$_2$. The observed anomalous Hall conductivity is large, exceeding 300 S/cm, and is in agreement with the Berry phase topological transport contribution. Our results open a new unexplored chapter of time-reversal symmetry breaking phenomena in the abundant class of collinear antiferromagnetic materials.Comment: 33 pages, 14 figures, 2 table

Publisher Correction: An anomalous Hall effect in altermagnetic ruthenium dioxide

In the version of this article initially published, square brackets and parentheses were incorrect in Fig. 1g and throughout Fig. 2 (excepting lower labels in Fig. 2d–f). Further, in the second paragraph of the “Consistency with theoretical prediction” subsection of the main article, in the text now reading “the reorientation-field scale, namely, HC = (H2 AE − H2 d) /Hd,” the term “H2 AE” wasn’t shown as squared. The changes have been made in the HTML and PDF versions of the article

A multi-proxy reconstruction of spatial and temporal variations in Asian summer temperatures over the last millennium

To investigate climate variability in Asia during the last millennium, the spatial and temporal evolution of summer (June–July–August; JJA) temperature in eastern and south-central Asia is reconstructed using multi-proxy records and the regularized expectation maximization (RegEM) algorithm with truncated total least squares (TTLS), under a point-by-point regression (PPR) framework. The temperature index reconstructions show that the late 20th century was the warmest period in Asia over the past millennium. The temperature field reconstructions illustrate that temperatures in central, eastern, and southern China during the 11th and 13th centuries, and in western Asia during the 12th century, were significantly higher than those in other regions, and comparable to levels in the 20th century. Except for the most recent warming, all identified warm events showed distinct regional expressions and none were uniform over the entire reconstruction area. The main finding of the study is that spatial temperature patterns have, on centennial time-scales, varied greatly over the last millennium. Moreover, seven climate model simulations, from the Coupled Model Intercomparison Project Phase 5 (CMIP5), over the same region of Asia, are all consistent with the temperature index reconstruction at the 99 % confidence level. Only spatial temperature patterns extracted as the first empirical orthogonal function (EOF) from the GISS-E2-R and MPI-ESM-P model simulations are significant and consistent with the temperature field reconstruction over the past millennium in Asia at the 90 % confidence level. This indicates that both the reconstruction and the simulations depict the temporal climate variability well over the past millennium. However, the spatial simulation or reconstruction capability of climate variability over the past millennium could be still limited. For reconstruction, some grid points do not pass validation tests and reveal the need for more proxies with high temporal resolution, accurate dating, and sensitive temperature signals, especially in central Asia and before AD 1400

FVRD: Fishing Vessel Relations Discovery Through VMS Trace Analysis

The sailing information of fishing vessels are recorded through Vessel Monitoring Systems (VMS), which provides the basis to discover the spatial-temporal pattern of fishing activities. Previous research calculates the fishing density distribution from VMS traces, however, the relationships between fishing vessels are neglected. These kinds of relations, such as fishing vessel groups, may have some impact on fishing density distributions. This paper reveals the potential to construct fishing vessel relations through VMS trace analysis. It takes the VMS traces of all fishing related vessels in the East China Sea in 2016 as the object. The proposed system exploits the parallel Map-Reduce computation model to evaluate the spatial closeness among vessels in order to construct the vessel relation model over different time periods. After constructing the relation model, some key conclusions are revealed from the relation model. It shows that 72% vessels share the cooperation relations over one week, confirming the intuition that most fishing vessels are sailing together for fishing. Moreover, 42% vessels are in the long-term cooperations (over four weeks), representing the core members in each individual group