On the Csorgo-Révész increments of finite dimensional Gaussian random fields

In this paper, we establish some limit theorems on the combined Csorgo-Révész increments with moduli of continuity for finite dimensional Gaussian random fields under mild conditions, via estimating upper bounds of large deviation probabilities on suprema of the finite dimensional Gaussian random fields.Csorgo-Révész increment; Gaussian process; random field; modulus of continuity; quasi-increasing; regularly varying function; large deviation probability.

Language Detoxification with Attribute-Discriminative Latent Space

Transformer-based Language Models (LMs) have achieved impressive results on natural language understanding tasks, but they can also generate toxic text such as insults, threats, and profanity, limiting their real-world applications. To overcome this issue, a few text generation approaches aim to detoxify toxic texts using additional LMs or perturbations. However, previous methods require excessive memory, computations, and time which are serious bottlenecks in their real-world application. To address such limitations, we propose an effective yet efficient method for language detoxification using an attribute-discriminative latent space. Specifically, we project the latent space of an original Transformer LM onto a discriminative latent space that well-separates texts by their attributes using a projection block and an attribute discriminator. This allows the LM to control the text generation to be non-toxic with minimal memory and computation overhead. We validate our model, Attribute-Discriminative Language Model (ADLM) on detoxified language and dialogue generation tasks, on which our method significantly outperforms baselines both in performance and efficiency.Comment: ACL 2023; *Equal contribution. Author ordering determined by coin fli

Context-dependent Instruction Tuning for Dialogue Response Generation

Recent language models have achieved impressive performance in natural language tasks by incorporating instructions with task input during fine-tuning. Since all samples in the same natural language task can be explained with the same task instructions, many instruction datasets only provide a few instructions for the entire task, without considering the input of each example in the task. However, this approach becomes ineffective in complex multi-turn dialogue generation tasks, where the input varies highly with each turn as the dialogue context changes, so that simple task instructions cannot improve the generation performance. To address this limitation, we introduce a context-based instruction fine-tuning framework for each multi-turn dialogue which generates both responses and instructions based on the previous context as input. During the evaluation, the model generates instructions based on the previous context to self-guide the response. The proposed framework produces comparable or even outstanding results compared to the baselines by aligning instructions to the input during fine-tuning with the instructions in quantitative evaluations on dialogue benchmark datasets with reduced computation budget.Comment: Work in Progres

Full Resolution Image Compression with Recurrent Neural Networks

This paper presents a set of full-resolution lossy image compression methods based on neural networks. Each of the architectures we describe can provide variable compression rates during deployment without requiring retraining of the network: each network need only be trained once. All of our architectures consist of a recurrent neural network (RNN)-based encoder and decoder, a binarizer, and a neural network for entropy coding. We compare RNN types (LSTM, associative LSTM) and introduce a new hybrid of GRU and ResNet. We also study "one-shot" versus additive reconstruction architectures and introduce a new scaled-additive framework. We compare to previous work, showing improvements of 4.3%-8.8% AUC (area under the rate-distortion curve), depending on the perceptual metric used. As far as we know, this is the first neural network architecture that is able to outperform JPEG at image compression across most bitrates on the rate-distortion curve on the Kodak dataset images, with and without the aid of entropy coding.Comment: Updated with content for CVPR and removed supplemental material to an external link for size limitation

Temperature-dependent evolutions of excitonic superfluid plasma frequency in a srong excitonic insulator candidate, Ta2_2NiSe5_5

We investigate an interesting anisotropic van der Waals material, Ta$_{2}$NiSe$_{5}$, using optical spectroscopy. Ta$_{2}$NiSe$_{5}$ has been known as one of the few excitonic insulators proposed over 50 years ago. Ta$_{2}$NiSe$_{5}$ has quasi-one dimensional chains along the $a$-axis. We have obtained anisotropic optical properties of a single crystal Ta$_{2}$NiSe$_{5}$ along the $a$- and $c$-axes. The measured $a$- and $c$-axis optical conductivities exhibit large anisotropic electronic and phononic properties. With regard to the $a$-axis optical conductivity, a sharp peak near 3050 cm$^{-1}$ at 9 K, with a well-defined optical gap ($\Delta^{EI} \simeq$ 1800 cm$^{-1}$) and a strong temperature-dependence, is observed. With an increase in temperature, this peak broadens and the optical energy gap closes around $\sim$325 K($T_c^{EI}$). The spectral weight redistribution with respect to the frequency and temperature indicates that the normalized optical energy gap ($\Delta^{EI}(T)/\Delta^{EI}(0)$) is $1-(T/T_c^{EI})^2$. The temperature-dependent superfluid plasma frequency of the excitonic condensation in Ta$_{2}$NiSe$_{5}$ has been determined from measured optical data. Our findings may be useful for future research on excitonic insulators.Comment: 17 pages, 5 figure

Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation

Language models have achieved impressive performances on dialogue generation tasks. However, when generating responses for a conversation that requires factual knowledge, they are far from perfect, due to an absence of mechanisms to retrieve, encode, and reflect the knowledge in the generated responses. Some knowledge-grounded dialogue generation methods tackle this problem by leveraging facts from Knowledge Graphs (KGs); however, they do not guarantee that the model utilizes a relevant piece of knowledge from the KG. To overcome this limitation, we propose SUbgraph Retrieval-augmented GEneration (SURGE), a framework for generating context-relevant and knowledge-grounded dialogues with the KG. Specifically, our SURGE framework first retrieves the relevant subgraph from the KG, and then enforces consistency across facts by perturbing their word embeddings conditioned by the retrieved subgraph. Then, we utilize contrastive learning to ensure that the generated texts have high similarity to the retrieved subgraphs. We validate our SURGE framework on OpendialKG and KOMODIS datasets, showing that it generates high-quality dialogues that faithfully reflect the knowledge from KG.Comment: Preprint. Under revie