AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content

Long document summarization systems are critical for domains with lengthy and jargonladen text, yet they present significant challenges to researchers and developers with limited computing resources. Existing solutions mainly focus on efficient attentions or divide-and-conquer strategies. The former reduces theoretical time complexity, but is still memory-heavy. The latter methods sacrifice global context, leading to uninformative and incoherent summaries. This work aims to leverage the memory-efficient nature of divide-and-conquer methods while preserving global context. Concretely, our framework AWESOME uses two novel mechanisms: (1) External memory mechanisms track previously encoded document segments and their corresponding summaries, to enhance global document understanding and summary coherence. (2) Global salient content is further identified beforehand to augment each document segment to support its summarization. Extensive experiments on diverse genres of text, including government reports, transcripts, scientific papers, and novels, show that AWESOME produces summaries with improved informativeness, faithfulness, and coherence than competitive baselines on longer documents, while having a similar or smaller GPU memory footprint

Non-equilibrium dynamics of Axion-like particles: the quantum master equation

We study the non-equilibrium dynamics of Axion-like particles (ALP) coupled to Standard Model degrees of freedom in thermal equilibrium. The Quantum Master Equation (QME) for the (ALP) reduced density matrix is derived to leading order in the coupling of the (ALP) to the thermal bath, but to \emph{all} orders of the bath couplings to degrees of freedom within or beyond the Standard Model other than the (ALP). The (QME) describes the damped oscillation dynamics of an initial misaligned (ALP) condensate, thermalization with the bath, decoherence and entropy production within a unifying framework. The (ALP) energy density $\mathcal{E}(t)$ features two components: a ``cold'' component from the misaligned condensate and a ``hot'' component from thermalization with the bath, with $\mathcal{E}(t)= \mathcal{E}_{c}\,e^{-\gamma(T)\,t}+\mathcal{E}_{h}(1-e^{-\gamma(T)\,t})$ thus providing a ``mixed dark matter'' scenario. Relaxation of the (ALP) condensate, thermalization, decoherence and entropy production occur on similar time scales. An explicit example with (ALP)-photon coupling, valid post recombination yields a relaxation rate $\gamma(T)$ with a substantial enhancement from thermal emission and absorption. A misaligned condensate is decaying at least since recombination and on the same time scale thermalizing with the cosmic microwave background (CMB). Possible consequences for birefringence of the (CMB) and (ALP) contribution to the effective number of ultrarelativistic species and galaxy formation are discussed.Comment: 28 page

Brownian Axion-like particles

We study the non-equilibrium dynamics of a pseudoscalar axion-like particle (ALP) weakly coupled to degrees of freedom in thermal equilibrium by obtaining its reduced density matrix. Its time evolution is determined by the in-in effective action which we obtain to leading order in the (ALP) coupling but to \emph{all orders} in the couplings of the bath to other fields within or beyond the standard model. The effective equation of motion for the (ALP) is a Langevin equation with noise and friction kernels obeying the fluctuation dissipation relation. A ``misaligned'' initial condition yields damped coherent oscillations, however, the (ALP) population increases towards thermalization with the bath. As a result, the energy density features a mixture of a cold component from misalignment and a hot component from thermalization with proportions that vary in time $(cold)\,e^{-\Gamma t}+(hot)\,(1-e^{-\Gamma t})$, providing a scenario wherein the ``warmth'' of the dark matter evolves in time from colder to hotter. As a specific example we consider the (ALP)-photon coupling $g a \vec{E}\cdot \vec{B}$ to lowest order, valid from recombination onwards. For $T \gg m_a$ the long-wavelength relaxation rate is substantially enhanced $\Gamma_T = \frac{g^2\,m^2_a\,T}{16\pi}$. The ultraviolet divergences of the (ALP) self-energy require higher order derivative terms in the effective action. We find that at high temperature, the finite temperature effective mass of the (ALP) is $m^2_a(T) = m^2_a(0)\Big[ 1-(T/T_c)^4\Big]$, with $T_c \propto \sqrt{m_a(0)/g}$, \emph{suggesting} the possibility of an inverted phase transition, which when combined with higher derivatives may possibly indicate exotic new phases. We discuss possible cosmological consequences on structure formation and the effective number of relativistic species.Comment: 40 pages, 5 fig

Effective field theory of particle mixing

We introduce an effective field theory to study \emph{indirect} mixing of two fields induced by their couplings to a common decay channel in a medium. The extension of the method of Lee, Oehme and Yang, the cornerstone of analysis of CP violation in flavored mesons, to include mixing of particles with different masses provides a guide to and benchmark for the effective field theory. The analysis reveals subtle caveats in the description of mixing in terms of the widely used non-Hermitian effective Hamiltonian, more acute in the non-degenerate case. The effective field theory describes the dynamics of field mixing where the common intermediate states populate a bath in thermal equilibrium, as an \emph{open quantum system}. We obtain the effective action up to second order in the couplings, where indirect mixing is a consequence of off-diagonal self-energy components. We find that if only one of the mixing fields features an initial expectation value, indirect mixing induces an expectation value of the other field. The equal time two point correlation functions exhibit asymptotic approach to a stationary thermal state, and the emergence of long-lived \emph{bath induced} coherence which display quantum beats as a consequence of interference of quasinormal modes in the medium. The amplitudes of the quantum beats are resonantly enhanced in the nearly degenerate case with potential observational consequences

Time-aware Prompting for Text Generation

In this paper, we study the effects of incorporating timestamps, such as document creation dates, into generation systems. Two types of time-aware prompts are investigated: (1) textual prompts that encode document timestamps in natural language sentences; and (2) linear prompts that convert timestamps into continuous vectors. To explore extrapolation to future data points, we further introduce a new data-to-text generation dataset, TempWikiBio, containing more than 4 millions of chronologically ordered revisions of biographical articles from English Wikipedia, each paired with structured personal profiles. Through data-to-text generation on TempWikiBio, text-to-text generation on the content transfer dataset, and summarization on XSum, we show that linear prompts on encoder and textual prompts improve the generation quality on all datasets. Despite having less performance drop when testing on data drawn from a later time, linear prompts focus more on non-temporal information and are less sensitive to the given timestamps, according to human evaluations and sensitivity analyses. Meanwhile, textual prompts establish the association between the given timestamps and the output dates, yielding more factual temporal information in the output.Comment: EMNLP 2022 Findings (short paper