GRAM: Fast Fine-tuning of Pre-trained Language Models for Content-based Collaborative Filtering

Content-based collaborative filtering (CCF) provides personalized item recommendations based on both users' interaction history and items' content information. Recently, pre-trained language models (PLM) have been used to extract high-quality item encodings for CCF. However, it is resource-intensive to finetune PLM in an end-to-end (E2E) manner in CCF due to its multi-modal nature: optimization involves redundant content encoding for interactions from users. For this, we propose GRAM (GRadient Accumulation for Multi-modality): (1) Single-step GRAM which aggregates gradients for each item while maintaining theoretical equivalence with E2E, and (2) Multi-step GRAM which further accumulates gradients across multiple training steps, with less than 40\% GPU memory footprint of E2E. We empirically confirm that GRAM achieves a remarkable boost in training efficiency based on five datasets from two task domains of Knowledge Tracing and News Recommendation, where single-step and multi-step GRAM achieve 4x and 45x training speedup on average, respectively.Comment: NAACL 2022 Main Conferenc

Topological acoustic triple point

Acoustic phonon in a crystalline solid is a well-known and ubiquitous example of elementary excitation with a triple degeneracy in the band structure. Because of the Nambu-Goldstone theorem, this triple degeneracy is always present in the phonon band structure. Here, we show that the triple degeneracy of acoustic phonons can be characterized by a topological charge $\mathfrak{q}$ that is a property of three-band systems with $\mathcal{PT}$ symmetry, where $\mathcal{P}$ and $\mathcal{T}$ are the inversion and the time-reversal symmetries, respectively. We therefore call triple points with nontrivial $\mathfrak{q}$ the topological acoustic triple point (TATP). The topological charge $\mathfrak{q}$ can equivalently be characterized by the skyrmion number of the longitudinal mode, or by the Euler number of the transverse modes, and this strongly constrains the nodal structure around the TATP. The TATP can also be symmetry-protected at high-symmetry momenta in the band structure of phonons and spinless electrons by the $O_h$ and the $T_h$ groups. The nontrivial wavefunction texture around the TATP can induce anomalous thermal transport in phononic systems and orbital Hall effect in electronic systems. Our theory demonstrates that the gapless points associated with the Nambu-Goldstone theorem are an avenue for discovering new classes of degeneracy points with distinct topological characteristics.Comment: 7+15 pages, 5+6 figure

A Case of Atrophoderma of Pasini and Pierini Associated with Borrelia burgdorferi Infection Successfully Treated with Oral Doxycycline

Atrophoderma of Pasini and Pierini is a form of dermal atrophy that manifests as either single or multiple, sharply demarcated, hyperpigmented, non-indurated patches. These patches are marked by a slight depression of the skin, with an abrupt edge (i.e., the "cliff-drop" borders), usually located on the backs of adolescents or young adults. The pathophysiology of the disease is unknown, but some authors have suggested a role of Borrelia burgdorferi infection. A 35-year-old woman visited our department because of asymptomatic, hypopigmented, depressed patches on her chest and back lasting for three months. Laboratory evaluations were normal, except for positive serum antibodies to Borrelia burgdorferi. Histologic examination revealed a significantly decreased thickness of the dermis. The patient underwent treatment with oral doxycycline 200 mg/day for six weeks, after which the depth of depression was improved. Herein, we report a case of atrophoderma of Pasini and Pierini, associated with Borrelia burgdorferi infection, successfully treated with oral doxycycline