Revisiting k-NN for Pre-trained Language Models

Pre-trained Language Models (PLMs), as parametric-based eager learners, have become the de-facto choice for current paradigms of Natural Language Processing (NLP). In contrast, k-Nearest-Neighbor (k-NN) classifiers, as the lazy learning paradigm, tend to mitigate over-fitting and isolated noise. In this paper, we revisit k-NN classifiers for augmenting the PLMs-based classifiers. From the methodological level, we propose to adopt k-NN with textual representations of PLMs in two steps: (1) Utilize k-NN as prior knowledge to calibrate the training process. (2) Linearly interpolate the probability distribution predicted by k-NN with that of the PLMs' classifier. At the heart of our approach is the implementation of k-NN-calibrated training, which treats predicted results as indicators for easy versus hard examples during the training process. From the perspective of the diversity of application scenarios, we conduct extensive experiments on fine-tuning, prompt-tuning paradigms and zero-shot, few-shot and fully-supervised settings, respectively, across eight diverse end-tasks. We hope our exploration will encourage the community to revisit the power of classical methods for efficient NLP\footnote{Code and datasets are available in https://github.com/zjunlp/Revisit-KNN.Comment: Work in progres

Editing Language Model-based Knowledge Graph Embeddings

Recently decades have witnessed the empirical success of framing Knowledge Graph (KG) embeddings via language models. However, language model-based KG embeddings are usually deployed as static artifacts, which are challenging to modify without re-training after deployment. To address this issue, we propose a new task of editing language model-based KG embeddings in this paper. The proposed task aims to enable data-efficient and fast updates to KG embeddings without damaging the performance of the rest. We build four new datasets: E-FB15k237, A-FB15k237, E-WN18RR, and A-WN18RR, and evaluate several knowledge editing baselines demonstrating the limited ability of previous models to handle the proposed challenging task. We further propose a simple yet strong baseline dubbed KGEditor, which utilizes additional parametric layers of the hyper network to edit/add facts. Comprehensive experimental results demonstrate that KGEditor can perform better when updating specific facts while not affecting the rest with low training resources. Code and datasets will be available in https://github.com/zjunlp/PromptKG/tree/main/deltaKG.Comment: Work in progress and the project website is https://zjunlp.github.io/project/KGE_Editing

Editing Large Language Models: Problems, Methods, and Opportunities

Despite the ability to train capable LLMs, the methodology for maintaining their relevancy and rectifying errors remains elusive. To this end, the past few years have witnessed a surge in techniques for editing LLMs, the objective of which is to efficiently alter the behavior of LLMs within a specific domain without negatively impacting performance across other inputs. This paper embarks on a deep exploration of the problems, methods, and opportunities related to model editing for LLMs. In particular, we provide an exhaustive overview of the task definition and challenges associated with model editing, along with an in-depth empirical analysis of the most progressive methods currently at our disposal. We also build a new benchmark dataset to facilitate a more robust evaluation and pinpoint enduring issues intrinsic to existing techniques. Our objective is to provide valuable insights into the effectiveness and feasibility of each editing technique, thereby assisting the community in making informed decisions on the selection of the most appropriate method for a specific task or context. Code and datasets are available at https://github.com/zjunlp/EasyEdit.Comment: EMNLP 2023. Updated with new experiment

EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models

Large Language Models (LLMs) usually suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to the outdated/noisy data. To this end, many knowledge editing approaches for LLMs have emerged -- aiming to subtly inject/edit updated knowledge or adjust undesired behavior while minimizing the impact on unrelated inputs. Nevertheless, due to significant differences among various knowledge editing methods and the variations in task setups, there is no standard implementation framework available for the community, which hinders practitioners to apply knowledge editing to applications. To address these issues, we propose EasyEdit, an easy-to-use knowledge editing framework for LLMs. It supports various cutting-edge knowledge editing approaches and can be readily apply to many well-known LLMs such as T5, GPT-J, LlaMA, etc. Empirically, we report the knowledge editing results on LlaMA-2 with EasyEdit, demonstrating that knowledge editing surpasses traditional fine-tuning in terms of reliability and generalization. We have released the source code on GitHub at https://github.com/zjunlp/EasyEdit, along with Google Colab tutorials and comprehensive documentation for beginners to get started. Besides, we present an online system for real-time knowledge editing, and a demo video at http://knowlm.zjukg.cn/easyedit.mp4.Comment: The project website is https://github.com/zjunlp/EasyEdi

The AST3-NIR Camera for the Kunlun Infrared Sky Survey

AST3-NIR is a new infrared camera for deployment with the AST3-3 wide-field survey telescope to Dome A on the Antarctic plateau. This project is designed to take advantage of the low Antarctic infrared sky thermal background (particularly within the Kdark near infrared atmospheric window at 2.4 μm) and the long Antarctic nights to provide high sensitivity temporal data from astronomical sources. The data collected from the Kunlun Infrared Sky Survey (KISS) will be used to conduct a range of astronomical science cases including the study of supernovae, exo-planets, variable stars, and the cosmic infrared background

Exoplanets in the Antarctic Sky I. The first data release of AST3-II (CHESPA) and new found variables within the southern CVZ of TESS

Located at Dome A, the highest point of the Antarctic plateau, the Chinese Kunlun station is considered to be one of the best ground-based photometric sites because of its extremely cold, dry, and stable atmosphere. A target can be monitored from there for over 40 days without diurnal interruption during a polar winter. This makes Kunlun station a perfect site to search for short-period transiting exoplanets. Since 2008, an observatory has existed at Kunlun station, and three telescopes are working there. Using these telescopes, the AST3 project has been carried out over the last 6 yr with a search for transiting exoplanets as one of its key programs (CHESPA). In the austral winters of 2016 and 2017, a set of target fields in the southern continuous viewing zone (CVZ) of TESS were monitored by the AST3-II telescope. In this paper, we introduce the CHESPA and present the first data release containing photometry of 26,578 bright stars (m(i) <= 15). The best photometric precision at the optimum magnitude for the survey is around 2 mmag. To demonstrate the data quality, we also present a catalog of 221 variables with a brightness variation greater than 5 mmag from the 2016 data. Among these variables, 179 are newly identified periodic variables not listed in the AAVSO database (https://www.aavso.org/), and 67 are listed in the Candidate Target List. These variables will require careful attention to avoid false-positive signals when searching for transiting exoplanets. Dozens of new transiting exoplanet candidates will be released in a subsequent paper

The AST3-NIR Camera for the Kunlun Infrared Sky Survey

AST3-NIR is a new infrared camera for deployment with the AST3-3 wide-field survey telescope to Dome A on the Antarctic plateau. This project is designed to take advantage of the low Antarctic infrared sky thermal background (particularly within the Kdark near infrared atmospheric window at 2.4 μm) and the long Antarctic nights to provide high sensitivity temporal data from astronomical sources. The data collected from the Kunlun Infrared Sky Survey (KISS) will be used to conduct a range of astronomical science cases including the study of supernovae, exo-planets, variable stars, and the cosmic infrared background

Exoplanets in the Antarctic Sky. II. 116 Transiting Exoplanet Candidates Found by AST3-II (CHESPA) within the Southern CVZ of TESS

We report first results from the CHinese Exoplanet Searching Program from Antarctica (CHESPA)-a wide-field high-resolution photometric survey for transiting exoplanets carried out using telescopes of the AST3 (Antarctic Survey Telescopes times 3) project. There are now three telescopes (AST3-I, AST3-II, and CSTAR-II) operating at Dome A-the highest point on the Antarctic Plateau-in a fully automatic and remote mode to exploit the superb observing conditions of the site, and its long and uninterrupted polar nights. The search for transiting exoplanets is one of the key projects for AST3. During the austral winters of 2016 and 2017 we used the AST3-II telescope to survey a set of target fields near the southern ecliptic pole, falling within the continuous viewing zone of the TESS mission. The first data release of the 2016 data, including images, catalogs, and light curves of 26,578 bright stars (7.5 <= m(i) <= 15), was presented in Zhang et al. The best precision, as measured by the rms of the light curves at the optimum magnitude of the survey (m(i) = 10), is around 2 mmag. We detect 222 objects with plausible transit signals from these data, 116 of which are plausible transiting exoplanet candidates according to their stellar properties as given by the TESS Input Catalog, Gaia DR2, and TESS-HERMES spectroscopy. With the first data release from TESS expected in late 2018, this candidate list will be timely for improving the rejection of potential false-positives

Challenging National Narratives: On the Origins of Sweet Potato in China as Global Commodity During the Early Modern Period

The introduction of American cereal crops is probably one of the most important events in China¿s agricultural history, having a great effect on the agriculture production, national life, the transformation of consumer behaviour and, to some extent, the nationalization of consumption. The sweet potato (Ipomoea Batatas L.), in Chinese g¿nsh¿ ¿¿, is a staple food crop for ancient Chinese society. Today it still plays an important role in Chinese daily life, as well as guaranteeing national food security.GECEM Project, Global Encounters between China and Europe: Trade Networks, Consumption and Cultural Exchanges in Macau and Marseille (1680-1840), ERC (European Research Council)- Starting Grant, programa Horizon 2020, número de ref. 679371, www.gecem.eu.Versión del edito

Invasion Consequences in Communities Maintained by Niche and Intransitive Coexistence Mechanisms

Understanding invasion mechanisms or identifying their potential outcomes has been a longstanding objective of invasion. Many recent empirical and theoretical works tend to frame a perspective of invasion biology within the field of coexistence theory. With increasing emphasis on indirect competitions, more researches hold that niche departure, intransitive loop structure or the integration of these two would be the potential mechanisms to promote native coexistence. But how invasion dynamics rely on these key properties of native competitive network is seldom investigated. Here, we introduce one alien species to a three-species competition system. By setting the structure of coexistence coefficient matrix, we consider three native coexistence mechanisms. After analyzing the equilibrium consequences of alien species invasion under these three mechanisms, we have found that (1) in the native communities supported by strong niche differentiation, alien species can certainly establish their population but would not pose great destruction to native species. (2) Invasion exclusion would happen in the community maintained by intransitive competition loop. However, whether alien species coexist with or exclude resident populations depends on both intraspecific and interspecific competition of invader. (3) The community assembled by the combination of these two mechanisms are most resistant to invasion, and where invasion consequences are more diverse. (4) Finally, the species long-term steady state and short-term respond always keep consistent. We have explicitly situated invasion process within the recent coexistence framework. Our results would broaden the understanding of invasion mechanisms and provide insights into the combination of invasion and coexistence theory