Recently, there has been a growing interest in knowledge editing for Large
Language Models (LLMs). Current approaches and evaluations merely explore the
instance-level editing, while whether LLMs possess the capability to modify
concepts remains unclear. This paper pioneers the investigation of editing
conceptual knowledge for LLMs, by constructing a novel benchmark dataset
ConceptEdit and establishing a suite of new metrics for evaluation. The
experimental results reveal that, although existing editing methods can
efficiently modify concept-level definition to some extent, they also have the
potential to distort the related instantial knowledge in LLMs, leading to poor
performance. We anticipate this can inspire further progress in better
understanding LLMs. Our project homepage is available at
https://zjunlp.github.io/project/ConceptEdit.Comment: Work in progress. Code: https://github.com/zjunlp/EasyEdit Dataset:
https://huggingface.co/datasets/zjunlp/ConceptEdi