Recent years have witnessed a proliferation of valuable original natural
language contents found in subscription-based media outlets, web novel
platforms, and outputs of large language models. However, these contents are
susceptible to illegal piracy and potential misuse without proper security
measures. This calls for a secure watermarking system to guarantee copyright
protection through leakage tracing or ownership identification. To effectively
combat piracy and protect copyrights, a multi-bit watermarking framework should
be able to embed adequate bits of information and extract the watermarks in a
robust manner despite possible corruption. In this work, we explore ways to
advance both payload and robustness by following a well-known proposition from
image watermarking and identify features in natural language that are invariant
to minor corruption. Through a systematic analysis of the possible sources of
errors, we further propose a corruption-resistant infill model. Our full method
improves upon the previous work on robustness by +16.8% point on average on
four datasets, three corruption types, and two corruption ratios. Code
available at https://github.com/bangawayoo/nlp-watermarking.Comment: ACL 2023 lon