The recent advancements in large language models (LLMs) have sparked a
growing apprehension regarding the potential misuse. One approach to mitigating
this risk is to incorporate watermarking techniques into LLMs, allowing for the
tracking and attribution of model outputs. This study examines a crucial aspect
of watermarking: how significantly watermarks impact the quality of
model-generated outputs. Previous studies have suggested a trade-off between
watermark strength and output quality. However, our research demonstrates that
it is possible to integrate watermarks without affecting the output probability
distribution with appropriate implementation. We refer to this type of
watermark as an unbiased watermark. This has significant implications for the
use of LLMs, as it becomes impossible for users to discern whether a service
provider has incorporated watermarks or not. Furthermore, the presence of
watermarks does not compromise the performance of the model in downstream
tasks, ensuring that the overall utility of the language model is preserved.
Our findings contribute to the ongoing discussion around responsible AI
development, suggesting that unbiased watermarks can serve as an effective
means of tracking and attributing model outputs without sacrificing output
quality