SourceBlack-Box Detection of Language Model Watermarks
The authors applied their three watermark detection tests (red-green logit-bias test, fixed-sampling diversity test, and cache-augmented distribution-shift test) to the public APIs of GPT-4, Claude 3, and Gemini 1.0 Pro. For every model-test combination, the null hypothesis (no watermark of that family is present) was not rejected at the 95% confidence level. The authors conclude that they cannot confirm the presence of a watermark on any of these three deployments. This is a null result: the absence of a detectable signal does not prove the absence of a watermark, but it does indicate that if a watermark is present, it is not from one of the three families tested or is implemented in a way that evades these specific statistical signatures.