Shallow Diffuse: Robust and Invisible Watermarking through Low-Dimensional Subspaces in Diffusion Models

Wenda Li*, Huijie Zhang*, Qing Qu

*Indicates Equal Contribution
Paper Code arXiv
MY ALT TEXT

Overview of Shallow Diffuse for T2I diffusion models. The server scenario (top left) shows watermark embedding during generation using CFG, while the user scenario (bottom left) demonstrates post-generation watermark embedding with DDIM inversion. In both scenarios, the watermark is applied within a low-dimensional subspace (top right), where most of the watermark resides in the null space of \( \boldsymbol{J}_{\boldsymbol{\theta}, t} \) due to its low dimensionality. The channel average technique is then used to produce the final watermarked image. The adversarial detection (bottom right) highlights the watermark's robustness, enabling the detector to retrieve the watermark even under adversarial attacks.

Abstract

The widespread use of AI-generated content from diffusion models has raised significant concerns regarding misinformation and copyright infringement. Watermarking is a crucial technique for identifying these AI-generated images and preventing their misuse. In this paper, we introduce Shallow Diffuse, a new watermarking technique that embeds robust and invisible watermarks into diffusion model outputs. Unlike existing approaches that integrate watermarking throughout the entire diffusion sampling process, Shallow Diffuse decouples these steps by leveraging the presence of a low-dimensional subspace in the image generation process. This method ensures that a substantial portion of the watermark lies in the null space of this subspace, effectively separating it from the image generation process. Our theoretical and empirical analyses show that this decoupling strategy greatly enhances the consistency of data generation and the detectability of the watermark. Extensive experiments further validate that our Shallow Diffuse outperforms existing watermarking methods in terms of robustness and consistency.

Limitations of Previous Methods

Sampling variance of Tree-Ring Watermarks, RingID and Shallow Diffuse.

Algorithm

Analysis

Experiments

Generation quality and watermark robustness under the server scenario.

Visualization of Generation Consistency in Server Scenarios.

Generation consistency and watermark robustness under the user scenario.

Visualization of Generation Consistency in User Scenarios.

Trade-off between consistency and robustness for Tree-Ring Watermarks, RingID, and Shallow Diffuse.


(a) Consistency

(b) Robustness

Ablation study of the watermark at different timestep \(t\).

BibTeX

@article{li2024shallow,
        title={Shallow Diffuse: Robust and Invisible Watermarking through Low-Dimensional Subspaces in Diffusion Models},
        author={Li, Wenda and Zhang, Huijie and Qu, Qing},
        journal={arXiv preprint arXiv:2410.21088},
        year={2024}}