[Pipeline] ReNoise: Real Image Inversion Through Iterative Noising #7509

Model/Pipeline/Scheduler description

Achieving faithful image-to-noise inversion with Denoising Diffusion models remains a challenge, particularly for more recent models trained to generate images with a small number of denoising steps. This work introduces an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. Building on reversing the diffusion sampling process, the method applies multiple fixed-point iterations to estimate the next inversion target at each noise level, and then averages the predictions to empirically increase image reconstruction quality. Furthermore, this method preserves editability through optimizing noise regularization losses in a fashion similar to Pix2PixZero.

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

• Arxiv PDF: https://arxiv.org/pdf/2403.14602.pdf
• Github: https://github.com/garibida/ReNoise-Inversion
• First Author: @garibida
• Notes: The code is not available yet, but the method should be easy to implement as a new pipeline similar to the existing StableDiffusionPix2PixZeroPipeline based on the pseudocode and discussions in the paper.