Repository navigation
A new pipeline for inpainting(object removal) is needed!! #4836
Description
Is your feature request related to a problem? Please describe.
Thanks for your great work!!
Right now I'm using StableDiffusionInpaintPipeline with fixed prompt(such as "the empty background") to do object removal task, but one thing i've found is that when the inpaint mask doesn't cover the shadow or hairline, the model will randomly generate objects else in the mask rather than generating pure background, which is frustrating. I wonder if the inpaint pipeline is trained for removing object ? If not, when will the inpaint model trained for object removal be released(works like LDM inpainting or LaMa)? As this feature can be really useful in practice!
Describe the solution you'd like
Plz release the inpaint pipeline specialized for object removal!
Describe alternatives you've considered
Give some advice about getting rid of unwanted object generation
unwanted object generation:
Activity
GitHub1712 commentedon Sep 11, 2023
Would be cool if we could train tasks like that ourself. I am missing inpaint training code example still very much.
lawsonxwl commentedon Sep 12, 2023
By modifying txt2img training code, inpainting training code can be easy to get.. But controlling the object generated by the model can be really hard!
GitHub1712 commentedon Sep 14, 2023
How could we do that? How did the inpaint models learned their abilty to continue structures against txt2img models?
lawsonxwl commentedon Sep 18, 2023
By finetuning on txt2img models and using random masking strategy, Stability AI trained and released the stable-diffusion inpainting model.. You can check it out on diffusers
GitHub1712 commentedon Oct 11, 2023
Thank you, I want to train own if possible based on inpainting sdxl, is there any documentation how this was exactly trained and created, training example code?
I am missing that for all inpaint models...
donggoing commentedon Dec 16, 2023
longing for the example code for finetuning, too.
sayakpaul commentedon Nov 30, 2024
@asomoza do we have any examples/resources for this now?
asomoza commentedon Nov 30, 2024
we have now object removal training-free guides and models but the discussion was also about training code examples for inpainting which we don't have yet.