Repository navigation
Adding support for ControlNet pipeline for img2img flows #2562
Description
It would be nice to achieve a somewhat feature parity with automatic1111 in terms using controlnet with img2img flows.
Current ControlNet pipeline doesn't allow an input source image (if I understand correctly the image parameter goes to controlnet as condition and not as src image for diffusion).
I think I might try and implement this myself, but wondering how this could be achieved best?
- To add an optional src_image parameter to the ControlNet pipeline?
- To create a separate file with ControlNetImg2Img?
- Something else?
@takuma104 - since you implemented the StableDiffusionControlNetPipeline what are your thoughts?
Activity
takuma104 commentedon Mar 6, 2023
Hi @mikegarts , How about #2561 that was opened a few hours ago?
realimposter commentedon Mar 6, 2023
@takuma104 looks like that pipeline is for inpainting + img2img, any idea how to modify it just for img2img?
mikegarts commentedon Mar 6, 2023
Indeed #2561 is for inpainting models :)
But I think that I might be able to write something similar for img2img and submit it as a community pipeline.
pd-brian-mccrindle commentedon Mar 6, 2023
@mikegarts
I'm also interested in getting ControlNet working with just Img2Img. If you have a branch implementing this, I can try and support where I can
mikegarts commentedon Mar 7, 2023
@pd-brian-mccrindle Thanks lot, I will run some tests and open a pr later today.
mikegarts commentedon Mar 7, 2023
@pedrogengo here it is: #2584