Repository navigation
The first image generated with LMSScheduler is completely green #454
Description
Describe the bug
The first image generated with LMSScheduler is just a green image.
Reproduction
Run with:
lms = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear"
pipe = StableDiffusionPipeline.from_pretrained(..., scheduler=lms)
instead of using the default scheduler. Then the first image is completely green. The workaround I am using is to run one step before the actual diffuse function:
if isinstance(pipe.scheduler, LMSDiscreteScheduler):
# Prevent an empty first frame
with autocast("cuda"):
image = diffuse(pipe, text_embeddings, init1, 1, settings["guidance_scale"], eta)
Afterward I compute images similarly, but using e.g. 50 steps instead of one and everything is fine. The default scheduler also seems to work. I did not test other schedulers yet.
System Info
Diffusers from git, python 3.10.6, CUDA with a RTX 3060 card
Activity
[-]The first image generated with LMSScheduler is green[/-][+]The first image generated with LMSScheduler is completely green[/+]patrickvonplaten commentedon Sep 13, 2022
cc @anton-l
patrickvonplaten commentedon Sep 13, 2022
Probably related to: #336
github-actions commentedon Oct 10, 2022
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
patrickvonplaten commentedon Oct 11, 2022
@allo- is this issue still relevant?
allo- commentedon Oct 11, 2022
I removed the workaround and cannot reproduce the issue. I guess I close here for now and if my scripts without the workarounds get the problem again I reopen or make a new issue.
anton-l commentedon Oct 12, 2022
Yes, the issue was caused by improper LMS timesteps initialization and should be fixed now :)
[WEB] Minor changes in the shark web (huggingface#454)