Conversation
Add a `sample_sigmas` parameter to FlowMatchEulerDiscreteScheduler that allows pre-computed sigma values to be stored in the scheduler config. When set, `set_timesteps` uses these values directly, bypassing dynamic shifting, shift-terminal stretching, and karras/exponential/beta conversions. This is essential for distilled models whose sampling grid is a fixed subset of the teacher's sigma schedule. The caller can still override by passing explicit `sigmas=` or `timesteps=` to `set_timesteps`, in which case the config value is ignored and the normal code path runs.
yiyixuxu
left a comment
There was a problem hiding this comment.
thanks, i left a comment
also i want to ask: does this checkpoint works only with this exact grid? e.g. would it still work well if user just pass num_inference_steps = 4?
| Custom values for timesteps to be used for each diffusion step. If `None`, the timesteps are computed | ||
| automatically. | ||
| """ | ||
| # Fast path: if `sample_sigmas` is set in config and the caller did not pass explicit |
There was a problem hiding this comment.
i think it's easier to make the sample_sigmas a pipeline config, instead of a scheduler one, because we try to keep our our scheduler to a standard set of config/arguments so that people can swap them.
also, I think we would not need to make any changes to scheduler when we pass sigmas directly if we just also disable dynamic shifting/shift-terminal stretching etc in the scheduler config for distilled checkpoint here (but for the distilled checkpoint instead) https://huggingface.co/Qwen/Qwen-Image-2.1/blob/main/scheduler/scheduler_config.json, e.g. we can just set use_dynamic_shifting=False shift=1.0 etc
|
Implemented the pipeline-config approach in #4, following the suggestion here.
On the real accelerated checkpoint (2048 脳 2048, 8 steps, bf16, one fixed prompt/seed), the new approach exactly matched this PR's implementation in sigmas, timesteps, every denoising latent, and final RGBA pixels (maximum difference 0). On the four-step question: quality with a different grid has not been evaluated. In the current implementation, |
thank you @yiyixuxu , yes currently we find with this specific grid the quality will be the best. but I think the community can explore other settings based on our recommendation. |
What does this PR do?
Distilled checkpoints can require a fixed sampling grid. Add
sample_sigmastoFlowMatchEulerDiscreteSchedulerso that grid can be saved in the scheduler config and used without applying shifting or sigma conversions again. Explicitsigmasortimestepsretain their existing behavior and override the config.Update
QwenImage21Pipelineto use the configured grid and its step count when no explicit sigmas are supplied. This also takes precedence over the pipeline'snum_inference_stepsargument. Add regression tests for the configured schedule and explicit sigma overrides.Validation
mainat578c9b2c6; both original patches are unchanged.make styleandmake fix-copiescompleted; unrelated formatter changes were discarded.Self-review
No blocking correctness issues or unused added code found for the supported fixed-schedule inputs. The scheduler branch is exercised through
set_timesteps, and the pipeline delegates denoising to the real scheduler.Documentation suggestion for review: the pipeline's argument docstring could explicitly state that configured
sample_sigmastakes precedence overnum_inference_steps. This precedence is covered by the regression tests and described above. The scheduler's terminal-sigma documentation could also mention the existinginvert_sigmas=Trueconvention (terminal 1).