Skip to content

Support fixed sampling sigmas in FlowMatchEulerDiscreteScheduler - #3

Open
naykun wants to merge 2 commits into
mainfrom
qwen-image-2.1-sample-sigmas-rebased
Open

naykun wants to merge 2 commits into
mainfrom
qwen-image-2.1-sample-sigmas-rebased

Conversation

@naykun

@naykun naykun commented Oct 1, 2026

Copy link
Copy Markdown
Owner

What does this PR do?

Distilled checkpoints can require a fixed sampling grid. Add sample_sigmas to FlowMatchEulerDiscreteScheduler so that grid can be saved in the scheduler config and used without applying shifting or sigma conversions again. Explicit sigmas or timesteps retain their existing behavior and override the config.

Update QwenImage21Pipeline to use the configured grid and its step count when no explicit sigmas are supplied. This also takes precedence over the pipeline's num_inference_steps argument. Add regression tests for the configured schedule and explicit sigma overrides.

Validation

  • Rebased onto upstream main at 578c9b2c6; both original patches are unchanged.
  • Ten additional local scheduler checks passed: conversion bypass, inversion, Euler stepping, state reset, step-count mismatch, explicit overrides, and config round-trip.
  • make style and make fix-copies completed; unrelated formatter changes were discarded.
  • Three pipeline regression cases passed on CPU (configured schedule with default/explicit step counts and explicit sigma override).

Self-review

No blocking correctness issues or unused added code found for the supported fixed-schedule inputs. The scheduler branch is exercised through set_timesteps, and the pipeline delegates denoising to the real scheduler.

Documentation suggestion for review: the pipeline's argument docstring could explicitly state that configured sample_sigmas takes precedence over num_inference_steps. This precedence is covered by the regression tests and described above. The scheduler's terminal-sigma documentation could also mention the existing invert_sigmas=True convention (terminal 1).

naykun and others added 2 commits October 1, 2026 23:23
Add a `sample_sigmas` parameter to FlowMatchEulerDiscreteScheduler that
allows pre-computed sigma values to be stored in the scheduler config.
When set, `set_timesteps` uses these values directly, bypassing dynamic
shifting, shift-terminal stretching, and karras/exponential/beta
conversions. This is essential for distilled models whose sampling grid
is a fixed subset of the teacher's sigma schedule.

The caller can still override by passing explicit `sigmas=` or
`timesteps=` to `set_timesteps`, in which case the config value is
ignored and the normal code path runs.

@yiyixuxu yiyixuxu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, i left a comment
also i want to ask: does this checkpoint works only with this exact grid? e.g. would it still work well if user just pass num_inference_steps = 4?

Custom values for timesteps to be used for each diffusion step. If `None`, the timesteps are computed
automatically.
"""
# Fast path: if `sample_sigmas` is set in config and the caller did not pass explicit

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it's easier to make the sample_sigmas a pipeline config, instead of a scheduler one, because we try to keep our our scheduler to a standard set of config/arguments so that people can swap them.

also, I think we would not need to make any changes to scheduler when we pass sigmas directly if we just also disable dynamic shifting/shift-terminal stretching etc in the scheduler config for distilled checkpoint here (but for the distilled checkpoint instead) https://huggingface.co/Qwen/Qwen-Image-2.1/blob/main/scheduler/scheduler_config.json, e.g. we can just set use_dynamic_shifting=False shift=1.0 etc

@naykun

naykun commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Implemented the pipeline-config approach in #4, following the suggestion here.

sample_sigmas now lives in model_index.json, and the scheduler code is unchanged. The distilled checkpoint config disables shifting/stretching/conversions; the pipeline passes the grid through the existing sigmas argument. The new PR also adds save/load and override tests and documents the checkpoint configuration.

On the real accelerated checkpoint (2048 脳 2048, 8 steps, bf16, one fixed prompt/seed), the new approach exactly matched this PR's implementation in sigmas, timesteps, every denoising latent, and final RGBA pixels (maximum difference 0).

On the four-step question: quality with a different grid has not been evaluated. In the current implementation, num_inference_steps=4 alone still uses the configured eight-point grid; an explicit sigmas list overrides it.

@naykun

naykun commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

thanks, i left a comment also i want to ask: does this checkpoint works only with this exact grid? e.g. would it still work well if user just pass num_inference_steps = 4?

thank you @yiyixuxu , yes currently we find with this specific grid the quality will be the best. but I think the community can explore other settings based on our recommendation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants