Repository navigation
FlowMatch schedulers - closing the gap #9607
Description
Activity
I'd have to dig into FlowMatch a bit more in the community implementations, but it looks like there's little difference between
EulerDiscreteandFlowMatchEulerDiscretesoFlowMatchEulerDiscretecould be integrated intoEulerDiscretecontrolled by some configuration option, this would avoid the need for separate duplicate variants and reduce maintenance burden - as there's already some divergence between schedulers, inset_timestepsfor example, where some supportsigmas/timestepsas input and others don't, support forfinal_sigmas_type, etc. Schedulers as a whole could probably do with an overhaul.Reacted by Aryan V S, Vladimir Mandic, Charchit Sharma, Dante and Sayak PaulThis issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
ping to remove stale.
any chance we get this one moving - given the popularity of flux and sd35 its one of the most frequent asks
@sayakpaul @yiyixuxu @asomoza as requested yesterday in a discussion, tagging this as a high-priority ask.
Reacted by YiYi XuThis issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
Any progress on this @yiyixuxu @sayakpaul ? Still feels like a huge gap between Comfy and diffusers, especially with new samplers coming out like the now very popular RES4LYF (e.g. Qwen is basically unusable with Euler, way too smooth outputs)
The design surrounding samplers/schedulers in Diffusers has always been a pain point, spanning all the way back to 2022 where the separation of
modelandschedulercomplicated the inclusion of 2nd order schedulers like DPM (#1308). More recently the rise ofFlow Matchmodels created the separate variants as per this issue, resulting in incompatability of various schedulers with Flux etc., some schedulers have since been "patched" withuse_flow_sigmas/prediction_type="flow_prediction"which created further issues as not allFlow Matchmodels use the same sigmas. Schedulers are also a blocker for async pipeline usage, the current recommendation for which is to provide each thread with its own scheduler instance, however this is not well documented.My recommendation is to continue the work of #10146/#11158 for a modular design with complete separation of
sampling method(euler, etc.), schedule type (Beta,Flow), sigma schedule modifications (Beta,Exponential,Karras, adjustments as in works likeDetail-Demon, etc.), ideally aim for everything to be stateless for example providingsigmasand thestep indexto thesampling methodrather than storing those in the class, resolve any discrepancies such as howsigmasare passed from pipeline i.e. whether the final sigma is included, debloat as much as possible (or refactor as# Copied fromcould be used in more places as the code is the same, just written differently), provide methods for computing onlypred original samplewithout actually making the sampling step as this is used in various works, and probably many other improvements can be made.note: https://github.com/beinsezii/skrample provides a diffusers-compatible wrapper for a scheduler/sampler zoo.
As stated here, lets close the scheduler gap!
Problem statement:
FlowMatchEulerDiscreteSchedulerwhich creates a massive gap!(just for completeness, there is also an older implementation of
FlowMatchHeunDiscreteScheduler, but its not updated to support requiredset_timestepsormuinputs, so it cannot be used with newer DiT models like Flux.1)on the other hand, ComfyUI allows use of any regular scheduler with DiT and does not require separate duplicate -FlowMatch variant of the same scheduler (e.g. EulerDiscrete vs FlowMatchEulerDiscrete) with Flux.1.
reference: matrix of different schedulers used with Flux.1.Dev via ComfyUI:
https://civitai.com/articles/6582/flux1-dev-sampler-scheduler-comparison
cc: @yiyixuxu @sayakpaul @DN6 @asomoza