Repository navigation
More Samplers/Schedulers, e.g. Euler #277
Description
Thank you guys for throwing in the documentation. Segregating this from the other Issue.
Is there any possibility to include more of the k-diffusion samplers? https://github.com/crowsonkb/k-diffusion/blob/master/k_diffusion/sampling.py
Heuler for example would be nice
Activity
ElleSimpsonEdin commentedon Aug 30, 2022
Seconded!
[-]More Samplers[/-][+]More Samplers/Schedulers, e.g. Euler[/+]mja commentedon Sep 2, 2022
Looking at the code and the equations referenced,
sample_eulerin k_diffusion is the same or very similar toKarrasVeSchedulerin diffusers. So that would be a good starting point for implementing the other samplers (sample_huen,sample_dpm_2) that take the same parameters (s_churn, s_tmin, s_tmax, s_noise).exo-pla-net commentedon Sep 8, 2022
TLDR: Now that we have optimized VRAM usage, I'd argue that the next feature to expand is samplers. Devs are attached to certain samplers, and their absence here is tempting devs to develop outside of huggingface. Given that a lot of devs are attached to k_euler_a in particular, I think adding it to Hugging Face should be a high priority.
Below is my overview of where we are with samplers, and what's important.
Currently, our diffusers project has these samplers available:
DDIM, LMS, or PNDM.It appears the gameplan is to make separate files for each sampler we implement.
What other samplers exist? Where can you try them out and see how they're implemented? I'd suggest the hlky Colab here.
Their samplers:
"DDIM", "PLMS", 'k_dpm_2_a', 'k_dpm_2', 'k_euler_a', 'k_euler', 'k_heun', 'k_lms'.(Our PNDM ~= their PLMS. Our LMS ~= their k_lms.)
But wait, you might ask, do samplers even matter? Is choosing the correct sampler important for making a great image? I have looked at several comparisons between samplers. Here are three of the best: one , two, and three. My conclusions:
-- Low CFG = washed out, greater intricacy.
-- High CFG == more defined, higher saturation.
So which is the best sampler? I think the following reasoning is a solid foundation for making that choice:
And thus, the best scheduler is the one that resolves high-CFG rainbow noise in 25 inference steps the fastest. For that purpose, these samplers are all equivalent: DDIM, k_euler, k_euler_a. Using any of those, you'll be able to explore seed+CFG space the fastest, and thereby create a great image the fastest.
However, I and other devs have developed a superstition about k_euler_a. There's just something to it. k_euler_a is a contrarian sampler that refuses to resolve, no matter how many steps you throw at it. It seems to often spit out higher quality images than other samplers. (I think k_euler_a is possibly just chaotic, and you'd find an equal number of great images in equal time, simply by exploring different seeds with another sampler, instead of pounding on the same seed with the chaotic k_euler_a. But then again, maybe not -- maybe k_euler_a truly is special. Can anybody math me and say something definitive?)
We want developers to feel like Hugging Face's diffusers is all they need to make state-of-the-art apps. Whether or not it's merely superstition that makes devs want to use other samplers, we should try to include those samplers, because great devs are not immune to superstition, and we want them building with Hugging Face.
(Plus, you never know, k_euler_a might add some extra spice to our incantations /~*~~ )
Inkorak commentedon Sep 9, 2022
Yeah, k_euler_a has some popularity. It is really worth implementing, at least as a selling point.
sidthekidder commentedon Sep 16, 2022
Thanks for the context above @exo-pla-net! I'm planning to try and implement the
sample_euler_ancestralsampler in a new scheduler. I read theschedulers/scheduling_karras_ve.pyimplementation that's similar tosample_eulerso I'll structure my code similarly.A question - since the Algorithm 2 euler implementation is also spread across the pipelines/stochatic_karras_ve/pipeline_stochastic_karras_ve.py file, does that mean sample_euler_ancestral would require a new pipeline as well? Or should I try to modify the existing
pipeline_stochastic_karras_ve.py, since it's similar apart from adding ancestral sampling?Arkitecc commentedon Sep 23, 2022
I'm very much looking forward to this, if anyone is able to get this down definitely tag me.
Suhail commentedon Oct 10, 2022
it would be useful to implement as many of the samplers that DreamStudio does: https://github.com/Stability-AI/stability-sdk
Users are becoming used to using them.
patrickvonplaten commentedon Oct 11, 2022
@patil-suraj @anton-l - plans/ideas on how to move forward here?
crowsonkb commentedon Oct 14, 2022
k-diffusion's Euler and Euler Ancestral samplers are just the VE versions of DDIM and the original DDPM sampling method though, they should actually produce the same outputs as their VP counterparts if you convert a VP diffusion process to VE...
patrickvonplaten commentedon Oct 27, 2022
@anton-l any updates here?
8 remaining items
keturn commentedon Jan 4, 2023
I think
not sure about the
_ancestralvariants, whether that corresponds to one of the many configuration options for DPMSolverMultistepScheduler, or if that's a crowsonkb special addition that wasn't in Cheng Lu's implementation for diffusers.patrickvonplaten commentedon Jan 4, 2023
Actually sorry I think there is some confusion because the names are so similar 😅
Here the conversion k-diffusion<>diffusers:
none-diffusersdoes not have this one as it's an unofficial one that Katherine created I think :-) Happy to add it thoughHope this helps a bit :-)
patrickvonplaten commentedon Jan 4, 2023
Here is a doc explaining how to use them: https://huggingface.co/docs/diffusers/using-diffusers/schedulers
Suhail commentedon Jan 4, 2023
Very helpful - thank you
diffusers: update sampler-to-scheduler mapping
usamaehsan commentedon Jan 14, 2023
Do you guys support DPM adaptive?
I am only getting good results with this scheduler
use 🧨diffusers model (#1583)
patrickvonplaten commentedon Jan 16, 2023
Hey @usamaehsan,
Can you send a link to DPM adaptive? Where is it from?
usamaehsan commentedon Jan 16, 2023
It's from k-diffusion:
https://github.com/crowsonkb/k-diffusion/blob/686dbad0f39640ea25c8a8c6a6e56bb40eacefa2/k_diffusion/sampling.py
I am using it with automatic 1111, stable diffusion web ui
fix nightly upload check (huggingface#277)
pete-finesse commentedon Mar 23, 2023
This scheduler k_dpmpp_2s_a (or k_dpmpp_2 generally) is mentioned in this thread before but I can't figure out how it maps to the diffusers schedulers. Does it exist and I'm just missing something? This is the method for reference
sayakpaul commentedon Nov 21, 2024
I think we now support a comprehensive list of schedulers and also overhauling its inter-pipeline support (like using a non-flow matching scheduler in a flow-matching model like Flux). So, closing this. Cc: @yiyixuxu
See: #9607