Add Singlestep DPM-Solver (singlestep high-order schedulers)

[LuChengTHU]

#1308

I've implemented the single-step high-order solvers in an elegant way. It is very similar to the implementations with the multi-step schedulers, and we do not need to modify pipelines.

If this implementation is fine for you, I will further add the tests and the flax version.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[LuChengTHU]

Emmm, I don't know what happened with the tests...

[LuChengTHU]

Here are some of my own suggestions:

1. For few-step guided sampling, I still recommend using multi-step DPM-Solver++.

2. For obtaining a converged sample with a faster convergence rate, I recommend using single-step DPM-Solver++. (Because it has a convergence order guarantee).

Nevertheless, the difference between multi-step DPM-Solver++ and single-step DPM-Solver++ on stable-diffusion seems very small.

Overall, I don't have much experience with which one is better, and I hope the community can have some more findings.

[LuChengTHU]

Here are some examples for comparing multi-step and single-step DPM-Solver++ (2nd order, midpoint):

[patrickvonplaten]

This looks great to me @LuChengTHU - thanks a lot!

Could you maybe add some quick tests to tests/test_scheduler.py ?

[patrickvonplaten]

Implementation is exactly in line with the current design so that's perfect :-)

[LuChengTHU]

No problem. I will add tests and flax version implementations :)

[patrickvonplaten]

Cool! Let me know if you need any help :-)