Reproducibility improvement of SD generation with xFormers

[takuma104]

I wrote a patch related to #1339. This patch makes xFormers use Flash Attention as a priority. More details to see: Generating (almost) reproducible pictures using Diffusers with xFormers

main...takuma104:diffusers:force_xformers_flash_attention

I would like to get some feedback.

• Is it worth opening a PR?
• I have put an argument to enable_xformers_memory_efficient_attention() to specify if this patch is enabled or not, is there a better way to specify it?
• Any other advice?

[patrickvonplaten]

I think I would be fine with adding a "use_flash_attention=True/False" argument to enable_xformers_memory_efficient_attention() - @patil-suraj @pcuenca what do you think?

[patil-suraj]

I am fine with this, but I think instead of a boolean like use_flash_attention we could let the user directly pass the xformers op like xformers.ops.MemoryEfficientAttentionFlashAttentionOp or xformers.ops.TritonFlashAttentionOp, this way the user can choose whatever op they want. This will also keep the code simple. I have wip implem here https://github.com/huggingface/diffusers/compare/xformers-attenion-op-arg, feel free to use it and open a PR.

[takuma104]

Thank you for the advices and code! I’ll merge it and open a PR this weekend.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.