Fal Flashpack

Is your feature request related to a problem? Please describe.
Safetensors loading seems it could be faster.

Describe the solution you'd like.
Recently fal has open sourced a new loading method for faster DIT and TE loading.

Describe alternatives you've considered.
A clear and concise description of any alternative solutions or features you've considered.

Additional context.
I just wanted to ask if anyone will be looking into it. Seems it could enhance the offloading mechanism.

Looking into it :)

@sayakpaul

hi @sayakpaul @yiyixuxu

i’m really looking forward to working on this integration!

since adding flashpack support will likely require major changes to files such as
modeling_utils.py, pipeline_utils.py, and some model-specific loader files,
i wanted to clarify a few things before proceeding:

which direction would you prefer to keep the design consistent with existing patterns?

option a: introduce new methods like from_pretrained_flashpack() and save_pretrained_flashpack()
option b: extend existing methods with a flag, e.g. from_pretrained(..., use_flashpack=True)

which model classes should be prioritized initially?

dit models (e.g. sd3, flux, etc.)
unet models
text encoders (e.g. clip, t5)

thanks! happy to start drafting a proposal or initial pr plan once we align on these points.

@devanshi00

Thanks @devanshi00! Option b sounds good!

@sayakpaul

hey @sayakpaul @yiyixuxu @DN6

i've been exploring this issue as well and have figured out the changes required in modeling_utils.py and pipeline_utils.py that align with option B, mimicking the safetensors implementation
this includes proper fallback mechanisms when Flashpack isn't available and dtype consistency checks in the Flashpack loading block before calling assign_from_file()

@devanshi00 would you be open to collaborating on this?

@devanshi00

Hey @devanshi00! I've already drafted the preliminary implementation while exploring this. Would it work if I handle the implementation and you work on the testing? That way we both contribute to important aspects. Let me know if you'd prefer a different split!

@RakshithaKowlikar

Hey @RakshithaKowlikar sure, I hope you have tested in on basic models like flux and there is an improvement in the performance as given in the benchmarking script this. Then I can run it on different models subject to the compute I have. Also please share your pr with me.

@devanshi00

Sure @devanshi00! I've already tested it on UNet from Stable Diffusion v1.5 to verify. I’m making a few small changes to the code and will then raise the PR along with a description of the changes here.

@sayakpaul

hey @sayakpaul @DN6 @yiyixuxu

I have opened a pr for this issue

Added use_flashpack parameter and conditional block
Added docstring for same
Pass use_flashpack to diffusers models when loading pipelines

Few questions:

The dtype in flashpack model and dtype requested by user (torch_dtype) should be same. For now if they don't match then it falls back to safetensors. Should i remove this and add it to the documentation?
All errors now go to the safetensors fallback since flashpack is an optional method. as of now if any errors the user isn't notified, the process isn't terminated and just silently goes to the safetensors block. Should i use logger.warning() for each possible error?
The loading will lead to problems if a model from transformers is loaded. To combat this i have made changes to pipeline_loading_utils.py. Do we plan to integrate flashpack to transformers repo as well?

Loading UNet from Stable Diffusion v1.5 took 2.39s normally and 1.31s(0.05s to load with flashpack and 1.26s to convert to flashpack format)using flashpack.

Looking forward to your feedback!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@DN6

#12999
@DN6 @yiyixuxu @sayakpaul
I’ve added the experimentation results as part of this PR.
One current limitation is that .flashpack files can’t be uploaded directly to the Hub yet, since push_to_hub=True only supports standard PyTorch / safetensors weights at the moment.
Do we have plans to extend Hub support for flashpack files, or to integrate flashpack more broadly into the transformers repository as well?
Looking forward to your feedback!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Closing because #12700 was merged.

Navigation Menu

Repository navigation

Fal Flashpack #12550

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fal Flashpack #12550

Description

Activity

DN6 commented on Oct 28, 2025

devanshi00 commented on Nov 13, 2025

sayakpaul commented on Nov 13, 2025

RakshithaKowlikar commented on Nov 13, 2025

RakshithaKowlikar commented on Nov 21, 2025

devanshi00 commented on Nov 21, 2025

RakshithaKowlikar commented on Nov 22, 2025

RakshithaKowlikar commented on Nov 24, 2025

github-actions commented on Jan 9, 2026

devanshi00 commented on Jan 19, 2026

github-actions commented on Feb 12, 2026

DN6 commented on Apr 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions