Skip to content

Add cross attention support for low precision attention API - #4142

Merged
howardzhang-cv merged 2 commits into
mainfrom
gh/howardzhang-cv/36/head
Mar 23, 2026
Merged

howardzhang-cv merged 2 commits into
mainfrom
gh/howardzhang-cv/36/head

Conversation

@howardzhang-cv

@howardzhang-cv howardzhang-cv commented Mar 23, 2026 •

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

Summary

  • Added support for cross attention, different Q and KV sequence length in the low precision attention API

Results

Wan2.1-T2V-1.3B-Diffusers with 480x832 frame size and 81 frames

Config Median Time (s) Speedup
bf16 baseline 100.85 1.00x
fp8_attn 67.15 1.50x
bf16 baseline + torch.compile 81.32 1.00x
fp8_attn + compile 46.88 1.73x

[ghstack-poisoned]
[ghstack-poisoned]
howardzhang-cv added a commit that referenced this pull request Mar 23, 2026
@pytorch-bot

pytorch-bot Bot commented Mar 23, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4142

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 84473e5 with merge base 34322b5 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 23, 2026
@howardzhang-cv howardzhang-cv added the module: not user facing Use this tag if you don't want this PR to show up in release notes label Mar 23, 2026

@drisspg drisspg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: small test

@howardzhang-cv
howardzhang-cv changed the base branch from gh/howardzhang-cv/36/base to main March 23, 2026 22:35
@howardzhang-cv
howardzhang-cv merged commit 5c97dbd into main Mar 23, 2026
34 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: not user facing Use this tag if you don't want this PR to show up in release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants