Repository navigation
Partial torch compile only, consider updating pytorch BUG! #311
Description
Activity
Yesterday did some tests and worked like a charm, but after iteration #4, I just get random color noise as video result.
Updates Comfy, updated dependencies, plugins, restarted GPU (Win+CTRL+B), restarted PC and still nothing. Didn't change any setting at all from seedI'm also seeing this on Linux. I use comfy-cli to manage my install and update everything. How badly does this impact performance?
Hmm, that's strange, any pytorch version > 2.8 should fix this.
For the record, I think fixing it speeds up Compile with GGUF Qwen Image significantly, like 1.5-2x depending on the output image size (smaller image gives bigger speedup).
@leriilx could you try this? thu-ml/SageAttention#162 (comment)
And what are you guys' pytorch version, from
pip show torch?@StrongerXi I'm showing Torch 2.7.1
/mnt/big/AI/applications/ComfyUI master ❯ uv pip show torch Name: torch Version: 2.7.1 Location: /mnt/big/AI/applications/ComfyUI/.venv/lib/python3.12/site-packages Requires: filelock, fsspec, jinja2, networkx, nvidia-cublas-cu12, nvidia-cuda-cupti-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12, nvidia-cufft-cu12, nvidia-cufile-cu12, nvidia-curand-cu12, nvidia-cusolver-cu12, nvidia-cusparse-cu12, nvidia-cusparselt-cu12, nvidia-nccl-cu12, nvidia-nvjitlink-cu12, nvidia-nvtx-cu12, setuptools, sympy, triton, typing-extensions Required-by: accelerate, groundingdino-py, kornia, peft, spandrel, timm, torchaudio, torchsde, torchvision, transparent-backgroundIs there a recommended way to upgrade it without breaking comfy-cli (honestly surprised that
comfy updatedoesn't upgrade it... if it's supposed to be upgraded)? I have other apps I use that use Pytorch 2.9 I believe, so I shouldn't have any issues running a higher version. (I'm running an RTX 4090.)Reacted by Hazem Mahmoud and dwojewodI am wondering about this too !, I have Torch2.7.1 cu128, & many other stuff in my Python libraries are also compiled for Torch271cu128 , will "upgrading to Torch 2.8 or 2.9" break other python compatiblities ?? is it safer for me to rollback/downgrade ComfyUi-gguf ?? if so, which version is better? Help !
I just realized that I've stepped into the club :)
pytorch version: 2.7.1+cu128Do you have any working solution guys?
I just realized that I've stepped into the club :)
pytorch version: 2.7.1+cu128Do you have any working solution guys?
No luck yet, I tried rolling back both gguf & comfyui gguf but still didn't solve this .... honestly I am not ready to make a move to Torch 2.8 or 2.9 "I am a humble digital artist with very humble experience about Python's bonanza!" since Qwen & WAN2.2 came out & I am living i na Python nightmare & each day 2 day Libs/ backward compatibility issues.
until the moment, for "us who have this warning & we own Torch2.7.1cu128 or older" the impact on performance is not that big, so , I guess we can live with this & still continue using all gguf nodes features, until a fix comes out from the community or the devs here.
I just realized that I've stepped into the club :)
pytorch version: 2.7.1+cu128
Do you have any working solution guys?No luck yet, I tried rolling back both gguf & comfyui gguf but still didn't solve this .... honestly I am not ready to make a move to Torch 2.8 or 2.9 "I am a humble digital artist with very humble experience about Python's bonanza!" since Qwen & WAN2.2 came out & I am living i na Python nightmare & each day 2 day Libs/ backward compatibility issues.
until the moment, for "us who have this warning & we own Torch2.7.1cu128 or older" the impact on performance is not that big, so , I guess we can live with this & still continue using all gguf nodes features, until a fix comes out from the community or the devs here.
Compatibility issues is not fun anymore. I have 2 different python virtual environments (3.12 and 3.13); but I won't change my pytorch version right now, everything works well; I don't want to poke the beehive :)
Compatibility issues is not fun anymore. I have 2 different python virtual environments (3.12 and 3.13); but I won't change my pytorch version right now, everything works well; I don't want to poke the beehive :)
LOL I totally get it, me too I kind of spent a hell of a time with losing my nerves, trying to reach the best compatible things altogether for Comfy's python Libs. I finally reached this moment lately for PyTorch 2.7.1cu128 Python 3.12.10 with a success of 99% (just few things that I can't get them to install or to work)... I AM NOT Ready for another ( Here we goooo again ) when I need to have all these features upgraded with newer PyTorch with all the other (Attention)s out there that really screwed my poor butts trying to install all of them together LOL.
Compatibility issues is not fun anymore. I have 2 different python virtual environments (3.12 and 3.13); but I won't change my pytorch version right now, everything works well; I don't want to poke the beehive :)
LOL I totally get it, me too I kind of spent a hell of a time with losing my nerves, trying to reach the best compatible things altogether for Comfy's python Libs. I finally reached this moment lately for PyTorch 2.7.1cu128 Python 3.12.10 with a success of 99% (just few things that I can't get them to install or to work)... I AM NOT Ready for another ( Here we goooo again ) when I need to have all these features upgraded with newer PyTorch with all the other (Attention)s out there that really screwed my poor butts trying to install all of them together LOL.
The most stable version for me, where all the custom nodes work, is the same one you're using. PyTorch 2.7.1cu128 and Python 3.12.
Since we're now experienced in these matters, I guess we won't easily do anything that would break venv :)
Reacted by Hazem MahmoudAnother user, excact same thing. Pytorch 2.7.1 . Macos.
C'mon people it's so easy...
there is a post, that it's expected to have min. torch2.8
but: the Nightly versions of 2.8, when this was programmed, were that time older than the meanwhile most actual 2.7.1i haven't had any issue, leaving as is, at first glance...
but: just edit ops.py and change the version to '2.7'
for me its workingif not chained_hasattr(torch, "compiler.disable"): logging.info("ComfyUI-GGUF: Torch too old for torch.compile - bypassing") return dummy_decorator # torch too old elif version.parse(torch.__version__) >= version.parse("2.7"): logging.info("ComfyUI-GGUF: Allowing full torch compile") return dummy_decorator # torch compile works if chained_hasattr(torch, "_dynamo.config.nontraceable_tensor_subclasses"): logging.info("ComfyUI-GGUF: Allowing full torch compile (nightly)") return dummy_decorator # torch compile works, nightly before 2.8 release else: logging.info("ComfyUI-GGUF: Partial torch compile only, consider updating pytorch") return torch.compiler.disableor:
- make a backup....
and just simply upgrade
python.exe -s -m pip install --upgrade torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu128
- Done
ho harm, no issues (for me)
and i did it meanwhile several timesDisclaimer:
for any accelerators, you will need to upgrade them
pip install -U xformers
pip install -U "triton-windows<=3.4"
for sageattention: go for a 2.2.0 release and use the appropriate URL from https://github.com/woct0rdho/SageAttention/releases/tag/v2.2.0-windows , as like for py3.12
pip install https://github.com/woct0rdho/SageAttention/releases/download/v2.2.0-windows/sageattention-2.2.0+cu128torch2.8.0-cp312-cp312-win_amd64.whlfor flash-attention, i recommend 2.8.2, from https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10
pip install https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.4.10/flash_attn-2.8.2+cu128torch2.8-cp312-cp312-win_amd64.whlregards, Hans
- make a backup....
@hwprinz Thanks for the detailed workaround,, when I am ready as am busy nowadays, I will try the upgrade process in the steps you mentioned, (important to have backups , just in case, but this idea on its own means that we may not be sure that all will work fine ! anyways, I will try out the upgrade with the steps you mentioned & my choice would be based on Torch2.8 cu128 (i still feel things may not be all OK yet for Cuda12.9), OR simply as you said I will just Edit the ops.py file, Thanks again :)
Edit:
Your instructions worked fine so far, I followed exactly everystep & with exact commands & I am now having Torch 2.8 Cuda12.8 & gguf says (ComfyUI-GGUF: Allowing full torch compile) ✅Just one change: I am still using Triton 3.2 & Triton-windows 3.2 , because I tried both 3.3 post 19 & 3.4 post 20 , both gave me issues with ComfyUi Whisper for Transcribe Audio effects (( Cannot set attribute 'src' directly. Use '_unsafe_update_src()' and manually clear ` .... )) ...etc
I could fix it by reverting back to Triton 3.2 with Sage Attn 2.2 . Otherthan that all seems to work fine so far until I discover something.thanks
btw, triton-windows3.4 would have been 'only' for cuda129....
as far as the compatibility matriox goes..
but good to know to be able to go back to 3.20
regards..I went from torch2.8 down to 2.7.1 because of the error:
" Skipping import of cpp extensions due to incompatible torch version 2.x.x+cu128 for torchao version 0.14.1+cu128 "
which I only was able to resolve with torchao 0.12 with torch 2.7.1...
but indeed I get the alert: "ComfyUI-GGUF: Partial torch compile only, consider updating pytorch"
I run 2x RTX3090ti with cu128, so I can't have FP8 support by hardware, so I suppose I better go back to torch 2.8 to get full GGUF support?
(I haven't tried GGUF yet, but may try it for the larger models like HunyuanVideo-gguf)BTW sageattention 2.2 doesn't work for me: core dump crash... v1.0.6 works fine (but I don't see much difference in speed vs. xformers attention)
I went from torch2.8 down to 2.7.1 because of the error:
" Skipping import of cpp extensions due to incompatible torch version 2.x.x+cu128 for torchao version 0.14.1+cu128 "
which I only was able to resolve with torchao 0.12 with torch 2.7.1...
but indeed I get the alert: "ComfyUI-GGUF: Partial torch compile only, consider updating pytorch"
I run 2x RTX3090ti with cu128, so I can't have FP8 support by hardware, so I suppose I better go back to torch 2.8 to get full GGUF support? (I haven't tried GGUF yet, but may try it for the larger models like HunyuanVideo-gguf)
BTW sageattention 2.2 doesn't work for me: core dump crash... v1.0.6 works fine (but I don't see much difference in speed vs. xformers attention)
just read this:
#311 (comment)the full compile is also supported, but for design reason it was said to be 2.8
just change the entry as shown to 2.7 and you're good to gosage attention 2.2.0 should work though, just check the link also above if you really use the right package
if you have downgraded from 2.8.0 to 2.7.1 you need to install a different sage package. also choose for the correct python version
runs like a charm....
but in case, (as i have one installation also running) use sage 2.1.1 from here
https://github.com/woct0rdho/SageAttention/releases/tag/v2.1.1-windowsReacted by NSR-007
ComfyUI-GGUF: Partial torch compile only, consider updating pytorchI even tested the nightly version but it didn't fix it.
Unfortunately, this problem still exists, even in the version
https://github.com/cardenluo/ComfyUI-Apt_Preset.git
in this gguf module
. It seems that gguf has completely crashed.