[Bug] offload-to-cpu casues the process to terminate

[dm3rd]

Git commit

3d6064b

Operating System & Version

windows 10 pro 22h2

GGML backends

Vulkan

Command-line arguments used

sd-cli.exe -M img_gen -p "person, formal clothes, city park" -n "ugly, bad art" --sampling-method dpm++2s_a --steps 6 -W 832 -H 1024 -b 1 --cfg-scale 1.5 -s -1 --clip-skip 2 --embd-dir F:\sdcppwebui\sd.cpp-webui\models/embeddings/ --lora-model-dir F:\sdcppwebui\sd.cpp-webui\models/loras/ -t 0 --rng cpu --sampler-rng cpu --lora-apply-mode auto -o F:\sdcppwebui\sd.cpp-webui\outputs/txt2img/20260509_184635_078417_6_steps.png --model F:\sdcppwebui\sd.cpp-webui\models/checkpoints/prefectPonyXL_v50.safetensors --vae F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors --type q4_K --scheduler karras --max-vram 3.8 --vae-tile-overlap 0.5 --vae-tile-size 32x32 --offload-to-cpu --vae-tiling --fa --diffusion-conv-direct --vae-conv-direct --mmap --color

Steps to reproduce

run standard gen with offload-to-cpu on

What you expected to happen

an image would generate

What actually happened

after a second the generation gets terminated

Logs / error messages / stack trace

sd-cli.exe -M img_gen -p "person, formal clothes, city park" -n "ugly, bad art" --sampling-method dpm++2s_a --steps 6 -W 832 -H 1024 -b 1 --cfg-scale 1.5 -s -1 --clip-skip 2 --embd-dir F:\sdcppwebui\sd.cpp-webui\models/embeddings/ --lora-model-dir F:\sdcppwebui\sd.cpp-webui\models/loras/ -t 0 --rng cpu --sampler-rng cpu --lora-apply-mode auto -o F:\sdcppwebui\sd.cpp-webui\outputs/txt2img/20260509_184635_078417_6_steps.png --model F:\sdcppwebui\sd.cpp-webui\models/checkpoints/prefectPonyXL_v50.safetensors --vae F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors --type q4_K --scheduler karras --max-vram 3.8 --vae-tile-overlap 0.5 --vae-tile-size 32x32 --offload-to-cpu --vae-tiling --fa --diffusion-conv-direct --vae-conv-direct --mmap --color

stable-diffusion.cpp:212 - loading model from 'F:\sdcppwebui\sd.cpp-webui\models/checkpoints/prefectPonyXL_v50.safetensors'
[INFO ] model.cpp:219 - load F:\sdcppwebui\sd.cpp-webui\models/checkpoints/prefectPonyXL_v50.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:280 - loading vae from 'F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors'
[INFO ] model.cpp:222 - load F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors using torch zip format
[INFO ] stable-diffusion.cpp:305 - Version: SDXL
[INFO ] stable-diffusion.cpp:333 - Weight type stat: f16: 2513
[INFO ] stable-diffusion.cpp:334 - Conditioner weight type stat: f16: 585
[INFO ] stable-diffusion.cpp:335 - Diffusion model weight type stat: f16: 1680
[INFO ] stable-diffusion.cpp:336 - VAE weight type stat: f16: 248
[INFO ]
Subprocess terminated.

Additional context / environment details

I downloaded and tested exes untill I could generate with the offload-to-cpu on.
I tested with different combos of performance options and everyone with offload-to-cpu would always terminate.
without it I could generate images but would have very limited vram

[master-592-b8079e2] would generate but [master-593-3d6064b] and all release till today would not

I am on a rather old under powered amd system, ryzen 4500 and a 4gb rx480

[Mohammadgb0078]

I have this problem like too

I used Vulkan Windows this release

F:\stable-diffusion.cpp>sd-cli.exe --diffusion-model "z_image_turbo-Q8_0.gguf" --vae "ae.safetensors" --llm "gemma-3-4b-it-Q5_K_M.gguf" --cfg-scale 1.0 -p "a red car on street" --steps 4 -W 1024 -H 1024 -o output.png --offload-to-cpu --vae-on-cpu --vae-tiling
[INFO ] stable-diffusion.cpp:219  - loading diffusion model from 'z_image_turbo-Q8_0.gguf'
[INFO ] model.cpp:216  - load z_image_turbo-Q8_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:266  - loading llm from 'gemma-3-4b-it-Q5_K_M.gguf'
[INFO ] model.cpp:216  - load gemma-3-4b-it-Q5_K_M.gguf using gguf format
[INFO ] stable-diffusion.cpp:280  - loading vae from 'ae.safetensors'
[INFO ] model.cpp:219  - load ae.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:305  - Version: Z-Image
[INFO ] stable-diffusion.cpp:333  - Weight type stat:                      f32: 694  |    q8_0: 180  |    q5_K: 204  |    q6_K: 35   |    bf16: 28
[INFO ] stable-diffusion.cpp:334  - Conditioner weight type stat:          f32: 205  |    q5_K: 204  |    q6_K: 35
[INFO ] stable-diffusion.cpp:335  - Diffusion model weight type stat:      f32: 245  |    q8_0: 180  |    bf16: 28
[INFO ] stable-diffusion.cpp:336  - VAE weight type stat:                  f32: 244

F:\stable-diffusion.cpp>

[wbruna]

@dm3rd :

[master-592-b8079e2] would generate but [master-593-3d6064b] and all release till today would not

That's odd: there is nothing on that commit even close to this code path. Are you 100% sure this issue started with this exact version? (unfortunately, I'm on Linux, so I can't test it)

And FWIW, with those parameters --offload-to-cpu may not help that much: with a Q4_K SDXL model, I see a peak VRAM reduction from ~2.7G to ~2.6G, likely from the ~150M VAE weights. How much are you getting with master-592?

@Mohammadgb0078 , you may have similar symptoms, but it's likely a completely different issue:

--diffusion-model "z_image_turbo-Q8_0.gguf" --vae "ae.safetensors" --llm "gemma-3-4b-it-Q5_K_M.gguf"

ZIT should be combined with Qwen3-4B.

[Mohammadgb0078]

@dm3rd :

[master-592-b8079e2] would generate but [master-593-3d6064b] and all release till today would not

That's odd: there is nothing on that commit even close to this code path. Are you 100% sure this issue started with this exact version? (unfortunately, I'm on Linux, so I can't test it)

And FWIW, with those parameters --offload-to-cpu may not help that much: with a Q4_K SDXL model, I see a peak VRAM reduction from ~2.7G to ~2.6G, likely from the ~150M VAE weights. How much are you getting with master-592?

@Mohammadgb0078 , you may have similar symptoms, but it's likely a completely different issue:

--diffusion-model "z_image_turbo-Q8_0.gguf" --vae "ae.safetensors" --llm "gemma-3-4b-it-Q5_K_M.gguf"

ZIT should be combined with Qwen3-4B.

@wbruna I have tried Qwen3-4B I have same problem again

F:\>sd-cli.exe --diffusion-model "z_image_turbo-Q8_0.gguf" --vae "ae.safetensors" --llm "Qwen3-4B-Q3_K_S.gguf" --cfg-scale 1.0 -p "a red car on street" --steps 4 -W 1024 -H 1024 -o output.png --offload-to
-cpu --vae-on-cpu --vae-tiling
[INFO ] stable-diffusion.cpp:219  - loading diffusion model from 'z_image_turbo-Q8_0.gguf'
[INFO ] model.cpp:216  - load z_image_turbo-Q8_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:266  - loading llm from 'Qwen3-4B-Q3_K_S.gguf'
[INFO ] model.cpp:216  - load Qwen3-4B-Q3_K_S.gguf using gguf format
[INFO ] stable-diffusion.cpp:280  - loading vae from 'ae.safetensors'
[INFO ] model.cpp:219  - load ae.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:305  - Version: Z-Image
[INFO ] stable-diffusion.cpp:333  - Weight type stat:                      f32: 634  |    q8_0: 180  |    q3_K: 252  |    q6_K: 1    |    bf16: 28
[INFO ] stable-diffusion.cpp:334  - Conditioner weight type stat:          f32: 145  |    q3_K: 252  |    q6_K: 1
[INFO ] stable-diffusion.cpp:335  - Diffusion model weight type stat:      f32: 245  |    q8_0: 180  |    bf16: 28
[INFO ] stable-diffusion.cpp:336  - VAE weight type stat:                  f32: 244

F:\>

[dm3rd]

@dm3rd :

[master-592-b8079e2] would generate but [master-593-3d6064b] and all release till today would not

That's odd: there is nothing on that commit even close to this code path. Are you 100% sure this issue started with this exact version? (unfortunately, I'm on Linux, so I can't test it)

And FWIW, with those parameters --offload-to-cpu may not help that much: with a Q4_K SDXL model, I see a peak VRAM reduction from ~2.7G to ~2.6G, likely from the ~150M VAE weights. How much are you getting with master-592?

with --offload-to-cpu I get 2.3G, and without it I get 2.4G. the offload option means I can use multiple loras, where as without it I get limited to a few or less, also blocks upscaling them too.

I went back and retested some of the compiled exes and 593 was the one that started the terminting with --offload-to-cpu
I even tried 589 and that one was fine too.

i'm not sure what I can test for anything more helpful.

[wbruna]

As a test, you could try lowering memory usage, removing --mmap and --emb-dir. If it still triggers it, lower the quant (say, to q4_0). I'd also suggest pre-converting the model: not sure if it would matter for memory usage, but testing would be much faster.

[Green-Sky]

Also try setting --max-vram 2 for graph splitting.

[dm3rd]

I tired the things recommended above and got this:

F:\sdcppwebui\sd.cpp-webui>sd-cli.exe -M img_gen -p "person, formal clothes, city park" -n "ugly, bad art" --sampling-method dpm++2s_a --steps 6 -W 832 -H 1024 -b 1 --cfg-scale 1.5 -s -1 --clip-skip 2 --lora-model-dir F:\sdcppwebui\sd.cpp-webui\models/loras/ -t 0 --rng cpu --sampler-rng cpu --lora-apply-mode auto -o F:\sdcppwebui\sd.cpp-webui\outputs/txt2img/test_184635_078417_6_steps.png --model F:\sdcppwebui\sd.cpp-webui\models/checkpoints/xavier_v10-q4_0.gguf --vae F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors --scheduler karras --vae-tile-overlap 0.5 --vae-tile-size 32x32 --offload-to-cpu --vae-tiling --fa --diffusion-conv-direct --vae-conv-direct --color
←[34;1m[INFO ]←[0m stable-diffusion.cpp:210 - loading model from 'F:\sdcppwebui\sd.cpp-webui\models/checkpoints/xavier_v10-q4_0.gguf'
←[35;1m[WARN ]←[0m model.cpp:231 - file F:\sdcppwebui\sd.cpp-webui\models/checkpoints/xavier_v10-q4_0.gguf not found
←[31;1m[ERROR]←[0m stable-diffusion.cpp:212 - init model loader from file failed: 'F:\sdcppwebui\sd.cpp-webui\models/checkpoints/xavier_v10-q4_0.gguf'
←[34;1m[INFO ]←[0m stable-diffusion.cpp:278 - loading vae from 'F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors'
←[34;1m[INFO ]←[0m model.cpp:222 - load F:\sdcppwebui\sd.cpp-webui\models/vae/irisXLVAE_luna.safetensors using torch zip format
←[31;1m[ERROR]←[0m stable-diffusion.cpp:297 - get sd version from file failed: 'F:\sdcppwebui\sd.cpp-webui\models/checkpoints/xavier_v10-q4_0.gguf'
←[34;1m[INFO ]←[0m main.cpp:713 - new_sd_ctx_t failed

[Mohammadgb0078]

@wbruna I tried some args that wants to mount something on cpu like --vae-on-cpu --ofload-to-cpu and when I use it process will terminate without any error in verbose mode or normal

This is a bug bro, plz check the codes

[wbruna]

I was able to reproduce the crash with master-596-90e87bc on Wine: Vulkan itself works, but anything on the CPU backend triggers it (--clip-on-cpu, --vae-on-cpu, --offload-to-cpu, even an empty GGML_VK_VISIBLE_DEVICES). master-592-b8079e2 works fine for me in all cases.

And here, the OS was more helpful:

[DEBUG] ggml_extend.hpp:110 - Initializing backend: CPU
wine: Unhandled illegal instruction at address 00006FFFF6628515 (thread 012c), starting debugger...

Which pointed to the cause:

[DEBUG] main.cpp:550 - version: stable-diffusion.cpp version unknown, commit 90e87bc
[DEBUG] main.cpp:551 - System Info:
SSE3 = 1 | AVX = 1 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WA
SM_SIMD = 0 | VSX = 0 |

The master-596-90e87bc build has AVX512 enabled, but neither my old 3400G, nor @dm3rd 's 4500, supports it. @Mohammadgb0078 , what about your CPU?

This is a bug bro, plz check the codes

Funny thing is, that commit didn't change anything about the build.

[Green-Sky]

Funny thing is, that commit didn't change anything about the build.

Ah yes, the unpredictability that is github actions.

[Mohammadgb0078]

@wbruna My CPU is intel i3-10100, I didn't understand what you meant; I haven't worked on these topics at an advanced/specialized level

just a question, what is the solution?

[wbruna]

@wbruna My CPU is intel i3-10100, I didn't understand what you meant

It was for confirming that the issue affects CPUs without AVX512 support. A quick Google search shows that's the case of your CPU, too.

just a question, what is the solution?

Ideally, changing the Github action config to force-disable AVX512 for the next Windows builds. That's up to @leejet ; I could make a PR, but I'd have no way to test it.

But since the support depends on the machine the action ran, we may get working binaries from time to time. For instance, I've just tested the Vulkan binaries from master-601-eeac950; they should work for you.

[dm3rd]

@wbruna My CPU is intel i3-10100, I didn't understand what you meant

It was for confirming that the issue affects CPUs without AVX512 support. A quick Google search shows that's the case of your CPU, too.

just a question, what is the solution?

Ideally, changing the Github action config to force-disable AVX512 for the next Windows builds. That's up to @leejet ; I could make a PR, but I'd have no way to test it.

But since the support depends on the machine the action ran, we may get working binaries from time to time. For instance, I've just tested the Vulkan binaries from master-601-eeac950; they should work for you.

you were right 601 does work for me it seems.

[Mohammadgb0078]

you were right 601 does work for me it seems.

@dm3rd it seems working for me too

@wbruna I think the problem solved