Skip to content

stable-diffusion.cpp doesn't compile with flash attention enabled #297

Description

@kmfouad

Trying to compile the program with flash attention enabled errors out
apparently, the ggml_flash_attn() function was replaced by ggml_flash_attn_ext() in ggml in this pr: ggml-org/ggml@1a81566
cmake configuration:
cmake -B build -DSD_FLASH_ATTN=ON
compilation:

$ cmake --build build
[ 15%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml.c.o
[ 15%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-backend.c.o
[ 23%] Building C object thirdparty/CMakeFiles/zip.dir/zip.c.o
[ 30%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-alloc.c.o
[ 38%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o
In file included from /home/kmf/pg/image/flash/thirdparty/zip.c:40:
/home/kmf/pg/image/flash/thirdparty/miniz.h:4988:9: note: ‘#pragma message: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.’
 4988 | #pragma message(                                                               \
      |         ^~~~~~~
[ 38%] Built target zip
[ 46%] Linking C static library libggml.a
[ 46%] Built target ggml
[ 61%] Building CXX object CMakeFiles/stable-diffusion.dir/upscaler.cpp.o
[ 69%] Building CXX object CMakeFiles/stable-diffusion.dir/model.cpp.o
[ 69%] Building CXX object CMakeFiles/stable-diffusion.dir/stable-diffusion.cpp.o
[ 76%] Building CXX object CMakeFiles/stable-diffusion.dir/util.cpp.o
In file included from /home/kmf/pg/image/flash/esrgan.hpp:4,
                 from /home/kmf/pg/image/flash/upscaler.cpp:1:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
In file included from /home/kmf/pg/image/flash/stable-diffusion.cpp:1:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
In file included from /home/kmf/pg/image/flash/preprocessing.hpp:4,
                 from /home/kmf/pg/image/flash/util.cpp:13:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:118: CMakeFiles/stable-diffusion.dir/util.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:104: CMakeFiles/stable-diffusion.dir/upscaler.cpp.o] Error 1
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:90: CMakeFiles/stable-diffusion.dir/stable-diffusion.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:167: CMakeFiles/stable-diffusion.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions