Gemma-3 generates garbage after several hundreds (maybe) of tokens #65

Activity

1. foldl

   added

   bugSomething isn't workingSomething isn't working

   ggml

   vukan

   on Jun 6, 2025
2. foldl commented on Jun 11, 2025

   

   foldl

   on Jun 11, 2025

   OwnerAuthor

   More actions

   • CPU: OK
   • Vulkan: an odd max_length? such as --max_length 32767
   • CUDA: ??

   👍React with 👍1Reacted by arch-btw
3. arch-btw commented on Jul 7, 2025

   

   arch-btw

   on Jul 7, 2025

   Contributor

   More actions

   Maybe it's the value that seems way too high:

   "model_max_length": 1000000000000000019884624838656

   Found in this file: https://huggingface.co/google/gemma-3-4b-it/blob/main/tokenizer_config.json

   There is a thread on it here:

   https://huggingface.co/google/gemma-3-4b-it/discussions/48
4. foldl commented on Jul 7, 2025

   

   foldl

   on Jul 7, 2025

   Last edited by foldl

   OwnerAuthor

   More actions

   Possibly no. model_max_length in tokenizer_config.json is not used.

   It is likely related to un-aligned memory access.

   👍React with 👍1Reacted by arch-btw
5. foldl

   added

   important

   on Jan 13, 2026
6. foldl commented on Mar 23, 2026

   

   foldl

   on Mar 23, 2026

   OwnerAuthor

   More actions

   fixed by 8a5018e
7. foldl

   closed this as completedon Mar 23, 2026