Skip to content

ADFA-5188 | Enable KV cache quantization and flash attention - #76

Open
jatezzz wants to merge 1 commit into
fix/ADFA-5187-dynamic-n-ctxfrom
fix/ADFA-5188-kv-cache-quantization
Open

ADFA-5188 | Enable KV cache quantization and flash attention#76
jatezzz wants to merge 1 commit into
fix/ADFA-5187-dynamic-n-ctxfrom
fix/ADFA-5188-kv-cache-quantization

feat(ai-agent-local): store the KV cache as q8_0 to double the context

24429e7
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar