Parent: #329 — Fully Ternary Transformer
Level 1 (needs D for quantizer) | ~150 LOC | New: src/hslm/ternary_attention.zig
Description
Replace float Q·K^T attention with ternary scoring: +1 (match), -1 (mismatch), 0 (ignore). Sparse attend with top-k → ternary attention weights at 33% density.
Implementation
ternaryScore(q: []const i2, k: []const i2) → i32 — count matches vs mismatchessparseAttend() — top-k scoring → ternary attention weights {-1, 0, +1}, 33% density- No softmax needed — just sign of top-k scores
- Value aggregation: ternary weights × ternary values = integer accumulation
tri CLI
Tests
- Score symmetry: score(q, k) == score(k, q)
- Sparse density: exactly 33% non-zero attention weights
- Attention output is valid ternary
Dependencies
- Needs D (Ternary Activations) for quantizer
Blocks
- Issue J (Full Ternary Inference)
Parent: #329 — Fully Ternary Transformer
Level 1 (needs D for quantizer) | ~150 LOC | New:
src/hslm/ternary_attention.zigDescription
Replace float Q·K^T attention with ternary scoring: +1 (match), -1 (mismatch), 0 (ignore). Sparse attend with top-k → ternary attention weights at 33% density.
Implementation
ternaryScore(q: []const i2, k: []const i2) → i32— count matches vs mismatchessparseAttend()— top-k scoring → ternary attention weights {-1, 0, +1}, 33% densitytri CLI
Tests
Dependencies
Blocks