## Task Adopt Trinity configuration for 16MB Parameter Golf budget. ## Config (downscaled from full Trinity for size) - VOCAB_SIZE = 729 (3⁶, padded: active 0-255, logits 256-728 = -inf) - HIDDEN_DIM = 243 (3⁵) ← downscaled from 729 for 16MB budget - EMBED_DIM = 243 (3⁵, tied with hidden) - CONTEXT_LEN = 81 (3⁴) - NUM_BLOCKS = 9 (3²) - HEADS = 9 (3²) - HEAD_DIM = 27 (3³) - FFN_HIDDEN = 729 (3×hidden, 3⁶) - Quantization = GF16 (4-bit) - Optimizer = LAMB, lr=3e-4, lamb_clamp=10.0, stable_ratio=0.02 - Schedule = cosine, lr_min=1e-5 - Batch = 66 - Grad clip = 1.0 ## Size Check - Embed tied: 729×729×2B = 1.06MB ✓ - 9 blocks × (attn + ffn) = 6.5M params × 2B (GF16) = 13MB ✓ - Total: ~14.1MB < 16MB ✓ ## Acceptance Criteria - All dims ∈ {3^k} - ctx % head_dim == 0 && is_power_of_3(ctx / head_dim) (EXP-012) - vocab=729 padded (bytes 0-255 active, 256-728 logits = -inf) - GF16 quantization (4-bit) - Model size < 16MB ## Refs - EXP-001: Trinity 3³ physics - EXP-010: Ternary block alignment - EXP-012: Square Attention theorem - Issue #64: simple_backward fix ## Commit "feat(config): adopt Trinity config (hidden=243, vocab=729-padded, GF16)" Refs: EXP-001, EXP-010, EXP-012, #64
Activity
gHashTag commentedon Apr 20, 2026
Part of: 🏆 EPIC: Parameter Golf Submission — Trinity + GF16