Skip to content

feat(brain): vecq-core 0.3.0 — keyed persistence, residual default, brain.vector_bits knob - #548

Merged
ajianaz merged 1 commit into
developfrom
feat/547-vecq-core-0.3.0
Aug 31, 2026
Merged

feat(brain): vecq-core 0.3.0 — keyed persistence, residual default, brain.vector_bits knob#548
ajianaz merged 1 commit into
developfrom
feat/547-vecq-core-0.3.0

Conversation

@ajianaz

@ajianaz ajianaz commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

What

Closes #547: vecq-core 0.2.0 → 0.3.0, and uses what it ships.

  • Keyed persistence (upstream vecq#32 closed, format v1.3+): load_or_create_vecq now deserializes for real and keeps a healthy keyed index as-is. Legacy keyless files (pre-0.3.0), corrupt files, and dim mismatches rebuild dirty so the next cora index re-embeds. A dim guard is mandatory — add_keyed panics on wrong-dim vectors.
  • brain.vector_bits knob (residual|4|5|6, lenient, default residual): 4-bit base codes + second-pass residual rescoring. A recall study on cora's own embeddings (real Rust symbol texts through the default hashing provider, ground-truth cosine) put residual at recall@10 0.930/0.917/0.863 at 1k/5k/13k symbols — ahead of plain 5-bit (0.880/0.850/0.840) at every scale, at 4-bit scan speed. A width change rebuilds the index once so the config takes effect; vecq_file_needs_rebuild is width-aware.
  • Cross-project heal: the vector index is one shared file across projects; a rebuild now clears embed fingerprints for ALL projects (previously only the active one — other projects kept "embedded" fingerprints while their vectors were gone). The usearch dims-mismatch deletion path joins the same heal.
  • BrainConfig gets a manual Default (derived Default produced empty strings, inconsistent with the serde defaults).

Why

#546's lazy-load made the search path backend-aware "until upstream lands key serialization" — this is that serialization. Together: the vecq signal is now fully alive in fresh processes, and index rebuilds stop being a per-run cost.

Testing

  • New: keyed roundtrip survives reload, legacy keyless rebuild, corrupt rebuild, dim-mismatch rebuild, width-switch rebuild, roundtrip across all four widths, lenient vector_bits parsing, config merge/defaults.
  • Full suite 947 pass (923 unit + 2 fresh-process integration from fix(brain): load vector index from disk on the search path #546 + 16 + 6), clippy -D warnings, fmt clean, cora review --staged pass (pre-commit hook).
  • E2E with the real binary: brain.vector_bits: "6" config → indexed → .vecq header verified on disk (config → binary → bytes).

Closes #547

…rain.vector_bits knob

Upgrade the opt-in vecq store to vecq-core 0.3.0 and close #547.

- Keyed persistence (upstream vecq#32 closed, format v1.3+): symbol-id
  keys survive reload. load_or_create_vecq now deserializes for real and
  keeps a healthy keyed index; legacy keyless files, corrupt files, and
  dim mismatches rebuild dirty so the next  re-embeds. A dim
  guard is mandatory: add_keyed panics on wrong-dim vectors.
- brain.vector_bits config (residual|4|5|6, lenient, default residual):
  residual is best recall@10 at 4-bit scan speed in a recall study on
  cora's own embeddings (ahead of plain 5-bit at 1k/5k/13k scales).
  A width change rebuilds the index once so the config takes effect;
  vecq_file_needs_rebuild is width-aware.
- BrainConfig gets a manual Default (derived Default produced empty
  strings, inconsistent with the serde defaults).
- Cross-project heal: the vector index is one shared file; a rebuild
  (width/dims/legacy/corrupt) now clears embed fingerprints for ALL
  projects, not just the active one, and the usearch dims-mismatch
  deletion path joins the same heal (pre-existing fingerprint leak).

Tests: keyed roundtrip, legacy keyless rebuild, corrupt rebuild,
dim-mismatch rebuild, width-switch rebuild, roundtrip across all four
widths, lenient parsing, config merge. 947 pass incl. the fresh-process
integration tests from #546.

Closes #547
@ajianaz
ajianaz merged commit d9f9bac into develop Aug 31, 2026
15 checks passed
@ajianaz
ajianaz deleted the feat/547-vecq-core-0.3.0 branch August 31, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore(deps): bump vecq-core to 0.3.0

1 participant