feat: per-tenant BackendStorageRouting (part 1/2)

[zzylol]

Summary

Follow-up to PR #333 (issue #46): scope the per-metric
BackendStorageRouting table per tenant so multi-tenant
deploys can isolate each tenant's routing decisions while sharing
the warm sketch tier and archive tier infrastructure.

This is part 1/2 — the data model + backend lookup + hot-reload
swap. Part 2 lands in ASAPCollector (feat: per-tenant BackendStorageRouting emit (part 2/2)) and threads the tenant
through the controller's emit path.

Data model decision

Hybrid: BackendStorageRouting carries a tenant: String field
(default "default"), and HotReloadBackendStorageRouting
wraps a HashMap<TenantId, Arc<BackendStorageRouting>> behind
a single ArcSwap. The wrapper is the multi-tenant indirection;
the table itself is tenant-scoped.

This minimises blast radius:

• Existing single-tenant call sites keep using snapshot() /
  swap(); both route to the default tenant.
• New multi-tenant call sites use snapshot_for_tenant(tenant) /
  swap_tenant(tenant, table) — per-tenant pushes don't touch
  other tenants.
• Existing YAML / JSON configs (no tenant: field) keep parsing
  as default-tenant tables.

Backend lookup behaviour

• HTTP query handler reads X-ASAP-Tenant header. Missing /
  empty / non-UTF-8 → default.
• resolve_metric_storage snapshots that tenant's table.
  Unknown-tenant → falls back internally to the default
  tenant's table (one-level fallback chain).
• The process_query_request dispatch path only changes its
  routing-table read; the rest (engine override, EngineRouter
  dispatch) is untouched.

Hot-reload semantics

POST /api/v1/storage_routing:

• Body tenant field is the source of truth.
• X-ASAP-Tenant header is the fallback signal when the body
  leaves tenant implicit (preserves existing controller's emit
  shape; the controller-side patch in part 2 makes it explicit).
• Missing both → swap lands in the default tenant slot.
• Per-tenant CAS-loop preserves concurrent-writer correctness.

GET /api/v1/storage_routing:

• Reports the tenant inferred from the header.
• Adds a tenants: [...] fleet-listing of every registered
  tenant id for operator diagnostics.

Test coverage

Unit (in routing::backend_storage_routing::tests):

• empty_routing_carries_default_tenant
• json_payload_tenant_field_is_optional_and_defaults_to_default
• json_payload_tenant_field_is_picked_up_when_present
• yaml_tenant_field_is_optional_and_defaults_to_default
• yaml_tenant_field_is_picked_up_when_present
• hot_reload_swap_tenant_replaces_only_one_tenant
• hot_reload_unknown_tenant_falls_back_to_default_tenant
• hot_reload_swap_tenant_overwrites_table_internal_tenant_id
• hot_reload_tenant_ids_lists_all_registered_tenants
• hot_reload_legacy_swap_routes_to_default_tenant

HTTP integration (in drivers::query::servers::http::tests):

• storage_routing_post_per_tenant_isolates_tenants — body's
  tenant lands in the right slot; default tenant unchanged.
• storage_routing_post_per_tenant_via_header_when_body_implicit
  — header fallback when controller emits tenant-agnostic body.
• storage_routing_post_no_tenant_falls_back_to_default — back-
  compat with existing controllers.
• storage_routing_get_per_tenant_lists_all_tenants.

Existing single-tenant tests (the legacy swap() / snapshot()
suite, the v6.1 / v7 YAML parse + lookup suite, and the
storage_routing_* HTTP suite) still pass byte-for-byte.

Out of scope (deliberate, MVP)

• Tenant-aware AUTH. X-ASAP-Tenant is unauthenticated.
  Anyone can pick any tenant by setting the header. Tenant-aware
  auth is deferred to a follow-up before any multi-tenant deploy
  is considered production-ready.
• Sketch state isolation. Sketches are still global; only
  the routing table is tenant-scoped. Per-tenant sketch state
  is a much larger refactor (touches ingest, storage, and the
  precompute engine).
• Wire-protocol changes beyond the X-ASAP-Tenant header
  and the JSON tenant field.

Test plan

• cargo build --release passes
• cargo test --release --lib — all routing + HTTP storage-
  routing tests green (54/54 storage-routing; 10 datafusion
  failures are pre-existing on origin/main)
• Pairs with ASAPCollector part 2/2 — see linked PR

Related

• Issue chore(tls): reqwest rustls-tls — drop libssl, make Dockerfile base-agnostic #46 — multi-stage demo runbook
• PR feat(gorilla-merger): decode-on-read for cold intchunk value chunks #333 (ASAPCollector) — left per-tenant scope as a follow-up

🤖 Generated with Claude Code