refactor: relocate controller into asap-query-backend + delete asap-common + scaffolding for centralized series_id

[zzylol]

Summary

Backend half of the controller-into-backend refactor. Companion PR in ASAPCollector: ProjectASAP/ASAPCollector#372.

Design doc: docs/design-controller-into-backend.md (target architecture, capability model, OTLP metadata + store layout, centralized series_id namespace with idempotency + ghost-sid handling + fault-tolerance, phase order).

What landed (per phase)

• Phase 8 — asap-common/ deleted; 4 crates relocated to crates/:

  • crates/asap_types, crates/asap_otel_proto, crates/promql_utilities, crates/datafusion_summary_library.
  • Audit (docs/phase8-asap-common-audit.md) found the collector-side Rust crates don't actually depend on any asap-common type — only prose comments. Bucket "wire-shared" was empty; no asap-wire-types crate was created.
  • Workspace Cargo.toml updated; asap-query-engine/Cargo.toml switches asap_otel_proto to .workspace = true. Dockerfile COPY asap-common → COPY crates.

• Phase 9 — controller/ moved verbatim from ASAPCollector. New controller/src/lib.rs exposes the crate as a library (was bin-only); main.rs rewritten from mod foo to use controller::foo so binary entrypoint and library surface share modules. asap-query-engine path-deps controller. (Companion PR removes the standalone controller container; OpAMP server now originates from backend host.)

• Phase 4 (scaffolding) — asap-query-engine/src/drivers/ingest/series_resolver.rs:

  • SeriesIdResolver — content-addressable + atomic compute-or-mint cache (DashMap-backed). Idempotent: same (metric, attrs) always returns the same sid for cache lifetime — the invariant that makes attribute-fallback recovery work.
  • canonical_attrs_fingerprint(attrs) — mirrors the patched OTel-Go exporter's attributesFingerprint bit-exactly so sender and resolver hash identically.
  • Unit tests for idempotency, distinctness, fingerprint key-sort independence.
  • otel.rs::Export now populates ExportMetricsServiceResponse.unknown_series_ids: Vec::new() (placeholder; Phase 4 finish populates on cache miss).

• Phase 5 (scaffolding) — asap-query-engine/src/stores/sketch_index.rs:

  • Capability enum: QuantileApprox(SketchKindHandle), CardinalityApprox, FrequencyTopk(SketchKindHandle). Replaces aggregation_id integer key.
  • SketchKindHandle enum mirrors controller::sketch_algebra::params::SketchKind for in-store lookup independence.
  • SketchConfig enum carries per-instance config (relative_accuracy / k / precision / rows×cols).
  • AccuracyBound::from_config derives (epsilon, confidence) per sketch family for response metadata.
  • SketchInstanceMetadata (sid → metadata) + SketchTimeSeries (per-series time-windowed state) two-level structure.
  • SidLookup::{Hit, Ghost, Unknown} drives the new query routing decision: warm-tier hit, ghost (registered identity, state never arrived because gateway merged it away upstream — fall through to Thanos), unknown (sender's cache stale — unknown_series_ids recovery).
  • Unit tests for ghost-sid classification, hit-after-append, accuracy bound derivation.

Deferred / out-of-scope (follow-up PRs)

• Phase 4 finish — ResolveSeriesIDs gRPC service definition + receive-path lookup that classifies incoming sids and either replies with SeriesAssignments or signals unknown_series_ids.
• Phase 5 finish — legacy SimpleMapStore call-site migration to the new SketchIndex. The legacy aggregation_id-keyed path stays in-tree until migration is complete; new types live alongside.
• Phase 7 — end-to-end empirical verification of centralized TSID + ghost-sid routing.

Caveats

• controller crate has 21 dead-code warnings — orthogonal cleanup, deferred.
• ASAP arm doesn't yet use the new types in the runtime path — they're plumbed and tested at unit level, awaiting wire-in.

Build / test plan

• cargo build --release clean for: workspace root, controller (lib + bin), query_engine_rust
• Companion ASAPCollector Rust crates (asap-precompute-rs, asap-gorilla-rust) build clean per Phase 8 audit
• Unit tests for SeriesIdResolver, SketchIndex pass
• Apply Phase 4 finishing PR; verify ResolveSeriesIDs RPC end-to-end
• Apply Phase 5 finishing PR; verify raw-name PromQL routes through capability lookup
• Apply Phase 7 verification PR; tcpdump shows compact sid-mode emission post-cache

🤖 Generated with Claude Code