Skip to content

feat(sketchdb): cold-query fallback with local-FS raw store (§5.2) - #54

Merged
zzylol merged 1 commit into
mainfrom
feat/sketchdb-cold-fallback
Apr 21, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/sketchdb-cold-fallback

Conversation

@zzylol

@zzylol zzylol commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Addresses TODO.md blocker #1 — the paper's "sketches for hot, exact for cold" story needs a cold path that serves capability-miss queries from raw samples instead of hitting the live Prometheus (or failing).

Design

  • New ColdStore trait + LocalFsColdStore impl (v1). On-disk layout is byte-identical to the future S3 key layout:
    ```
    /raw//YYYY/MM/DD/HH/part-NNNNNN.jsonl
    ```
    Each line: `{"ts_ms": int64, "labels": {...}, "value": f64}`. Swapping to S3 is later a new impl of `ColdStore` — no adapter changes.
  • New `ColdFallback<S: ColdStore>` implements `FallbackClient`. Parses PromQL, extracts `(metric, predicates, op)`, scans the cold store, computes the answer. Supported shapes:
    • bare instant vector `metric{labels}`
    • no-grouping scalar aggregations (`sum|count|avg|min|max`)
    • matchers: `=`, `!=`
  • Unsupported shapes (regex matchers, `rate()`, `by (...)`, binary ops, etc.) delegate to the optional inner `FallbackClient` (chain-of-responsibility — typically the existing Prometheus proxy).
  • Hot / cold telemetry counters (`queryengine_hot_queries_total`, `queryengine_cold_queries_total`, `queryengine_cold_bytes_served_total`), keyed by `(metric, shape)`. Mirror the PR feat(metrics): wire §6.3 barrier counter into all OTLP ingest paths #51 pattern.
  • `AdapterConfig::prometheus_promql_with_cold` convenience constructor.

Routing

No engine changes. `process_query_request` already falls through to `FallbackClient` on engine-miss; configuring a `ColdFallback` naturally routes Purged / capability-miss queries through the cold tier, with the Prometheus proxy as tail-of-chain for shapes the cold adapter doesn't recognise.

Tests

752 → 777 (+25):

  • 5 unit tests on format + hour-prefix helpers
  • 4 on `LocalFsColdStore::scan` (range filter, missing prefix, hour-boundary span)
  • 10 on `ColdFallback` plan extraction + aggregation + float formatting
  • 6 end-to-end HTTP integration tests in `tests/cold_fallback_tests.rs` (bare selector, sum aggregation, label filtering, unsupported-shape delegation, telemetry increments, and the canonical "Purged range served from raw samples" paper claim).

clippy clean, fmt clean. One pre-existing flaky elastic forwarding test is unrelated (passes in isolation).

🤖 Generated with Claude Code

Addresses TODO.md blocker #1. Unblocks the paper's "sketches for
hot, exact for cold" story: capability-miss queries now serve
exact answers from raw observability samples in the cold tier
instead of hitting Prometheus (or failing) unconditionally.

## What's new

* `drivers/query/fallback/cold_store/` — storage-agnostic
  `ColdStore` trait + `LocalFsColdStore` impl. On-disk layout
  (`raw/<metric>/YYYY/MM/DD/HH/part-NNNNNN.jsonl`) is identical
  to what a future S3 cold store will use, so the adapter stays
  source-compatible when we swap backends.
* `drivers/query/fallback/s3_adapter.rs` — `ColdFallback<S>`
  implements `FallbackClient`. Parses PromQL, extracts
  `(metric, predicates, op)`, scans the cold store, computes
  the answer. Supported shapes for v1:
  - bare instant vector selector (`metric{labels}`)
  - no-grouping scalar aggregation
    (`sum|count|avg|min|max(metric{...})`)
  Label matchers: `=`, `!=` — regex delegated upstream.
  Anything outside this surface falls through to the optional
  inner `FallbackClient` (chain-of-responsibility, typically
  the existing Prometheus proxy).
* `drivers/query/fallback/metrics.rs` — hot/cold telemetry
  counters (`queryengine_hot_queries_total`,
  `queryengine_cold_queries_total`,
  `queryengine_cold_bytes_served_total`), keyed by
  `(metric, shape)`. Mirrors the PR #51
  schema-barrier-counter pattern.
* `AdapterConfig::prometheus_promql_with_cold` convenience
  constructor that composes the cold adapter in front of a
  Prometheus proxy.

## Routing

No engine changes needed. The existing `process_query_request`
already falls through to `FallbackClient` on engine-miss —
configuring the fallback as a `ColdFallback` naturally routes
Purged-segment / capability-miss queries through the cold tier
with the Prometheus proxy as the tail-of-chain for unsupported
shapes.

## Tests

752 → 777 tests (+25):

* 5 unit tests on the JSONL format + hour-prefix helpers
* 4 on `LocalFsColdStore::scan` (range filter, missing prefix,
  hour-boundary span)
* 10 on `ColdFallback` plan extraction + aggregation + float
  formatting
* 6 end-to-end HTTP integration tests in
  `tests/cold_fallback_tests.rs` covering: bare selector, sum
  aggregation, label filtering, unsupported-shape delegation,
  telemetry-counter increments, and the "Purged range served
  from raw" paper story.
@zzylol
zzylol merged commit 869ee79 into main Apr 21, 2026
@zzylol
zzylol deleted the feat/sketchdb-cold-fallback branch May 9, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant