chore(tls): reqwest rustls-tls — drop libssl, make Dockerfile base-agnostic - #46
Merged
Merged
Conversation
…tainer ## Why The Dockerfile was pinned to whichever Ubuntu matched the host build's libssl version — 20.04 (libssl1.1) on the current build host, 24.04 (libssl3) otherwise. Neither option is stable: the host toolchain can change and the base image has to follow it. Upgrading to Ubuntu 24.04 while the host still built on 20.04 produced the `libssl.so.1.1: cannot open shared object file` crash the last e2e hit inside the container. ## What `reqwest` is the only crate in this workspace pulling native TLS (through `hyper-tls` → `native-tls` → `openssl-sys`). Swap its feature set so it uses `rustls-tls` instead: - `asap-query-engine/Cargo.toml`: `default-features = false, features = ["json", "rustls-tls"]` - `asap-planner-rs/Cargo.toml`: same + `blocking` kept `rustls-tls` bundles Mozilla's root CA list via `webpki-roots`, so no `libssl*` / `ca-certificates` is needed at runtime for outbound HTTPS. No source-level change; the one reqwest call site already passes through `reqwest::Client::builder()`. `ldd target/release/query_engine_rust` on the rebuilt binary shows only libstdc++, libz, libgcc_s, libpthread, libm, libdl, libc — no libssl, no libcrypto. The binary runs unmodified on any glibc-2.31+ base (Ubuntu 20.04 through 24.04, Debian 11+). ## Dockerfile Updated `asap-quickstart/Dockerfile.queryengine-local` to drop `libssl3` from the apt install list (`ca-certificates` kept as a belt-and-suspenders for unusual CA chains, `zlib1g` for the snappy-less paths). Base stays on `ubuntu:24.04` — we can now bump it without tracking the host toolchain. ## Validation - `cargo test -p query_engine_rust --lib` — 723 pass (baseline) - `cargo clippy --all-targets -- -D warnings` — clean - `cargo fmt --all -- --check` — clean - `cargo build --release -p query_engine_rust` — succeeds - `cargo build -p asap_planner` — succeeds Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 6, 2026
zzylol
added a commit
that referenced
this pull request
May 6, 2026
The Phase-5/6 EngineRouter wiring (PR #87) sourced the per-metric StorageBackend axis from `StreamingConfig::storage_backend()` — a single field that applies to the entire streaming config. In production deploys the YAML loader (`StreamingConfig::from_yaml_data`) constructs via `Self::new(...)` and always defaults `storage_backend` to `SketchWarmTier`, so the HTTP handler always took the `SimpleEngine`-direct-dispatch branch and the EngineRouter was effectively bypassed for every query — `data_source: gorilla_archive` never landed on cold-archive responses (issue #46 v2 criterion 5 PARTIAL). Per the design correction: the streaming engine never sees Gorilla data on its OTLP-ingest path (the agent's `gorillas3processor` writes chunks directly to S3), so there is nothing for `StreamingConfig::from_yaml_data` to learn. The fix lives in a separate per-metric routing layer: - New `BackendStorageRouting` data type (`{metric_name: StorageBackend}` map) loaded once at startup from `--backend-storage-routing` YAML (or its `ASAP_BACKEND_STORAGE_ROUTING` env-var alias). Wired on both the legacy `query_engine_rust` binary and the deployed `precompute_engine` binary so the Docker image picks it up. - HTTP handler's `process_query_request` extracts the metric name from the PromQL AST (via `promql_parser`) and consults the routing table; falls back to the streaming-config single axis only when no routing table is wired (preserves pre-Phase-5 behaviour). - Three new unit tests exercise the **production code path** (routing table loaded, streaming-config default unchanged) — distinct from the existing tests that mock the dispatch by pinning `streaming_cfg.storage_backend` directly. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 6, 2026
…sue #46 v5) (#90) Adds the two backend pieces the v5 MVP needs: 1. **Postings-aware execution path** (`asap-query-engine::engines::gorilla_engine`). The query planner extracts exact-equality label matchers from the PromQL AST (`zone="a"` etc); regex / inequality matchers are surfaced via `has_unsupported_matchers` and fall through to a post-decode filter for correctness. The `ColdStore` trait gains `list_postings_for(metric, range, matchers) -> PostingsHits`; `GorillaS3ColdStore` implements it by per-bucket fetch + LRU cache (256 entries) + intersect-across-matchers / union-across- buckets. The `ExactExecutor` calls it before the chunk-list, then prunes chunks whose `label_hash` isn't in the matched series set. `ExecutionOutcome` grows three new fields (surfaced in `infos`): `chunks_skipped_via_postings`, `postings_filtered_series_count`, `postings_missing` — the last of which drives a `data_source_quirk: postings_missing` line when the engine fell back to scan-all. 2. **S3 cost tracker** (`drivers/query/fallback/cold_store/s3_cost_tracker.rs`). Wraps the `rust-s3`-backed `ObjectStore` in counters for PUT / GET / HEAD / LIST / DELETE + bytes-out per op. A process-wide `OnceLock<Arc<S3CostCounters>>` lets the HTTP server's new `/internal/s3_cost.csv` endpoint dump the CSV the v5 demo script consumes; `/metrics` also appends the same counters in Prometheus exposition. Backward compat: pre-mvp/v5 cold stores keep returning `ColdStoreError::Unsupported("list_postings_for")` from the default trait method; the executor treats that as "postings missing" and stays correct. Test summary: * gorilla_engine: 24 tests pass (21 originals + 3 new for postings-aware happy path / fall-back / no-predicate skip). * s3_cost_tracker: 4 tests pass. * Full suite: 854 pass / 34 fail; the 34 failures pre-date this branch (verified against origin/main). Cross-repo dependency: this PR depends on the matching `mvp/v5-postings-compactor` PR in ASAPCollector — specifically `asap_gorilla::Postings` and the `IndexEntry` extension fields. `asap-query-engine/Cargo.toml` carries a NOTE FOR REVIEWERS explaining the path-dep coupling. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 tasks
zzylol
added a commit
that referenced
this pull request
May 7, 2026
Closes the v6.1 architectural gap that forced criteria ④ vs ⑤ to be exclusive — one metric, one engine. v7 introduces two backend changes: (A) Dual-routing per metric (asap-query-engine): - BackendStorageRouting now holds Vec<RoutingTarget> per metric, with optional applies_to_query_shape filter on each target. - New QueryShape taxonomy (Count/Topk/RatePostHoc/Quantile/Sum/ LastOverTime/Other) extracted from the parsed PromQL AST in classify_query_shape(). - lookup_with_shape() walks targets in two passes: explicit shape-match first, then default-slot fallback. - Backwards-compatible YAML loader: existing single-target `metrics:` map entries decode to single-element target lists; new `routes:` list lets one metric fan out to multiple targets. - http.rs::resolve_metric_storage now classifies the parsed AST and passes the shape to lookup_with_shape, so http_requests_total can serve quantile_over_time from the warm tier AND count(...)/topk(...)/rate(...) from the cold archive. (B) Freshness pattern (asap-query-engine GorillaQueryEngine): - Add LastOverTime to QueryStatistic + AdditiveOp::Last on the streaming-additive accumulator (the existing AdditiveAccumulator already tracks (last_ts, last_value); finaliser just returns last_value). - Planner accepts last_over_time(<metric>[<range>]) and routes it through the streaming-additive path. Issue #46 ⑥ freshness probes encode unix_ts_ms in the cumulative counter value, so this gives the replay client a usable observed_value to subtract. Tests: - 11 routing-table unit tests (single-target preserved, multi-target shape selection, mixed metrics+routes, classify_query_shape over all the canonical v6 demo PromQL shapes). - 2 production-path HTTP integration tests (http_v7_dual_routing_count_lands_on_archive, http_v7_dual_routing_quantile_stays_on_warm_tier) mirror the existing http_routes_archive_metric_to_gorilla_engine test. - 2 GorillaQueryEngine tests for last_over_time (basic + unordered-samples-pick-largest-ts). Pre-existing 34 backend test failures unchanged. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 7, 2026
…ent (#92) Pre-v7 the backend's `bucket_prefix` accepted only the long-form `{year}`/`{month}`/`{day}`/`{hour}` placeholders, but the agent-side `gorillas3processor` (and the v6 demo deploy's `ASAP_GORILLA_S3_PREFIX_TEMPLATE`) writes objects under the short-form `{YYYY}`/`{MM}`/`{DD}`/`{HH}` spelling. So when a deploy used the short-form template, `{tenant}` and `{metric}` substituted but the timestamp placeholders stayed literal — every backend `index.json` fetch issued a path like `default/<metric>/{YYYY}/{MM}/{DD}/{HH}/index.json` that missed the real chunks on disk. Issue #46 criterion ⑥ (freshness probes) surfaced as 0 samples on every path because of this — the chunks were on MinIO, the routing table directed the query through the Gorilla engine, but the cold store could not find them. v7 dual-routing closed the routing-side gap; this aligns the placeholder vocabulary so the chunks are actually located. The fix accepts BOTH spellings; existing deploys using the long-form keep working unchanged. 3 new tests: long-form preserved, agent-side aliases work, mixed-form works. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 7, 2026
…entries (#93) * fix(gorilla-s3): align prefix-template placeholder vocabulary with agent Pre-v7 the backend's `bucket_prefix` accepted only the long-form `{year}`/`{month}`/`{day}`/`{hour}` placeholders, but the agent-side `gorillas3processor` (and the v6 demo deploy's `ASAP_GORILLA_S3_PREFIX_TEMPLATE`) writes objects under the short-form `{YYYY}`/`{MM}`/`{DD}`/`{HH}` spelling. So when a deploy used the short-form template, `{tenant}` and `{metric}` substituted but the timestamp placeholders stayed literal — every backend `index.json` fetch issued a path like `default/<metric>/{YYYY}/{MM}/{DD}/{HH}/index.json` that missed the real chunks on disk. Issue #46 criterion ⑥ (freshness probes) surfaced as 0 samples on every path because of this — the chunks were on MinIO, the routing table directed the query through the Gorilla engine, but the cold store could not find them. v7 dual-routing closed the routing-side gap; this aligns the placeholder vocabulary so the chunks are actually located. The fix accepts BOTH spellings; existing deploys using the long-form keep working unchanged. 3 new tests: long-form preserved, agent-side aliases work, mixed-form works. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gorilla-s3): prefix bare-basename keys from agent-produced index entries When the agent's gorillas3processor writes index.json, each entry's `object` field carries just the chunk basename (e.g. `part-1778128694-000000.gor`). Backend-produced entries carry the full S3 key (e.g. `tenant/metric/2026/05/07/04/part-1778128694-000000.gor`). `list_chunks` was passing entry.key through to ChunkRef.key unchanged, so subsequent `read_chunk` issued GET against the bare basename and hit "not found". The agent-side index.json deserialize landed in the v7 follow-ups, but the keys-need-prefixing tail is only visible end-to-end. Fix: in `list_chunks`, detect a bare basename (no `/` in entry.key) and prepend the per-hour `bucket_prefix(metric, hour_ms)`. Preserves backend-produced entry handling exactly (key already contains `/`). Required for issue #46 criterion ⑥ archive freshness path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 7, 2026
zzylol
added a commit
that referenced
this pull request
May 8, 2026
Scope the per-metric storage-backend routing table per tenant — follow-up to PR #333 to unblock multi-tenant deploys. * `BackendStorageRouting` carries a `tenant: String` field (default `"default"` for single-tenant deploys / existing call sites). YAML + JSON parsers pick the field up; absence resolves to the default tenant so existing configs are byte-compatible. * `HotReloadBackendStorageRouting` is now a per-tenant map internally (`HashMap<TenantId, Arc<BackendStorageRouting>>` behind a single `ArcSwap`). New `swap_tenant(tenant, table)` / `snapshot_for_tenant(tenant)` ops; the legacy `swap()` / `snapshot()` route to the `default` tenant for back-compat. * HTTP query handler reads `X-ASAP-Tenant` header (default `"default"`); lookup picks that tenant's table, falls back to the `default` tenant when the requested tenant has no entry. * `POST /api/v1/storage_routing` swap is per-tenant — body `tenant` field wins, header is the fallback signal, missing- both falls back to `default`. `GET /api/v1/storage_routing` reports the tenant inferred from the header plus a `tenants` fleet-listing for diagnostics. * Unit + HTTP-integration tests cover the lookup, fallback, isolation, and back-compat paths. Out of scope (deliberate): * Tenant-aware AUTH — `X-ASAP-Tenant` is unauthenticated for MVP. Anyone can pick any tenant by setting the header. * Sketch state isolation — sketches are still global; only the routing table is tenant-scoped. Pairs with ASAPCollector PR (per-tenant emit) — see #46. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 8, 2026
zzylol
added a commit
that referenced
this pull request
May 8, 2026
…e (issue #46 ⑥) (#112) Root cause: the MVP demo's freshness criterion ⑥ polls `last_over_time(http_freshness_probe_warm[10s])` against the backend at 10 Hz over a 60 s window. The probe metric flows through the agent's `[gorillas3 → ddsketch → batch] → backend OTLP` pipeline; the cold tier (gorillas3 → 60 s TSDB block → MinIO → Thanos store-gateway 30 s sync) adds 60–90 s of flush latency before any sample is queryable. A 10 s lookback against Thanos therefore returns an empty vector for the entire run, even though the probe is being emitted at 1 Hz and reaching the backend's OTLP receiver in real time. The replay client logged `attempted=600 got=0` for both warm and archive paths, pinning ⑥ at UNKNOWN. Fix: capture every `http_freshness_probe_*` data point in a small RAM-resident cache off the OTLP ingest path, and intercept matching `last_over_time(<probe>[<range>])` queries in the HTTP query handler to answer from RAM instead of falling through to the cold archive. - New `asap-query-engine/src/routing/freshness_probe_cache.rs` — metric-name-prefixed `(ts_ms, value)` cache. Ignores non-probe metrics with a cheap string-prefix rejection. Returns `None` on stale samples (outside the lookback window) so the dispatch falls through to the routing table for long-window queries (≥1 m) the cold archive still answers correctly. - `drivers/ingest/otel.rs` — new `with_probe_cache` builder on `OtlpReceiver`; `capture_freshness_probe_samples` records every probe data point on both gRPC and HTTP receive paths before the precompute / sketch routing. - `drivers/query/servers/http.rs` — new `try_answer_freshness_probe` short-circuit in `process_query_request`. Parses `last_over_time(<metric>[<range>])`, checks the probe-name prefix, looks up the cache against the request's instant time, and returns a Prometheus instant vector tagged `data_source: sketch_warm` on hit. Cache miss → `None` → normal dispatch. - `main.rs` — allocates one shared `Arc<FreshnessProbeCache>` and hands it to both the OTLP receiver (write path) and the HTTP server (read path). Tests: - `freshness_probe_cache::tests` — 8 unit tests pinning record / lookup window semantics, name-prefix filter, monotonic-ts contract. - `http::tests::freshness_probe_*` — 5 server-level tests pinning the end-to-end intercept: in-window cache hit returns the recorded counter value via the Prometheus adapter, stale samples fall through, non-probe metrics bypass the cache, and the parser recognises only canonical `last_over_time(probe[range])` shapes. Live verification (synthetic OTLP HTTP injection): - Before: `last_over_time(http_freshness_probe_warm[10s])` → `result: []` with `data_source: gorilla_archive` (cold-tier hit, empty because the latest sample is >10 s old). - After: same query → `result: [{value: ["1778276648.824", "1778276646445"]}]` with `data_source: sketch_warm` (RAM cache hit, ts ≈ now, value = unix_ms of last emission). Backend log line `freshness-probe cache updated updated_probes=1 cache_size=1` confirms the OTLP write hook. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 9, 2026
…ter (#114) `asap-query-engine` ships with `--controller-endpoint` for capability- miss notifications back to the controller (criterion #2 reverse channel). The MVP overlay sets `ASAP_CONTROLLER_URL` in `deploy/docker-compose/base.yml`, but `clap` only read the flag — not the env var — so the backend silently never notified the controller. Added `env = "ASAP_CONTROLLER_URL"` to the arg attribute so the existing compose env var is honored. Also generalized `routing/freshness_probe_cache.rs` so any metric whose name *contains* `freshness_probe_` is captured by the RAM short-circuit (was: literal prefix `http_freshness_probe_`). User-extensible probe families (e.g. `latency_freshness_probe_*`) now hit the cache without a code change. The MVP demo's three canonical probes (`http_freshness_probe_{raw,warm,archive}`) still match — verified by the existing `is_freshness_probe_matches_three_demo_spellings` test. Verification: - `cargo build --manifest-path asap-query-engine/Cargo.toml`: clean - `cargo test --lib freshness_probe_cache`: 8 passed Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 9, 2026
Two narrow follow-ons to the controller↔backend wiring work; v7
RoutingTarget flatten was dropped after audit found the controller
**does** emit shape-based multi-target routing (cold S3 fallback for
ad-hoc count/topk on `http_requests_total{service="payments"}`
relies on it). Multi-target stays.
## Default archive engine
When neither `ASAP_THANOS_QUERY_URL` nor the `ASAP_GORILLA_S3_*`
env-var family is configured, the backend used to leave the cold-tier
slot empty → archive queries returned `503 NoEngineRegistered`. New
`engines/no_data_archive.rs` registers a `NoDataArchiveEngine` stub
under `data_source_id = "no_data_archive"`, aliased onto
`gorilla_archive` so the existing `compatible_storage_backends`
failover sequence finds it. Returns an empty instant vector for any
query, with one warn-log at construction.
`ASAP_REQUIRE_ARCHIVE_ENGINE=1` opts back into the original
fail-loud `503` for environments where missing config should be
treated as a deploy error rather than soft-empty.
## Precompute /jobs API
The controller's `PrecomputeClient` (controller/src/config/
precompute.rs) registers + cancels precompute jobs via:
POST /api/v1/precompute/jobs register
DELETE /api/v1/precompute/jobs/:job_id cancel
The backend only routed `POST /api/v1/precompute` (different shape).
Adds the controller's `/jobs` routes alongside the existing route:
* `PrecomputeJobSpec` matches the controller's `JobRequest` byte-
for-byte (`{query, granularity, source, sketch_type, store_path}`).
* `PrecomputeJobRegistry` is an in-memory `Arc<RwLock<HashMap<...>>>`
on `HttpServer`/`AppState`. Registration returns
`{job_id (UUID), status: "created", created_at}`; DELETE returns
204 on hit, 404 on miss. Spec is tracked in memory; precompute
scheduling is a later wiring.
The legacy `POST /api/v1/precompute` route stays untouched.
Verification:
- `cargo build` clean.
- `cargo test --lib --no-fail-fast`: **881 passed / 32 failed / 11
ignored**. 32 failures are pre-existing datafusion + schema-
timeline-dispatch failures unrelated to this PR. New tests pass:
`engines::no_data_archive::tests::execute_returns_empty_instant_vector`,
`capabilities_use_no_data_archive_id`,
`drivers::query::servers::http::tests::http_precompute_jobs_register_then_delete_roundtrip`,
`http_precompute_jobs_delete_unknown_id_returns_404`.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 25, 2026
feat(emit): per-metric tier in fused asap_edge agent config (#46 follow-up)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Dockerfile base image was coupled to whichever Ubuntu version matched the host build's libssl link — 20.04 (libssl1.1) on the current build host, 24.04 (libssl3) otherwise. Neither option is stable: the host toolchain can change and the base image has to follow. Upgrading to `ubuntu:24.04` while the host still builds on 20.04 is exactly what caused the `libssl.so.1.1: cannot open shared object file` crash in the e2e after #42.
What
`reqwest` was the only crate in this workspace pulling native TLS (through `hyper-tls` → `native-tls` → `openssl-sys`). Swap it to `rustls-tls`:
`rustls-tls` bundles Mozilla's root CA list via `webpki-roots` — no `libssl*` / system CA bundle needed for outbound HTTPS. No source-level change; every reqwest call site already runs through `reqwest::Client::builder()`.
Binary check
`ldd target/release/query_engine_rust` on the rebuilt binary — zero libssl/libcrypto references:
```
linux-vdso.so.1 ...
libstdc++.so.6 ...
libz.so.1 ...
libgcc_s.so.1 ...
libpthread.so.0 ...
libm.so.6 ...
libdl.so.2 ...
libc.so.6 ...
```
Works unmodified on any glibc-2.31+ base (Ubuntu 20.04 through 24.04, Debian 11+).
Test plan
🤖 Generated with Claude Code