mvp #46: default archive engine + precompute /jobs API alignment - #119
Merged
Merged
Conversation
Two narrow follow-ons to the controller↔backend wiring work; v7
RoutingTarget flatten was dropped after audit found the controller
**does** emit shape-based multi-target routing (cold S3 fallback for
ad-hoc count/topk on `http_requests_total{service="payments"}`
relies on it). Multi-target stays.
## Default archive engine
When neither `ASAP_THANOS_QUERY_URL` nor the `ASAP_GORILLA_S3_*`
env-var family is configured, the backend used to leave the cold-tier
slot empty → archive queries returned `503 NoEngineRegistered`. New
`engines/no_data_archive.rs` registers a `NoDataArchiveEngine` stub
under `data_source_id = "no_data_archive"`, aliased onto
`gorilla_archive` so the existing `compatible_storage_backends`
failover sequence finds it. Returns an empty instant vector for any
query, with one warn-log at construction.
`ASAP_REQUIRE_ARCHIVE_ENGINE=1` opts back into the original
fail-loud `503` for environments where missing config should be
treated as a deploy error rather than soft-empty.
## Precompute /jobs API
The controller's `PrecomputeClient` (controller/src/config/
precompute.rs) registers + cancels precompute jobs via:
POST /api/v1/precompute/jobs register
DELETE /api/v1/precompute/jobs/:job_id cancel
The backend only routed `POST /api/v1/precompute` (different shape).
Adds the controller's `/jobs` routes alongside the existing route:
* `PrecomputeJobSpec` matches the controller's `JobRequest` byte-
for-byte (`{query, granularity, source, sketch_type, store_path}`).
* `PrecomputeJobRegistry` is an in-memory `Arc<RwLock<HashMap<...>>>`
on `HttpServer`/`AppState`. Registration returns
`{job_id (UUID), status: "created", created_at}`; DELETE returns
204 on hit, 404 on miss. Spec is tracked in memory; precompute
scheduling is a later wiring.
The legacy `POST /api/v1/precompute` route stays untouched.
Verification:
- `cargo build` clean.
- `cargo test --lib --no-fail-fast`: **881 passed / 32 failed / 11
ignored**. 32 failures are pre-existing datafusion + schema-
timeline-dispatch failures unrelated to this PR. New tests pass:
`engines::no_data_archive::tests::execute_returns_empty_instant_vector`,
`capabilities_use_no_data_archive_id`,
`drivers::query::servers::http::tests::http_precompute_jobs_register_then_delete_roundtrip`,
`http_precompute_jobs_delete_unknown_id_returns_404`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two narrow follow-ons to the controller↔backend wiring work. v7 RoutingTarget flatten was dropped after audit found the controller does emit shape-based multi-target routing (cold S3 fallback for ad-hoc count/topk on
http_requests_total{service="payments"}relies on it). Multi-target stays.4 files, +368 / −12.
(1) Default archive engine
When neither
ASAP_THANOS_QUERY_URLnorASAP_GORILLA_S3_*is set, the backend used to leave the cold-tier slot empty → archive queries returned503 NoEngineRegistered. Now:engines/no_data_archive.rsregisters a stub aliased ontogorilla_archive. Returns empty instant vector for any query with one warn-log at construction.Opt back into fail-loud via
ASAP_REQUIRE_ARCHIVE_ENGINE=1.(2) Precompute /jobs API
Controller's
PrecomputeClientcallsPOST /api/v1/precompute/jobsandDELETE /api/v1/precompute/jobs/:job_id. Backend only routedPOST /api/v1/precompute(different shape). Adds the controller's routes alongside the existing one:PrecomputeJobSpecmatches the controller'sJobRequestbyte-for-bytePrecomputeJobRegistryis anArc<RwLock<HashMap>>onHttpServer/AppState{job_id, status, created_at}; DELETE returns 204 / 404Test plan
cargo buildcleancargo test --lib --no-fail-fast: 881 passed / 32 failed / 11 ignored (failures match pre-existing main baseline; 4 new tests pass)mvp-multi-stage.ymloverlay withoutASAP_THANOS_QUERY_URLand confirm cold queries return empty instead of 503🤖 Generated with Claude Code