Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 28 additions & 16 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,22 +48,34 @@ Deliverable: `AccuracyProfile::derive(&AggregationConfig)`
returns concrete (ε, δ) per sketch type — not a stub. Unit tests
against the sketchlib-bench corpus.

### 3. End-to-end capability-miss feedback loop test

The flow exists as code (`ControllerClient::create_plan` on
backend side, `/api/v1/plan` route on controller side), but
isn't tested end-to-end. For the paper's "controller reacts to
workload drift" claim we need:

- Test harness spins up controller + backend in-process (or
via Docker compose fixture)
- Issues query that causes capability-miss
- Asserts a new `StreamingConfig` arrives at the backend within
`N` seconds
- Asserts next matching query hits (no miss)
- Measures time-to-plan-ready + time-to-first-hit

Not new engineering — testing the existing code path end-to-end.
### 3. End-to-end capability-miss feedback loop test — **done (HTTP round-trip)**

HTTP-level e2e landed in
[`asap-query-engine/src/tests/capability_miss_http_e2e_tests.rs`](asap-query-engine/src/tests/capability_miss_http_e2e_tests.rs).
Spins up a real backend HTTP server + mock controller HTTP
server, fires a PromQL `sum(metric)` query that capability-misses,
and measures wall-clock `time_to_plan_ready` from query issue to
the backend observing the new `StreamingConfig` via
`GET /api/v1/streaming-config`. Localhost floor: ~20 ms.

Also asserts:
- Mock controller received the HTTP notify with the documented
`{kind: "capability_miss", ...}` payload
- Backend's hot-reload handle has the exact `agg_id` the
controller pushed
- A repeat query on the same metric does **not** fire a second
notify (loop is idempotent under query replay)

Follow-up (not paper-blocking):

- **Cross-process test.** Today's test is single-process with
two HTTP servers. A docker-compose harness that wires a real
DataCollector controller binary against the real backend
binary is tracked by blocker #6 of `DataCollector/TODO.md`.
- **Time-to-first-hit over real data.** The "next query returns
data" half of the story needs OTLP ingestion between the
plan push and the repeat query — still tracked as an
operational follow-up in the DataCollector repo.

### 4. Serialization format versioning tests

Expand Down
Loading