fix: give ClickHouse backfill jobs a typed table source - #612
Merged
Merged
Conversation
zzylol
marked this pull request as ready for review
September 10, 2026 23:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ClickHouse backfill jobs currently masquerade as Prometheus jobs using a reserved URL and always read the deployment's global table. Give each job an explicit ClickHouse database/table source, while keeping connection settings and credentials local to the deployment.
Before this PR, a
Prometheus { url: "clickhouse://configured" }job selected the configured default table. After this PR,ClickHouse { database, table }selects the job's validated table and enters the existing backfill lifecycle. Reject jobs whose table differs from the installed materialization or whose database differs from the deployment database before reading. Update process fixtures and documentation; the old sentinel is removed.Verification: both ClickHouse reader unit tests pass. The real compiled-publication mixed-DAG process E2E passes against ClickHouse, with a deliberately incorrect deployment default table to verify that the job table is used. The mismatch regression verifies that no reader is created; all five accelerator tests pass, including real reader and runtime tests. Source serde coverage includes ClickHouse. Existing unrelated compiler warnings remain.
This change does not yet add SQL population predicates, automatic materialization selection, streaming response decoding, or general catalog column/projection binding; those remain separate production-path work. No performance claim.