Evaluate 27 ClickHouse SQL queries on current main - #595
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The merged ClickHouse shared-DAG path had not been evaluated against the 27-query o11y SQL corpus on current
main, so the old #560 result could no longer serve as acceptance evidence. This PR adds a reproducible current-main audit and records the resulting parser → Planner → publication → data-plane routing matrix.The run is pinned to
791f7d7b(which contains #589/#588/#592) and reuses only the corpus input. It invokes the production SQL frontend and post-ASAP selector, attempts a validated unifiedQueryPlanpublication for every plannable query, installs a current-generation catalog in the data plane, and asks the catalog-backed ClickHouse accelerator to route all 27 requests.Before this PR, the latest checked-in result described the removed sidecar architecture and reported 27 exact fallbacks without current pipeline evidence. After this PR,
tools/o11y-sql-main-eval/artifacts/matrix.jsonrecords 0 warm, 0 partial hybrid, 27 exact fallback, and 0 request failures. Twenty-four queries fail closed in the ClickHouse SQL frontend. q05, q06, and q23 reach post-ASAP MinMax plans but publication rejects their unboundmetricpopulation predicate, after which the data plane returnsCatalogMissand uses exact fallback. All 27 were sent through the production ClickHouse HTTP listener to a live ClickHouse 26.8.2.7 backend: every response was HTTP 200 and byte-for-byte equal to a direct exact request. Artifacts distinguishfallback_requested,exact_executed,exact_success, and direct-response equality.The report also audits IDs, schemas, bindings, and execution nodes. Current production uses one
QueryPlanNode,QueryNodeId,MaterializationBinding,SummaryDefinitionId,SummarySchema, and backendQueryLanguage; no SQL sidecar catalog or parallel activation lifecycle remains. SQL-specific structures are protocol/planning context adapters around the shared plan. Hand-built alternate plans are confined to tests.Verification:
tools/o11y-sql-main-eval/reproduce.shcargo test -q -p control_plane clickhouse(1 passed)cargo test -q -p data_plane query_engines::asap_clickhouse_query_engine(12 passed)cargo fmt --checkgit diff --checkLimitations: no query in this corpus can currently be published warm or hybrid, so this PR does not claim accelerated-versus-exact value equality or performance benefit. The report identifies the exact frontend and SDS population-binding gaps that prevent that experiment.