fix(sql): a MATERIALIZED VIEW refuses a HAVING its transform cannot express

[fupelaqu]

A MATERIALIZED VIEW whose HAVING an Elasticsearch transform cannot express is now refused at parse, instead of deploying a view that silently materialises the wrong groups.

Why a view differs from a search

The same HAVING is correct in a SELECT and unbuildable in a view. A search applies a key predicate through the terms include/exclude channel and arithmetic over aggregates through a bucket_script; a transform's pivot has neither — TransformGroupBy.TermsGroupBy emits {"terms":{"field":…}} and nothing else, and AggregateConversion.toTransformAggregation computes only MIN / MAX / SUM / AVG / COUNT. So the only correct answer inside the engine is to refuse, and to say why.

Every refused shape currently deploys a broken view: a bucket_selector reading a params.* its buckets_path never declares (which rejects every bucket — the view materialises empty), or no selector at all (the filter vanishes — every group materialises). All at HTTP 200.

The seven rules

#	refused	because
1	HAVING with no GROUP BY	no pivot exists, so no selector can
2	a condition on a grouping key	a transform's group_by has no include/exclude channel
3	an aggregate no transform can compute (STDDEV, VARIANCE, percentiles, ranking)	toTransformAggregation answers None; the path would name an aggregation never created
4	a SELECT bucket_script alias (MAX(x) - MIN(x) AS d … HAVING d > 3)	a pivot has no bucket_script channel; buckets_path comes back empty
5	an aggregate on the value side of a comparison	the path builder walks the identifier side only, so the right operand is never declared
6	an aggregate the SELECT list does not publish	empty or partial buckets_path
7	backstop — any HAVING leaf naming a metric the pivot does not create	catches what the six miss, relation predicates included

Rules 2–6 stay in front of the backstop for their specific remedies, though it subsumes several: a generic message would lose the actionable part.

Coverage — derived, not sampled

A Cartesian derivation over 2,917 statements (aggregate leaves enumerated from the sealed type by reflection; operand position × publication × expression shape × reference form × grouping × connective; the extensions consumer transcribed from source and re-transcribed independently for the re-run) classified every cell as CORRECT / REFUSED-RIGHTLY / UNCOVERED / OVER-REFUSED.

	CORRECT	REFUSED-RIGHTLY	UNCOVERED	OVER-REFUSED
control 04b1c694	185	0	1,835	0
this branch	185	1,821	14	0

1,821 of the control's 1,835 silent mis-deployments are closed, and nothing sound is refused.

⚠️ Honest provenance of those numbers. An earlier derivation over a different corpus reported 0 over-refusals for a six-rule version; an independent re-derivation with a rebuilt corpus then found 74 — statements correct on main that the rules refused, because the first corpus contained no case where the right-hand aggregate is also declared by a sibling conjunct. That defect is fixed, and the table above is measured on the corpus that caught it. It is not a third independent derivation; if you want one before this leaves draft, say so.

Rule 5's sole-owner population went 99 → 25 cells, all 25 unsound — it now owns only what is genuinely broken.

⚠️ Rules 1, 3 and 4 own zero cells the backstop does not also catch. They are kept for their specific remedies, by decision, not because they add coverage — and the scaladoc now says so with the numbers, so nobody mistakes seven rules for seven rules' worth of protection.

Nothing sound is refused and no statement that behaved correctly changes: every refused shape was already broken on main.

🔴 Each rule's remedy is applied literally in the test suite and asserted to produce an accepted statement. Three rounds running, a refusal's own advice was the defect — including "spelled exactly as the HAVING spells it", which is precisely the instruction that produced a broken artefact, because a HAVING never spells an alias. That is now pinned, not rediscovered.

MV-only, in both directions

These rules apply to CREATE MATERIALIZED VIEW and to nothing else. Every refused body, as a plain SELECT, is still accepted and emits a byte-identical query to main — measured across the whole corpus, and guarded by a mutation that leaks the rules into SingleSearch.validate() and reddens 6 tests.

Verification

sql 1661 · core 1146 · bridge 238 · es6bridge 234 — green. + compile on 2.12.20 and 2.13.16. The exact CI lint line. Integration on real Elasticsearch 8.18.3 (47/47) and 6.8.23 (47/47). GrammarDiffProbe 18,532 inputs → 0 differing; emission differential over 1,624 GROUP BY/HAVING inputs → 0 differing. 22 of 24 mutations red; the 2 greens are second-line-of-defence guards measured unreachable because an earlier rule refuses their population first — documented at both sites, reported as green-with-reason rather than as passes.

Cost: 6 interleaved rounds with the run order rotated and a byte-identical second control as the must-be-zero floor — no signal on any instrument. (A fixed order manufactured a phantom +8 % earlier in this work; rotation collapsed it. Worth knowing for anyone measuring this way.)

Release note

A materialized view whose HAVING filters on the grouping key, has no GROUP BY, references an aggregate the view does not publish or cannot compute, compares two aggregates, or reads a SELECT bucket-script alias, now fails at CREATE where it previously succeeded and silently materialised the wrong groups. The commonest case by far is an aggregate with no SELECT alias — SELECT city, SUM(amount) FROM t GROUP BY city HAVING SUM(amount) > 5 — which is fixed by writing SUM(amount) AS total. Every refusal names the clause and the remedy.

Residual, measured and NOT fixed

14 cells — MATCH (…) AGAINST (…) in a HAVING. MultiMatchCriteria is neither a Predicate, an Expression nor an ElasticRelation, so it falls into the case _ arm of three separate walks. Not a regression and not view-specific: the same conjunct is dropped in a plain SELECT too, identically on main (HAVING SUM(x) > 5 AND MATCH (name) AGAINST ('y') renders only the SUM half; HAVING MATCH(...) alone renders 1 == 1). That is #389's conflation still live for MultiMatchCriteria, and it belongs to the search surface rather than to this change.

Recorded, not fixed — extensions-side, and the honest root

1. Stage.extractAggregatePaths walks expr.identifier and never expr.maybeValue, while core's Expression.extractAllMetricsPath walks both. That asymmetry is why rule 5 exists.
2. The name-identity disagreement behind the no-alias family: core mints a generated alias that reaches the HAVING identifier, extensions names the aggregation from the user alias or the source field. Refusing is the workaround; making the two agree is the repair.
3. toTransformAggregation drops DISTINCT outside COUNT, so SUM(DISTINCT x) emits a plain SUM — in the view column as well as the filter. Pre-existing and wider than HAVING.
4. Adjacent, outside this surface: a view selecting only an uncomputable aggregate deploys a pivot with no aggregations at all, so the metric column is silently absent.

Neither candidate repair in (1) or (2) has been tested. (1) is insufficient alone — it declares the right operand only when that operand is published — and (2) has no obviously-correct side: naming the aggregation after the generated alias would put __cN in the view's schema. They are recorded as measurements, not as a plan.

🤖 Generated with Claude Code