Skip to content

HAVING <function>(<aggregate>) silently drops the entire HAVING clause — every group is returned #389

Description

@fupelaqu

A HAVING predicate whose aggregate is wrapped in any function emits no bucket_selector, so
the group filter never reaches Elasticsearch and the query returns every group. HTTP 200, no
warning. The #205 / #253 silent-wrong-answer family.

Measured on origin/main 975aa87b

Generated Elasticsearch query, via softclient4es-sql-bridge (SelectStatement(sql).query).

The control — this works, and proves the machinery is present:

SELECT status, COUNT(*) AS c FROM t GROUP BY status HAVING COUNT(*) > 1
{"aggs":{"status":{"terms":{"field":"status","size":65536,"min_doc_count":1},
  "aggs":{"c":{"value_count":{"field":"_index"}},
    "having_filter":{"bucket_selector":{"buckets_path":{"c":"c"},
      "script":{"source":"(params.c == null ? false : (params.c > 1))"}}}}}}}

Wrap the same aggregate in a function and the filter disappears:

SELECT status, COUNT(*) AS c FROM t GROUP BY status HAVING NULLIF(COUNT(*), 0) > 1
{"aggs":{"status":{"terms":{"field":"status","size":65536,"min_doc_count":1},
  "aggs":{"c":{"value_count":{"field":"_index"}}}}}}

No having_filter, no bucket_selector — the HAVING is simply gone.

statement (all SELECT status, COUNT(*) AS c FROM t GROUP BY status …) bucket_selector
HAVING COUNT(*) > 1 ✅ emitted
HAVING NULLIF(COUNT(*), 0) > 1 ❌ dropped
HAVING COALESCE(COUNT(*), 0) > 1 ❌ dropped
HAVING ABS(COUNT(*)) > 1 ❌ dropped
HAVING NULLIF(c, 0) > 1 (the SELECT alias) ❌ dropped
HAVING COUNT(*) + 1 > 2 parse rejection — loud, a different gap

So it is not NULLIF-specific: it is any function over an aggregate, including one reached
through its SELECT alias.

Cause

ElasticAggregation.metricSelectorForBucket (bridge/.../ElasticAggregation.scala:717-766) renders
the HAVING criteria to a Painless script and then:

if (fullScript.isEmpty) {
  return ""            // <- "" is read by the caller as "no filter needed"
}

An aggregate rendered inside a function produces an empty fragment (the same empty-rendering the
NULLIF(COUNT(*), 0) projection shows: def param1 = == 0 ? null : ; param1), so fullScript is
empty and the method returns "". The caller cannot distinguish that from "this predicate needs no
selector" and omits the aggregation entirely.

The same conflation appears a second time, twelve lines below: conditions whose metrics do not
resolve are filtered out, and an empty relevantConditions again returns "".

⇒ "" means two different things — "nothing to filter" and "I cannot express this filter" — and
only the first is safe.

Impact

The query returns MORE groups than the SQL asks for, silently. HAVING COALESCE(COUNT(*), 0) > 1 is
an ordinary shape for a BI tool to generate, and HAVING NULLIF(c, 0) > 1 reaches it through the
alias a user naturally writes.

Suggested direction

An unrepresentable HAVING must FAIL LOUDLY rather than vanish (feedback_nonsense_input_fails_loudly):
distinguish "empty because there is nothing to filter" from "empty because the predicate could not be
rendered", and reject the statement naming the clause. Emitting a correct selector for a function over
an aggregate would be better still, but the loudness is the part that stops a wrong answer.

Evidence level

The generated query is measured, on clean main, for all six shapes above. It was NOT executed
against a cluster to observe the extra groups arriving — the absent bucket_selector is conclusive
about what is sent, and the control shows what a working filter looks like in the same position.

Found 2026-09-23 while reviewing #382 (which does not touch this path: the emissions are identical
before and after that branch).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions