Skip to content

PromQL: histogram_quantile classic-bucket vs sketch-able quantile is a structural heuristic — drive it from sample type/metadata #79

Description

@zzylol

Context

histogram_quantile(φ, <arg>) has two lowerings (added alongside #43):

  • classic cumulative-bucket form → AggIntent::HistogramQuantile { q } — exact interpolation over le buckets; not sketch-able.
  • native-histogram / raw-samples form → generic AggIntent::Quantile { q, accuracy } — sketch-able (L4 picks KLL/DDSketch to hit the accuracy target).

The distinction matters operationally: pre-aggregated bucket counts can't be re-sketched (you can't reconstruct the distribution from cumulative counts), while raw samples / native histograms can be, with an accuracy target.

The gap: the discriminator is a structural heuristic

The true signal — the argument's sample type (classic histogram vs native histogram vs raw float samples) — is not visible at lowering time. is_classic_bucket_arg (crates/frontend-promql/src/promql.rs) currently proxies it by recognising, in the argument tree, any of:

  • a by (le) grouping (sum by (le) (…)),
  • a selector on a _bucket metric (bare name or __name__),
  • an le label matcher ({le="…"}).

Everything else → generic (sketch-able) Quantile.

This is right for the common cases but has failure modes:

  • False positive — a metric merely named …_bucket that isn't a classic histogram routes to HistogramQuantile (interpolation) instead of the sketch path. (Low impact: histogram_quantile over a non-histogram is meaningless in stock PromQL anyway.)
  • False negative — a classic histogram exposed without the _bucket suffix and queried without le grouping/matcher routes to the sketch path — which would be wrong (can't sketch buckets).
  • Native histogram named …_bucket (unusual) would misroute to interpolation.
  • Relies on the raw-samples extension (histogram_quantile(φ, <raw metric>) → sketch) being distinguishable only by the absence of bucket signals.

Proposed direction

Drive the choice from type/metadata, not query structure, once available:

  • a schema/catalog that tags a series (or metric) as classic-histogram / native-histogram / raw, consulted during lowering (or deferred to an L4 rule that has the sample-type), or
  • an explicit annotation on the workload / query (e.g. the client declares it holds raw samples it can sketch).

Until then the structural heuristic stands; this issue tracks replacing it with a type-driven decision and pinning the edge cases with tests.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions