Repository navigation
explore (errors dataset): duplicate rows and a non-reproducible total while hasMore is false #1519
Description
Summary
Multi-dimension sentry explore --dataset errors returns duplicate rows, and the sum of count() over the returned rows does not reconcile with the scalar count() for the same filter. Repeating the identical command produces a different set of duplicates and a different sum each time, while every response reports hasMore: false.
Environment
sentry 0.40.0
node v24.18.0
Windows 10.0.26200
Steps to reproduce
# ground truth for the filter
sentry api "/organizations/<org>/events/?field=count()&query=release%3A%22<release>%22\
&statsPeriod=14d&project=<project-id>&dataset=errors" --json
# -> count() = 7283
# same filter, grouped by two tags — run this three times
sentry explore <org>/<project> --dataset errors --period 14d \
--query 'release:"<release>"' \
--field 'task.name' --field 'failure.node' --field 'count()' \
--limit 1000 --fresh --jsonActual
Three consecutive identical invocations (ground truth for the filter: 7283 events):
| run | rows returned | distinct task.name|failure.node keys |
sum of count() |
hasMore |
|---|---|---|---|---|
| 1 | 351 | 264 | 7721 | false |
| 2 | 351 | 280 | 6809 | false |
| 3 | 351 | 267 | 7006 | false |
So the same query yields 71–87 duplicated rows, and a total that lands anywhere from ~7% below to ~6% above the true total. Row count is stable at 351 while the content varies, which looks like a page-merge / cursor overlap rather than a server-side sampling difference.
Adding more --field dimensions makes the divergence larger (a 5-field variant of the same query summed to 6101 against a true total of 7052).
Verified with the shim bypassed (node .../sentry/dist/bin.cjs explore ...), so it is not a shell/argument artifact.
Expected
- No duplicate rows for the same dimension tuple.
- The sum over returned rows reconciles with the scalar
count()for the same filter, orhasMore: trueindicates the result is partial. - Repeated identical queries return the same aggregate.
Impact
Aggregates are the primary way to answer "what is broken in this release". Because the duplicates and gaps are nondeterministic, they fabricate findings that look exactly like real signal:
- rows present in release B and absent in release A read as "new in this release" — I hit six such false positives in one investigation, all disproved by re-querying each node individually;
- the same tuple appearing twice inflates a task or node that is actually flat.
There is no in-band way to detect this: hasMore says the result is complete.
Workaround
Deduplicate client-side, reconcile the sum against a scalar count(), and re-query every row before citing it — which defeats the purpose of the aggregate.
Activity
Windsland52 commentedon Sep 1, 2026
Related: getsentry/toolkit#1477 — even with correct, reconciling aggregates, the
errorsdataset cannot be ordered throughexplore, so ranking still requires pulling the full result set. The two together are what make release triage onerrorsunreliable.BYK commentedon Sep 1, 2026
Jared, make sure this is not an API-side issue first
jared-outpost commentedon Sep 1, 2026
traced this end to end — the nondeterministic duplicates/gaps are API-side, and the CLI turns them into a result that looks complete. quick writeup so we can decide where the fix belongs.
mechanism
for
sentry explore --dataset errors --limit 1000with group-by fields:limit (1000) > API_MAX_PER_PAGE (100), soqueryEventsauto-paginates — it follows thecursorfrom eachLinkheader and concatenatesdataacross pages until a page returnsresults="false".errorsdataset the CLI sends nosort.resolveDatasetConfigonly forwards a sort forspans; everything else issort: undefined(the test "omits sort for non-spans dataset" pins this). it even computes-count()and then throws it away./organizations/{org}/events/cursor for grouped aggregates is offset-based (cursor="0:100:0"=value:offset:is_prev). offset paging is only correct if the server returns the grouped rows in a stable total order across the separate page requests. with noORDER BY, the group ordering isn't guaranteed stable between independent executions, so page 2's offset overlaps and skips vs page 1 — which is exactly the reported signature: 264–280 distinct of 351 rows, a sum drifting either side of the scalarcount(), different content each run, row count stable.hasMoreis taken from the last page's Link only. the terminal page isresults="false", so the CLI reportshasMore: falseeven though the merged aggregate is internally inconsistent.so: the instability is generated server-side (unstable ordering of grouped-events cursor pagination); the CLI amplifies it by silently merging pages and asserting completeness.
the fix hinges on one API question
the clean CLI fix is to send a deterministic
sort(e.g.-count()/ first aggregate) onerrors/discoverso the cursor is stable and the whole bug disappears. but the CLI currently forbids sort on non-spans datasets because/events/returned400for it in the past — see the matching comments inapi/dashboards.tsandcommands/explore.tsand the existing test. i can't hit a live org from the sandbox to confirm.@BYK — can you confirm whether
/organizations/{org}/events/?dataset=errors&field=count()&sort=-count()is accepted (no 400) on a current SaaS org?hasMoreto reflect the auto-paginated tail). fully CLI-side.sentry(stable grouped-events pagination). the CLI change then shrinks to not lying: stop silently merging + reportinghasMore: falsefor multi-page grouped aggregate queries (cap to one page / surfacehasMorehonestly / dedupe by tuple with a reliability warning).lean is (A) if the endpoint takes the sort. let me know and i'll ship it.
Windsland52 commentedon Sep 2, 2026
Answering @BYK's question with a live-org probe (SaaS,
dataset=errorsgrouped aggregate, fixed filter, 14d window, per_page=12,field=failure.node+field=count()):sort=-count()(run 1)ReturnMain=1828,(empty)=1763,TargetCountStartReplay=584,controller_initialization_failed=508,HomeFlag=357, ... — strictly descendingsort=-count()(run 2)sort=count()(run 1)SwitchLastAccountComplete=1,SOSBuyItems=1,LimboStarsMainFlag=1,CollectMailAwards=1, ... — strictly ascendingsort=count()(run 2)sort(run 1)(empty)=1763,AwardsMainFlag=68,Sub_MailBadge=2,AutoSelectLeast=2,TranceEnd=2, ... — a third, non-sorted ordersort(run 2)The endpoint accepts
sorton groupederrorsaggregates — no 400 — and honors it:-count()andcount()each produce a strictly monotone order that repeats identically, i.e. the stable total order the offset-based cursor needs for correct pagination. That is the precondition for your path A.Both probes above ran against SaaS via
sentry api(CLI 0.40.0), so the API-side behavior this issue was waiting on is confirmed from a real org.Windsland52 commentedon Sep 2, 2026
Reproduced against a fresh install of
sentry0.44.1 (npm latest as of today), same SaaS org, same filter,--limit 1000, group-bytask.name+failure.node:Scalar
count()for the same filter: 7283. So the nondeterministic duplicates/gaps are all present on 0.44.1, and--sort '-count()'still emits the sameIgnoring sort for errorswarning.Separately, the API block was probed with
sentry api(CLI 0.40.0):/organizations/{org}/events/?dataset=errors&field=failure.node&field=count()&sort=-count()&...is accepted (no 400) and deterministic — two identical descending results;sort=count()gives the ascending mirror; no-sort gives a third, unsorted order. If useful, happy to post the full table.jared-outpost commentedon Sep 2, 2026
thanks for the live-org probe — that confirmed path A. opened #1520.
the fix threads a deterministic sort (
-<firstAggregate>, e.g.-count()) through for theerrors/discoverdatasets instead of dropping it. with a stable total order the offset-based cursor no longer overlaps/skips between pages, so the merged multi-page aggregate is deterministic andhasMorereflects reality.metrics/logsstill 400 on sort, so they stay unsorted, and theIgnoring sortwarning now only fires when you explicitly pass--sorton one of those.draft PR while CI runs: #1520
docs(skill): drop the fixed sentry CLI caveats and the version anchor
fix(cli): Sort every Events dataset through a shared resolver (#1521)