[fix](nereids) fix limit + offset overflow when pushing down TopN/Limit - #64633
Merged
924060929 merged 1 commit intoJul 2, 2026
Merged
Conversation
924060929
requested review from
englefly,
morrySnow and
starocean999
as code owners
June 18, 2026 08:22
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Requesting changes because the overflow fix is incomplete.
Critical checkpoint conclusions:
- Goal/test proof: The PR adds a saturating helper and applies it to several rewrite rules, but it does not accomplish the stated goal of clamping
limit + offseteverywhere a pushed child/scan/agg limit is derived. A plain TopN can still become a negative local physical TopN inLogicalTopNToPhysicalTopN, and several Limit/TopN parallel paths still use raw addition. The tests cover the helper and one Union reproducer only, so they do not prove the end-to-end fix. - Scope/focus: The helper is small and clear, but the applied scope is inconsistent with the PR description and leaves equivalent code paths unfixed.
- Concurrency/lifecycle/config/compatibility: No new concurrency, lifecycle, config, persistence, FE-BE protocol, or rolling-upgrade compatibility concerns found.
- Parallel code paths: Not fully handled. Remaining raw arithmetic exists in TopN implementation/post-processing, Limit distinct pushdown, Limit window pushdown, TopN-to-agg, scan sort/vector/score pushdowns, and translator sort-limit pushdown.
- Special checks: Threshold checks that compare against raw
limit + offsetcan still flip from huge positive to negative and enable optimizations that should be disabled. - Test coverage/results:
UtilsTestis useful, but the regression case uses direct Groovy assertions for a deterministic result and does not update the.outfile, which violates the local regression-test standard. Additional regression coverage is needed for a forced two-phase TopN and at least one Limit path. - Observability/performance/data correctness: No extra observability appears necessary. The intended clamp is semantically safe for non-negative limits and avoids the BE timeout behavior, but only once every equivalent derived-limit path is fixed.
- User focus: No additional user-provided focus points were present.
| * no relation can hold more than {@code Long.MAX_VALUE} rows, clamping to {@code Long.MAX_VALUE} | ||
| * ("all rows") is the semantically correct upper bound and never drops rows the parent may need. | ||
| */ | ||
| public static long saturatedAdd(long a, long b) { |
Contributor
There was a problem hiding this comment.
This helper is the right primitive, but the PR does not route all equivalent limit + offset row-count derivations through it. A plain TopN still follows this path:
LogicalTopN(limit=Long.MAX_VALUE, offset=Long.MAX_VALUE, order by id)
Scan/Union
LogicalTopNToPhysicalTopN.twoPhaseSort then builds a local child with logicalTopN.getLimit() + logicalTopN.getOffset() in fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalTopNToPhysicalTopN.java:50, so the physical tree can still contain:
PhysicalTopN(MERGE_SORT limit=MAX offset=MAX)
PhysicalTopN(LOCAL_SORT limit=-2 offset=0)
With SET sort_phase_num = 2 this is not just an unused alternative. The same raw computation also remains in PushDownLimit.java, PushDownLimitDistinctThroughJoin.java, PushDownLimitDistinctThroughUnion.java, LimitAggToTopNAgg.java, PushTopnToAgg.java, the score/vector TopN scan pushdowns, and PhysicalPlanTranslator#setSortLimit. Please replace these parallel derived child/scan/agg limits, or centralize the operation on Limit/TopN, otherwise the same negative limit can still reach BE outside the specific Union rewrite covered here.
| ) s; | ||
| """ | ||
| assertEquals(1, topnUnionOverflowRes.size()) | ||
| assertEquals(0L, topnUnionOverflowRes[0][0]) |
Contributor
There was a problem hiding this comment.
This deterministic result should be recorded with a qt_... query and the generated .out file, not Groovy assertions. The local regression-test rules require determined expected results to be generated through qt_sql/order_qt style output files; here count(*) always returns one row with 0, but that expected result is invisible in regression-test/data/nereids_rules_p0/push_down_top_n/push_down_top_n_through_union.out. Please convert this to a qt_topn_union_overflow block and regenerate the .out file.
morrySnow
marked this pull request as draft
June 18, 2026 08:36
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
TPC-H: Total hot run time: 29205 ms |
Contributor
TPC-DS: Total hot run time: 175320 ms |
Contributor
ClickBench: Total hot run time: 25.76 s |
924060929
force-pushed
the
fix-push-down-topn-union-overflow
branch
from
June 22, 2026 02:55
e9fd2a7 to
9407729
Compare
924060929
changed the title
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
Contributor
There was a problem hiding this comment.
Automated review summary:
The PR is aimed at fixing limit + offset overflow for TopN/Limit pushdown. The helper itself is correct, but the current patch is not complete: the project-over-join TopN rewrite variants still bypass the existing topn_opt_limit_threshold gate and can push a saturated Long.MAX_VALUE child TopN under joins. Existing review threads also already cover the remaining raw limit + offset production paths and the regression assertion style issue.
Critical checkpoint conclusions:
- Goal/test proof: not fully satisfied; helper arithmetic is tested, but most changed rule paths are not covered.
- Scope/focus: the change is focused, but the fix is inconsistent across parallel rule shapes.
- Concurrency/lifecycle/persistence/protocol: not applicable; this is planner rewrite logic with no new locks, lifecycle, storage format, or FE-BE protocol state.
- Configuration: existing
topn_opt_limit_thresholdremains relevant, but two project-over-join paths bypass it. - Parallel paths: issues remain; see inline comments and the existing raw-addition thread.
- Test coverage/results: insufficient; the new deterministic regression also needs generated
.outcoverage per the existing thread.
Subagent conclusions: optimizer candidates OR-1 and OR-2 became inline comments; OR-3 was suppressed as a duplicate of the existing Utils.java:143 thread. Test candidate TS-2 became an inline comment; TS-1 and TS-3 were suppressed as duplicates of existing review threads. Convergence round 1 ended with both live subagents replying NO_NEW_VALUABLE_FINDINGS for the same ledger/comment set.
| List<Slot> childRequired = requiredOutputSlots.stream() | ||
| .filter(childOutputSet::contains) | ||
| .collect(Collectors.toList()); | ||
| if (childRequired.isEmpty()) { |
Contributor
There was a problem hiding this comment.
The direct TopN -> Join pattern now uses topnOptLimitThreshold >= Utils.saturatedAdd(...) before it pushes, but the sibling TopN -> Project -> Join pattern has no equivalent guard and still reaches this helper. For a reachable shape like TopN(limit=MAX, offset=MAX, order by l.k) -> Project(l.k, ...) -> LeftOuterJoin, the project branch calls pushLimitThroughJoin and this changed line inserts a child LogicalTopN(limit=Long.MAX_VALUE, offset=0) under the join. With the default topn_opt_limit_threshold=1024, the same plan without the project would be rejected by the direct branch. Please apply the same threshold/context predicate to the project-over-join branch, or centralize the saturated-limit refusal inside pushLimitThroughJoin before constructing child TopNs.
| List<OrderKey> pushedOrderKeys = getPushedOrderKeys(groupBySlots, | ||
| join.left().getOutputSet(), topN.getOrderKeys()); | ||
| if (!pushedOrderKeys.isEmpty()) { | ||
| LogicalTopN<Plan> left = topN.withLimitOrderKeyAndChild( |
Contributor
There was a problem hiding this comment.
The direct distinct join pattern is gated by topnOptLimitThreshold >= Utils.saturatedAdd(...), but the TopN -> Aggregate(distinct) -> Project(allSlots) -> Join pattern above it has no equivalent guard and still reaches this changed child TopN construction. With limit=MAX and offset=MAX, the direct branch is disabled under the default finite threshold, while this project branch can push LogicalTopN(limit=Long.MAX_VALUE, offset=0) plus PlanUtils.distinct(child) under the join. Please add the same threshold/context predicate to the project branch, or check it once inside pushTopNThroughJoin before constructing child TopNs.
| } | ||
|
|
||
| @Test | ||
| public void testSaturatedAdd() { |
Contributor
There was a problem hiding this comment.
This unit test proves the arithmetic helper, but it does not prove that the overflow fix is wired through the changed planner rules. The PR changes SplitLimit, both TopN-through-join rules, distinct-union, and window pushdown, while the only new SQL overflow case covers plain UNION ALL and is already missing generated .out coverage in the existing regression thread. Please add generated-output regressions or focused planner tests for the overflow boundary through at least split limit and a session-gated join/project-join path; otherwise the rule wiring and threshold behavior can regress while this helper test still passes.
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
TPC-H: Total hot run time: 29117 ms |
Contributor
TPC-DS: Total hot run time: 173359 ms |
Contributor
ClickBench: Total hot run time: 25.33 s |
924060929
force-pushed
the
fix-push-down-topn-union-overflow
branch
from
June 22, 2026 09:48
9407729 to
996d1a7
Compare
924060929
changed the title
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
924060929
force-pushed
the
fix-push-down-topn-union-overflow
branch
from
June 22, 2026 10:01
996d1a7 to
5b351d3
Compare
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
924060929
force-pushed
the
fix-push-down-topn-union-overflow
branch
from
June 22, 2026 10:11
5b351d3 to
54a2b74
Compare
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
github-actions
Bot
removed
the
approved
Contributor
There was a problem hiding this comment.
Automated review completed for PR 64633.
I reviewed the authoritative PR file list and the related Nereids limit/topN rewrite paths. I am not adding new inline comments because every substantiated suspicious point I found is already covered by existing review threads:
- SplitLimit / LogicalTopNToPhysicalTopN still turn overflowing MAX LIMIT/OFFSET shapes into user-visible errors: already covered by r3466290098, r3466290104, r3479484089, and r3479484084.
- MergeLimits / MergeTopNs nested offset overflow behavior: already covered by r3465721367, r3465800781, and r3465800793.
- Project-over-join branches that bypass the direct branch topn_opt_limit_threshold / ConnectContext gate: already covered by r3449585806, r3449585809, r3465721375, and r3465800790.
- Test coverage gaps around generated outputs and rule-wiring coverage: already covered by r3434307880 and r3449585811.
Critical checkpoint conclusions:
- Goal: The PR adds overflow detection for limit + offset and wires it through the changed TopN/Limit pushdown paths. The main remaining semantic concerns are already present in existing review threads.
- Scope/focus: The patch is narrowly focused on Nereids limit/topN overflow handling.
- Concurrency/lifecycle: No new concurrency or lifecycle-sensitive state was introduced in the reviewed FE rewrite paths.
- Config/session behavior: No new config was added. Existing session-threshold propagation gaps in project-over-join paths are already threaded.
- Compatibility: No FE-BE protocol or storage format change was introduced; the change prevents negative pushed limits from reaching legacy plan nodes.
- Parallel paths: Union, join, distinct, window, scan, aggregate, CTE consumer, physical translator, and constructor invariant paths were checked. No new non-duplicate parallel-path issue was found.
- Tests: The added helper test and regression error cases exist, but broader rule-wiring/generated-output concerns are already covered by existing comments.
- Observability, transaction/persistence, and data-write paths: Not applicable to this FE planner-only change.
- Performance: The guarded paths either skip an optimization or fail before constructing invalid negative limits; known threshold-related pushdown concerns are already threaded.
Subagent conclusions:
- optimizer-rewrite: no new valuable findings after duplicate suppression; candidate areas overlapped existing threads.
- tests-session-config: no new valuable findings after duplicate suppression; remaining test/config concerns overlapped existing threads.
- Final convergence round 1: both live subagents replied NO_NEW_VALUABLE_FINDINGS for the same current ledger and empty proposed inline comment set.
Validation:
git diff --check 32a2651f66b9f0f341c90e95f8ed304b34927111 88eadf46e1f3583ac1a0baf7ab5e3ae2d92ae654 -- $(cat .code-review.tHgXba/pr_changed_files.txt)passed.- FE build/tests/regression tests were not run because
thirdparty/installed/bin/protocis absent in this runner.
User focus:
- No additional user-provided focus points were supplied.
Contributor
Author
|
run buildall |
Contributor
There was a problem hiding this comment.
Automated review completed for PR 64633 at live head 88eadf46e1f3583ac1a0baf7ab5e3ae2d92ae654.
I reviewed the authoritative 24-file PR scope and related Nereids limit/topN rewrite paths. I am not adding inline comments because every substantiated suspicious point I found is already covered by existing review threads:
SplitLimit/LogicalTopNToPhysicalTopNstill turn overflowing MAXLIMIT/OFFSETshapes into user-visible errors: already covered by r3466290098, r3466290104, r3479484089, and r3479484084.MergeLimits/MergeTopNsnested offset overflow behavior: already covered by r3465721367, r3465800781, and r3465800793.- Project-over-join branches that bypass the direct branch
topn_opt_limit_threshold/ConnectContextgate: already covered by r3449585806, r3449585809, r3465721375, and r3465800790. - Test coverage gaps around generated outputs, semantic error expectations, and rule-wiring/session coverage: already covered by r3434307880, r3449585811, r3466290098, and r3466290104.
Critical checkpoint conclusions:
- Goal/test proof: The PR adds
limit + offsetoverflow checks and wires them through many changed TopN/Limit paths. Remaining semantic and test concerns are already in existing threads. - Scope/focus: The reviewed live patch is limited to FE Nereids overflow handling and the associated regression/helper tests.
- Concurrency/lifecycle: No new concurrency or lifecycle-sensitive state was introduced.
- Config/session behavior: No new config was added. Existing session-threshold gaps are already threaded.
- Compatibility: No FE-BE protocol or storage format change was introduced.
- Parallel paths: Union, join, distinct, window, scan, aggregate, CTE consumer, physical translator, and constructor invariant paths were checked. No new non-duplicate parallel-path issue was found.
- Tests: Added helper and regression error cases exist, but broader rule-wiring/generated-output concerns are already covered by existing comments.
- Observability, transaction/persistence, and data-write paths: Not applicable to this FE planner-only change.
- Performance: Guarded paths skip optimizations or fail before constructing invalid negative limits; known threshold-related pushdown concerns are already threaded.
Subagent conclusions:
optimizer-rewrite:NO_NEW_VALUABLE_FINDINGSafter duplicate suppression.tests-session-config: live candidates were duplicates or stale-bundle evidence; final convergence returnedNO_NEW_VALUABLE_FINDINGS.- Final convergence round 1 ended with both live subagents reporting
NO_NEW_VALUABLE_FINDINGSfor the same ledger and empty proposed inline comment set.
Validation:
git diff --check c02808aae79b29e8c02e48f06baf89b43d4c4152..88eadf46e1f3583ac1a0baf7ab5e3ae2d92ae654 -- $(cat .code-review.cDnlMI/pr_changed_files.txt)passed.- FE build/tests/regression tests were not run because
thirdparty/installed/bin/protocis absent in this runner.
User focus:
- No additional user-provided focus points were supplied.
Contributor
|
Codex automated review failed and did not complete. Error: Codex completed, but no new pull request review was submitted for the current head SHA. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
Contributor
TPC-H: Total hot run time: 29898 ms |
Contributor
TPC-DS: Total hot run time: 174241 ms |
Contributor
ClickBench: Total hot run time: 25.29 s |
When combining a limit and an offset into the number of rows a child must keep, the raw limit + offset overflows the long range when both are close to BIGINT_MAX and wraps to a negative value, producing an illegal plan that hangs in BE. Add Utils.addOverflows and, on overflow, decline the optimization and fall back to correct unoptimized execution. For paths that cannot decline (twoPhaseSort, SplitLimit, MergeLimits, MergeTopNs), report an error since the overflowing limit + offset cannot be correctly represented anywhere in the FE-BE pipeline. As defense-in-depth, the LogicalTopN / PhysicalTopN / LogicalLimit / PhysicalLimit constructors assert that limit and offset are non-negative.
924060929
force-pushed
the
fix-push-down-topn-union-overflow
branch
from
July 1, 2026 06:59
88eadf4 to
bb2f83c
Compare
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
Contributor
There was a problem hiding this comment.
Automated follow-up review (2026-07-01 UTC)
Review opinion: I found no new non-duplicate inline comments in this pass. The existing review threads remain the relevant unresolved context for the overflow-to-error semantics, nested merge overflow handling, and project-over-join threshold gates; I am not re-posting those issues here.
Critical checkpoint conclusions:
- Goal and tests: The PR aims to prevent
limit + offsetoverflow from producing negative child limits. The helper and many pushdown paths are wired, with unit/regression coverage added; the semantic concern that some valid empty-result queries now error is already covered by existing threads. - Scope: FE/Nereids-only changes, scoped to TopN/Limit overflow handling and related test coverage.
- Concurrency, lifecycle, persistence, compatibility: No new concurrency, special lifecycle, edit-log, storage-format, or FE-BE thrift compatibility surface found in this diff.
- Configuration/session behavior: No new config items. Existing
sort_phase_numandtopn_opt_limit_thresholdbehavior was reviewed; remaining threshold mismatch concerns are already covered by existing project-over-join threads. - Parallel paths: Checked union, join, distinct join/union, window, scan pushdown, aggregate pushdown, physical translation, constructor invariants, and remaining raw
limit + offsetsites. No distinct new issue was substantiated beyond existing threads. - Tests:
UtilsTest#testAddOverflowscovers the arithmetic helper. The added regression blocks aretest { exception ... }, so generated.outoutput is not required for those blocks; the concern that they encode the error semantics is already raised in existing review context. - Validation:
git diff --check 9d93a33fa1a95db77348af7812d65f02e894fde9..bb2f83ceafb3e9c3487c5934735179f69d33d142 --passed. I did not run FE or regression tests becausethirdparty/installedandthirdparty/installed/bin/protocare missing in this runner.
User focus: No additional user-provided review focus was supplied.
Subagent conclusions:
optimizer-rewrite: recorded duplicate notes for the TopN implementation throw, SplitLimit throw, nested limit/topn merge throws, and project-over-join threshold bypasses. Convergence round 1 returnedNO_NEW_VALUABLE_FINDINGSfor the current ledger and empty proposed inline set.tests-session-config: recorded one duplicate note for the overflow regression tests expectinglimit + offset overflows. Convergence round 1 returnedNO_NEW_VALUABLE_FINDINGSfor the current ledger and empty proposed inline set.
Contributor
TPC-H: Total hot run time: 30009 ms |
Contributor
TPC-DS: Total hot run time: 173783 ms |
Contributor
ClickBench: Total hot run time: 25.36 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
FE UT Coverage ReportIncrement line coverage |
morrySnow
approved these changes
Jul 2, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
924060929
changed the title
16 tasks
16 tasks
airborne12
added a commit
that referenced
this pull request
Sep 1, 2026
…5821 (#67327) ### What problem does this PR solve? Issue Number: N/A Related PR: #65821 (master), picked from commit 2b6a45e Problem Summary: Backport of #65821 to branch-4.1. Search score TopN pushdown may return incorrect results when the search predicate is combined with additional predicates, because the pushed TopN limit can be applied before the remaining predicates are evaluated. This change disables the pushed search TopN limit in those cases while preserving the virtual score column pushdown, and adds regression coverage (search + equality / range / match / score range / multiple search predicates, plus limit+offset overflow). **Hunk audit (source diff → this PR):** | Source hunk | Status | |---|---| | `PushDownScoreTopNIntoOlapScan.java` `@@ -194,17 +194,22 @@` (overflow guard rework + pushedScoreLimit) | **Adapted ×2**: ① branch-4.1 never had the #64633 overflow-guard block, so the hunk's removed lines have no counterpart here; ② `Utils.addOverflows` does not exist on 4.1 (#64633 not backported) — inlined the equivalent check `topN.getLimit() > Long.MAX_VALUE - topN.getOffset()` (identical to the master helper's implementation). | | `PushDownScoreTopNIntoOlapScan.java` `@@ -243,6 +248,19 @@` (`shouldDisableSearchTopN` helper) | Ported | | `test_search_score_topn_predicates.out` (new) | Ported (verbatim) | | `test_search_score_topn_predicates.groovy` (new) | **Adapted**: dropped `set enable_segment_limit_pushdown = true` — the variable comes from #62222 which is not on 4.1; it defaults to true on master and only controls a BE-side segment limit optimization, unrelated to this FE-plan-level fix. | **Local verification on this branch:** full ASAN BE+FE build green; `run-regression-test.sh -d inverted_index_p0 -s test_search_score_topn_predicates` → 1 suite, 0 failed against a local 1FE+1BE cluster built from this PR. No FE UT exists for this rule on 4.1 and the source PR added none (its coverage is the regression suite above). ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [x] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> Co-authored-by: liangj777 <106017102+LIANG751234313@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
When combining a
limitand anoffsetinto the number of rows a child must keep — pushing aTopN/Limitdown, building a two-phase sort, splitting a limit into local + global phases, etc. —the raw
limit + offsetoverflows thelongrange when both are close toBIGINT_MAX(e.g.
LIMIT 9223372036854775807 OFFSET 9223372036854775807) and wraps to a negative value.A negative limit is an illegal plan. On the BE side it is reinterpreted as a huge unsigned value
(
uint64_t limit = _offset + _limitin the sorter /_heap_size(limit + offset)in HeapSorter),so a trivial query that should immediately return an empty set instead runs until it hits the query
timeout, or triggers UBSan (signed integer overflow is undefined behavior in C++).
Minimal reproducer (no table required)
PUSH_DOWN_TOP_N_THROUGH_UNIONdisabled: returns0immediately.Fix
Add
Utils.addOverflows(long, long)to detect whenlimit + offsetexceeds thelongrange.On overflow, the behavior depends on whether the code path can be declined:
Skip the optimization (can decline — the unoptimized plan is correct):
TopN/Limitpush-down throughUnion/Join/Project-Join/Windowand their distinctvariants skip the rewrite; the parent
TopN/Limitstill applies the originallimit/offset.topn_opt_limit_thresholdgates (PushDownTopNThroughJoin/PushDownTopNDistinctThroughJoin/PushDownLimitDistinctThroughJoin/LimitAggToTopNAgg/PushTopnToAgg) treat overflow asover-threshold and do not fire.
TopN-into-OlapScanpush-downs and the sort-limit-into-OlapScantranslationin
PhysicalPlanTranslatorare skipped.CollectLimitAboveConsumerskips recording the consumer row count on overflow, leaving the CTEproducer unbounded (the consumer's own limit still applies).
Report an error (cannot decline — overflowing
limit + offsetcannot be correctly represented):LogicalTopNToPhysicalTopN: the two-phase local sort needslimit + offsetrows, and even asingle-phase
GATHER_SORTwould pass the overflowing values to BE whereHeapSorteralso computeslimit + offset(triggering UBSan). Since no valid physical plan can be produced, report an error.SplitLimit: the two-phase local limit needslimit + offset; an unsplitORIGINlimit is silentlydropped during translation, and
Long.MAX_VALUEis not treated as "unlimited" by the legacyPlanNodelayer (hasLimit()returnslimit > -1). Report an error.MergeLimits.mergeOffset/MergeTopNs: consecutive limits/TopNs must be merged, and the mergedoffset (
upperOffset + bottomOffset) cannot be represented on overflow. Report an error.For non-overflowing inputs the behavior is unchanged.
Defense-in-depth: the
LogicalTopN/PhysicalTopN/LogicalLimit/PhysicalLimitconstructors now assert that
limitandoffsetare non-negative. Any future path that reintroducesan overflowing
limit + offsetfails fast during planning instead of silently hanging in BE.Tests
UtilsTest#testAddOverflowscovers the overflow detection boundary.push_down_top_n_through_unionassert that the reproducer (both TopN and Limitpaths with BIGINT_MAX limit/offset) reports an error instead of timing out.