Skip to content

[fix](subquery) Resolve correlated qualified columns before dereference - #67438

Open
CalvinKirs wants to merge 2 commits into
apache:masterfrom
CalvinKirs:fix-correlated-dereference-binding
Open

[fix](subquery) Resolve correlated qualified columns before dereference#67438
CalvinKirs wants to merge 2 commits into
apache:masterfrom
CalvinKirs:fix-correlated-dereference-binding

Conversation

@CalvinKirs

@CalvinKirs CalvinKirs commented Sep 2, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

In a correlated subquery, a multipart reference can represent either a relation-qualified column (table_alias.column) or a nested-field dereference (column.field). Nereids previously searched the inner scope completely before checking the outer scope. If an inner table had a physical column with the same name as an outer table alias, the outer reference could therefore be interpreted as a nested field of that inner column.

For scalar inner columns this raised a No such field analysis error. For complex inner columns it could bind successfully to the wrong expression and produce incorrect results.

Example

CREATE TABLE outer_events (
    id INT,
    `@event_name` VARCHAR(32)
)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES ("replication_num" = "1");

CREATE TABLE inner_events (
    id INT,
    t1 INT
)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES ("replication_num" = "1");

INSERT INTO outer_events VALUES (1, 'blocked'), (2, 'kept');
INSERT INTO inner_events VALUES (1, 0);

SELECT t1.id, t1.`@event_name`
FROM outer_events t1
WHERE NOT EXISTS (
    SELECT 1
    FROM inner_events inner_alias
    WHERE t1.`@event_name` = 'blocked'
)
ORDER BY t1.id;

Before this fix, the outer alias t1 conflicted with the physical inner column inner_events.t1. Nereids treated ``t1.`@event_name``` as a nested-field access on the inner scalar column and failed during analysis:

No such field '@event_name' in 't1'

After this fix, ``t1.`@event_name``` is correctly bound to the outer relation alias and the query returns:

+------+-------------+
| id   | @event_name |
+------+-------------+
|    2 | kept        |
+------+-------------+

This change resolves relation-qualified columns in the current and outer scopes before falling back to first-part-as-column dereference. The relation-only phase follows each analyzer's complete local scope order before searching the outer scope, so custom HAVING and QUALIFY scopes preserve normal nearest-relation shadowing.

Lambda lexical scope is preserved: in array_map(x -> x.value, x.items), the first x inside the lambda resolves to the lambda argument while the second x resolves to the enclosing table alias. When an outer qualified reference contains nested fields, such as outer_alias.payload.k, the underlying payload slot is also recorded as a correlated slot.

Release note

Fix incorrect Nereids column binding when an outer table alias conflicts with an inner column name in a correlated subquery.

Check List (For Author)

  • Test: Regression test / Unit Test
    • ./run-fe-ut.sh --run org.apache.doris.nereids.rules.analysis.TestDereference
    • Result: 10 tests passed, 0 failures
    • Added query_p0/test_dereference regression coverage for scalar and complex inner columns, lambda lexical scope, outer nested-field correlation, reused inner/outer aliases in HAVING and QUALIFY, and local qualifier shadowing
    • ./run-regression-test.sh --run -d query_p0 -s test_dereference
    • Result: suite passed against the generated snapshot
  • Behavior changed: Yes (relation-qualified correlated references now take priority over inner column-field dereference while preserving complete local-scope precedence)
  • Does this need documentation: No

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@morrySnow morrySnow changed the title [fix](fe) Resolve correlated qualified columns before dereference [fix](subquery) Resolve correlated qualified columns before dereference Sep 2, 2026
@CalvinKirs

Copy link
Copy Markdown
Member Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review of PR #67438 at head 3e6ad0f found two blocking correctness issues, recorded inline below.

Review checkpoints:

  • Goal: relation-qualified correlated references should win over inner column-field dereference. The direct scalar and variant conflicts are implemented and covered by deterministic FE/regression tests, but the new lookup is too broad and breaks lambda lexical scoping and nested outer-field correlation metadata.
  • Data correctness: not satisfied until MF-1 and MF-2 are fixed; both can change query results or produce an invalid uncorrelated Apply.
  • Scope/concurrency/lifecycle: no thread or lifecycle changes; the relevant scope hierarchy is covered by the inline findings.
  • Compatibility/configuration/protocol: no configuration, persistence, or FE-BE protocol changes identified.
  • Error handling/observability: no new ignored Status or RPC boundary; the main concern is silent misbinding.
  • Tests: added tests follow ordering and setup conventions and exercise the intended direct conflicts, but do not cover lambda-argument shadowing or nested outer fields.
  • User focus: review_focus.txt contains no additional focus.

No builds or tests were run because the review prompt explicitly forbids builds/code modifications in this runner.

@CalvinKirs
CalvinKirs force-pushed the fix-correlated-dereference-binding branch from 3e6ad0f to 3a350a0 Compare September 2, 2026 09:07
@CalvinKirs

Copy link
Copy Markdown
Member Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review status: complete after 2 rounds. All round-2 normal and risk-focused reviewers returned NO_NEW_VALUABLE_FINDINGS; one distinct correctness issue remains and is requested inline.

Critical checkpoint conclusions:

  • Goal and data correctness: The ordinary correlated-filter conflict, lambda lexical-precedence case, and nested correlation-slot propagation are addressed, but the goal is incomplete for custom HAVING/QUALIFY binders because an outer relation can preempt a valid inner binding.
  • Scope and clarity: The production change is localized and the relation-only helper is clear, but the generic prepass assumes getScope() represents every analyzer's complete local lookup policy; that assumption is false for custom binders.
  • Concurrency: Not involved; analysis uses per-query scopes and this change adds no shared concurrent state or locking.
  • Lifecycle/static initialization: Not involved; no new static/global lifetime or ownership behavior.
  • Configuration: No configuration item is added or changed.
  • Compatibility: No FE-BE protocol, storage-format, persisted-metadata, symbol, or rolling-upgrade surface changes.
  • Parallel paths: Blocking issue found. HAVING, QUALIFY, and related custom secondary-scope paths are not correctly covered by the new ordering.
  • Special conditions: The lambda-specific shouldPrioritizeRelationQualifier=false guard is justified and fixes the prior lexical-scope thread, but equivalent complete-local-scope handling is missing for custom binders.
  • Tests: The five targeted FE tests are reported passing by the author, and the added unit/regression cases distinguish scalar conflict, complex-column conflict, lambda binding, and nested correlation. The regression suite was not run by the author, and there is no reused-alias HAVING/QUALIFY case for the blocking path. Per the review-runner instruction, this review did not build or modify source.
  • Test results: The added deterministic expectations and existing negative dereference case are consistent with the intended behavior; no incorrect expected result was found.
  • Observability: No new runtime or distributed operational path; additional logs or metrics are not applicable.
  • Transactions/persistence and data writes: Not involved.
  • FE-BE variables: No variable or request-field propagation is added.
  • Performance: The extra bounded scope lookups do not establish a distinct performance defect; no hot runtime path is changed.
  • Other issues: The two existing inline reports (lambda precedence and nested correlation extraction) are verified fixed at head 3a350a08727a9b62b78470b056c71a9a6c5cf981; no duplicate was submitted.
  • User focus: No additional user-provided focus was specified; the full PR was reviewed.

@924060929

Copy link
Copy Markdown
Contributor

The intended precedence is correct: for a multipart name such as a.b, Doris should first preserve the historical relation_alias.column interpretation, and only fall back to column.nested_field after no visible relation qualifier can bind it. The old all-in-one current-scope lookup must therefore change; otherwise an unrelated inner column named a can hide an outer relation alias a, causing either an analysis error or a silent wrong binding.

The remaining issue is not that relation-qualified lookup is prioritized, but that the new prepass assumes getScope() is every analyzer's complete local lookup scope:

bind relation in getScope()
bind relation in outerScope
bindSlotByThisScope()

That assumption does not hold for analyzers such as HAVING/QUALIFY, where bindSlotByThisScope() is overridden to search additional local scopes. In those analyzers, a valid inner binding can be skipped and an outer relation can win prematurely.

I suggest making the relation-only phase follow the same analyzer-specific local lookup policy as the full binding phase, for example with a protected bindSlotByRelationQualifierInThisScope() hook parallel to bindSlotByThisScope(). Custom binders can then override it using the same local scope order. The intended contract would be:

1. explicit lexical/local bindings that must shadow outer names
2. relation-qualified lookup across the analyzer's complete local scope policy
3. relation-qualified lookup in the outer query scope
4. local nested-field fallback using the same complete local scope policy
5. outer nested-field fallback

This keeps the compatibility fix while avoiding special-case regressions caused by bypassing custom local binders. Please also add a reused-alias HAVING/QUALIFY test so the scope contract is fixed by coverage rather than by the current implementation shape.

@CalvinKirs
CalvinKirs force-pushed the fix-correlated-dereference-binding branch from 3a350a0 to 902e8b9 Compare September 2, 2026 09:55
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: In a correlated subquery, a multipart reference could be interpreted as a nested field of an inner column before checking whether its first part was an outer table alias. This caused scalar inner columns to raise a no-such-field error and complex inner columns to bind silently to the wrong expression. Resolve relation-qualified columns across the current and outer scopes before falling back to column-field dereference, while preserving lambda lexical scope precedence, custom analyzer local-scope precedence, and recording the input slots of correlated nested-field expressions.

### Release note

Fix incorrect Nereids column binding when an outer table alias conflicts with an inner column name in a correlated subquery.

### Check List (For Author)

- Test: Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.nereids.rules.analysis.TestDereference
    - Added query_p0 regression coverage for scalar and complex inner columns, lambda lexical scope, nested correlated fields, and reused aliases in HAVING
- Behavior changed: Yes (relation-qualified correlated references now take priority over inner column-field dereference)
- Does this need documentation: No
@CalvinKirs
CalvinKirs force-pushed the fix-correlated-dereference-binding branch from 902e8b9 to 9fdbf32 Compare September 2, 2026 10:06
@CalvinKirs

Copy link
Copy Markdown
Member Author

/review

@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review status: complete after 2 rounds. Both normal reviewers and the separate risk reviewer returned NO_NEW_VALUABLE_FINDINGS; two distinct issues remain, so I am requesting changes.

Critical checkpoint conclusions:

  • Goal and data correctness: the direct correlated-filter conflict, lambda lexical precedence, nested correlation extraction, and custom HAVING/QUALIFY scope ordering are addressed, but qualifier occupancy is still lost when a nearer reused alias lacks the requested column. That can silently turn a local nested-field binding into an outer correlation.
  • Scope and minimality: the production change is localized, and the custom binder callback plumbing is mechanically consistent. The remaining defect is in the relation-only lookup's inability to distinguish an absent qualifier from an occupied qualifier with no matching column.
  • Parallel paths and special conditions: aggregate/generic HAVING, project/aggregate QUALIFY, GROUP BY, ORDER BY, lambda, and join rejection were audited. Their variants either behave consistently or reduce to the first inline finding; no separate issue remains.
  • Concurrency, lifecycle, configuration, compatibility, persistence, transactions, data writes, and FE-BE propagation: not involved in this analyzer-only change.
  • Error behavior: the first finding can replace a local bind/error path with a valid correlated plan, so failure semantics are not preserved.
  • Tests and results: the added unit expectations and deterministic row values are semantically coherent, and the author reports TestDereference passing 7/7. This review did not run builds or tests because the review prompt prohibits them. Live compile and CheckStyle pass; FE UT is pending, and the regression jobs currently report immediate TeamCity failures. Coverage still lacks the qualifier-present/column-absent collision, and the six new success cases do not follow the required generated-snapshot convention.
  • Performance and observability: the extra analyzer lookups do not establish a material performance issue; runtime observability is not applicable.
  • Existing threads: lambda precedence, nested correlation bookkeeping, and the custom HAVING/QUALIFY omission were verified and treated as hard duplicate fences on this head.
  • User focus: no additional review focus was provided, so the full PR was reviewed.

Comment thread regression-test/suites/query_p0/test_dereference.groovy
@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 92.11% (70/76) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16701 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9fdbf32e60c4ed7df3868aa9b63dbb9b4bade8e7, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17581	3051	3062	3051
q2	2144	263	229	229
q3	10193	870	517	517
q4	4668	259	201	201
q5	7660	571	383	383
q6	137	116	95	95
q7	523	495	402	402
q8	9233	935	945	935
q9	3510	2392	2381	2381
q10	6491	867	727	727
q11	389	199	177	177
q12	603	251	207	207
q13	18144	1552	1183	1183
q14	160	161	139	139
q15	q16	445	397	369	369
q17	1361	868	744	744
q18	3625	2225	2231	2225
q19	1151	935	774	774
q20	363	285	197	197
q21	5395	1537	1769	1537
q22	323	263	228	228
Total cold run time: 94099 ms
Total hot run time: 16701 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3393	3353	3315	3315
q2	505	389	362	362
q3	2262	2318	2104	2104
q4	1181	1161	888	888
q5	2170	2091	2101	2091
q6	162	115	89	89
q7	1015	911	861	861
q8	1563	1391	1392	1391
q9	3136	3114	3081	3081
q10	1854	1795	1642	1642
q11	354	272	252	252
q12	447	425	346	346
q13	1477	1525	1173	1173
q14	166	176	167	167
q15	q16	393	404	364	364
q17	3575	3263	3263	3263
q18	4815	4545	4708	4545
q19	863	860	897	860
q20	1018	975	835	835
q21	3846	3174	3168	3168
q22	389	347	317	317
Total cold run time: 34584 ms
Total hot run time: 31114 ms

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 81.93% (68/83) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81807 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9fdbf32e60c4ed7df3868aa9b63dbb9b4bade8e7, data reload: false

query5	4295	424	351	351
query6	396	146	138	138
query7	4900	417	242	242
query8	301	125	121	121
query9	8735	2869	2881	2869
query10	409	230	185	185
query11	5382	1037	901	901
query12	124	69	68	68
query13	1185	462	329	329
query14	6118	2198	2091	2091
query14_1	1956	1944	1948	1944
query15	172	116	108	108
query16	898	367	356	356
query17	776	459	389	389
query18	2345	328	237	237
query19	166	139	111	111
query20	73	73	72	72
query21	199	102	91	91
query22	5468	5369	5323	5323
query23	6762	6077	5866	5866
query23_1	6260	5988	6069	5988
query24	7293	1091	765	765
query24_1	812	756	781	756
query25	436	314	266	266
query26	1238	232	136	136
query27	2774	413	264	264
query28	4661	1491	1504	1491
query29	938	451	356	356
query30	251	159	131	131
query31	831	407	333	333
query32	133	80	74	74
query33	469	220	192	192
query34	987	815	493	493
query35	408	411	346	346
query36	562	584	540	540
query37	123	80	74	74
query38	995	853	841	841
query39	518	495	471	471
query39_1	476	465	471	465
query40	201	94	80	80
query41	60	57	57	57
query42	76	72	71	71
query43	244	242	213	213
query44	1028	556	548	548
query45	110	105	102	102
query46	774	849	534	534
query47	781	762	700	700
query48	312	310	224	224
query49	536	247	202	202
query50	751	272	197	197
query51	8086	8178	7968	7968
query52	66	66	65	65
query53	194	207	142	142
query54	218	176	155	155
query55	75	61	58	58
query56	189	182	178	178
query57	676	664	710	664
query58	221	171	177	171
query59	1236	1244	1113	1113
query60	230	199	187	187
query61	119	111	120	111
query62	348	220	178	178
query63	164	137	140	137
query64	2852	708	597	597
query65	1661	1586	1663	1586
query66	1849	277	196	196
query67	9930	9623	9535	9535
query68	3017	1229	742	742
query69	347	215	196	196
query70	666	611	610	610
query71	253	170	165	165
query72	2355	1741	1573	1573
query73	662	621	348	348
query74	2016	1221	1151	1151
query75	1194	1099	948	948
query76	2385	723	525	525
query77	257	258	207	207
query78	4071	3720	3169	3169
query79	2785	850	576	576
query80	1614	327	283	283
query81	517	156	131	131
query82	634	123	97	97
query83	294	214	195	195
query84	298	117	90	90
query85	818	354	298	298
query86	471	171	166	166
query87	1018	979	904	904
query88	3079	2139	2122	2122
query89	335	195	180	180
query90	2216	136	132	132
query91	134	121	100	100
query92	95	68	73	68
query93	2381	1031	675	675
query94	643	261	231	231
query95	512	276	298	276
query96	815	573	266	266
query97	1030	1053	1025	1025
query98	178	132	134	132
query99	438	349	299	299
Total cold run time: 180755 ms
Total hot run time: 81807 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9fdbf32e60c4ed7df3868aa9b63dbb9b4bade8e7, data reload: false

query1	0.00	0.00	0.01
query2	0.08	0.04	0.03
query3	0.26	0.11	0.11
query4	1.61	0.10	0.10
query5	0.18	0.16	0.16
query6	1.24	0.72	0.68
query7	0.04	0.01	0.00
query8	0.05	0.03	0.02
query9	0.29	0.26	0.22
query10	0.35	0.35	0.35
query11	0.16	0.12	0.11
query12	0.15	0.12	0.13
query13	0.33	0.31	0.32
query14	0.45	0.46	0.46
query15	0.36	0.35	0.35
query16	0.22	0.21	0.23
query17	0.67	0.76	0.66
query18	0.19	0.17	0.17
query19	1.22	1.22	1.17
query20	0.02	0.01	0.01
query21	15.47	0.18	0.11
query22	5.08	0.04	0.04
query23	16.15	0.24	0.11
query24	3.03	0.35	0.25
query25	0.11	0.05	0.04
query26	0.72	0.17	0.13
query27	0.03	0.04	0.02
query28	3.63	0.53	0.27
query29	12.45	3.21	2.59
query30	0.26	0.11	0.12
query31	2.76	0.39	0.17
query32	3.51	0.32	0.24
query33	1.48	1.55	1.49
query34	15.36	2.22	1.78
query35	1.76	1.76	1.76
query36	0.46	0.29	0.29
query37	0.07	0.04	0.04
query38	0.04	0.02	0.02
query39	0.03	0.02	0.02
query40	0.12	0.08	0.08
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.54 s
Total hot run time: 14.87 s

### What problem does this PR solve?

Issue Number: None

Related PR: apache#67438

Problem Summary: Relation-only lookup in correlated subqueries did not distinguish an absent local qualifier from a local relation alias whose requested column was absent. That allowed an outer alias with the same name to win before local nested-field binding or local error handling. Propagate qualifier occupancy through normal and custom analyzer scopes, preserve same-scope nested-field fallback, and add unit and regression coverage for local columns, nested fields, and missing fields.

### Release note

Fix incorrect correlated column binding when an inner relation alias shadows an outer alias.

### Check List (For Author)

- Test: Regression test / Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.nereids.rules.analysis.TestDereference
    - ./run-regression-test.sh --run -d query_p0 -s test_dereference
- Behavior changed: Yes (inner relation aliases now shadow outer aliases while same-scope nested-field fallback remains supported)
- Does this need documentation: No
@CalvinKirs

Copy link
Copy Markdown
Member Author

/review

@CalvinKirs

Copy link
Copy Markdown
Member Author

run buildall

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review outcome: no new blocking issues found at PR head c649dac66a7478637b9678deab74b235fd16c34b.

The change meets its stated goal: relation-qualified references in correlated subqueries are resolved before nested STRUCT/MAP/VARIANT dereference, while local qualifier shadowing, custom HAVING/QUALIFY/GROUP BY/ORDER BY binders, and lambda lexical scope remain intact. The earlier review concerns in the existing threads (lambda precedence, nested-expression correlation bookkeeping, custom-scope precedence, local alias shadowing, and regression snapshot format) are addressed in this head and were treated as duplicate fences; no distinct residual issue was substantiated.

Critical checkpoints:

  • Goal and behavior: the new relation-only phase and qualifier-occupancy gate preserve nearest-scope binding and valid outer correlation; the added plan assertions and generated regression cases cover the changed behavior.
  • Scope/concurrency/lifecycle: this is per-query analyzer state; no new cross-thread lifecycle or lock interaction was introduced.
  • Error handling: missing scalar/struct fields still fail during analysis; no status, RPC, transaction, persistence, or protocol paths were changed.
  • Compatibility and parallel paths: normal and custom analyzer paths, nested expressions, aggregate scopes, and lambda analysis were reviewed; no configuration or storage-format change is involved.
  • Tests and performance: unit coverage plus deterministic order_qt snapshots were inspected; no build or test command was run in accordance with the review-only prompt. The extra relation-only pass is limited to multipart names when an outer scope exists and reuses the existing matching helpers.
  • User focus: review_focus.txt specifies no additional focus, so the full PR was reviewed.

No inline comments are proposed; this is a summary-only review.

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16702 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c649dac66a7478637b9678deab74b235fd16c34b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17565	3060	3027	3027
q2	2126	263	235	235
q3	10199	884	501	501
q4	4674	246	199	199
q5	7685	572	383	383
q6	134	114	94	94
q7	529	483	376	376
q8	9238	843	905	843
q9	3421	2349	2358	2349
q10	6496	861	698	698
q11	394	206	183	183
q12	607	263	197	197
q13	18273	1559	1189	1189
q14	161	162	148	148
q15	q16	491	405	373	373
q17	1444	963	746	746
q18	3079	2242	2247	2242
q19	1256	784	785	784
q20	366	280	208	208
q21	5618	1705	1850	1705
q22	328	263	222	222
Total cold run time: 94084 ms
Total hot run time: 16702 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3394	3333	3307	3307
q2	498	386	367	367
q3	2200	2327	2126	2126
q4	1186	1153	874	874
q5	2189	2106	2091	2091
q6	163	115	89	89
q7	1011	884	848	848
q8	1587	1391	1384	1384
q9	3117	3120	3081	3081
q10	1871	1782	1599	1599
q11	356	267	249	249
q12	446	432	351	351
q13	1505	1516	1142	1142
q14	175	164	160	160
q15	q16	401	396	367	367
q17	3510	3339	3209	3209
q18	4791	4439	4724	4439
q19	859	780	900	780
q20	1015	963	861	861
q21	3879	3105	3260	3105
q22	404	335	321	321
Total cold run time: 34557 ms
Total hot run time: 30750 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81967 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c649dac66a7478637b9678deab74b235fd16c34b, data reload: false

query5	4236	405	330	330
query6	388	164	130	130
query7	4947	433	231	231
query8	291	118	117	117
query9	8669	2901	2878	2878
query10	404	228	192	192
query11	5386	1061	916	916
query12	121	74	70	70
query13	1197	469	331	331
query14	6056	2187	2066	2066
query14_1	1959	1945	1938	1938
query15	172	117	108	108
query16	903	369	360	360
query17	787	448	352	352
query18	2321	318	230	230
query19	158	135	105	105
query20	71	69	66	66
query21	201	99	86	86
query22	5434	5308	5194	5194
query23	6738	6081	6128	6081
query23_1	6109	6007	6112	6007
query24	7324	1116	750	750
query24_1	773	771	762	762
query25	441	319	259	259
query26	1228	239	133	133
query27	2781	417	266	266
query28	4702	1514	1514	1514
query29	943	455	363	363
query30	250	154	130	130
query31	823	402	339	339
query32	142	78	81	78
query33	467	228	187	187
query34	990	838	497	497
query35	441	409	353	353
query36	598	585	532	532
query37	125	78	71	71
query38	1001	852	802	802
query39	506	496	479	479
query39_1	507	494	463	463
query40	202	92	82	82
query41	60	57	58	57
query42	76	74	72	72
query43	245	243	213	213
query44	1016	560	540	540
query45	108	103	108	103
query46	772	837	531	531
query47	758	770	708	708
query48	311	334	241	241
query49	539	257	192	192
query50	784	268	207	207
query51	8137	8206	7959	7959
query52	71	71	60	60
query53	198	199	156	156
query54	225	175	176	175
query55	71	60	60	60
query56	215	170	161	161
query57	701	740	659	659
query58	203	165	158	158
query59	1247	1217	1097	1097
query60	236	180	174	174
query61	113	122	137	122
query62	372	209	205	205
query63	166	140	139	139
query64	2769	738	579	579
query65	1608	1584	1571	1571
query66	1891	263	217	217
query67	9953	9933	9875	9875
query68	2778	1229	749	749
query69	341	231	193	193
query70	668	626	577	577
query71	262	178	164	164
query72	2326	1757	1547	1547
query73	682	574	347	347
query74	1566	1225	1148	1148
query75	1181	1090	959	959
query76	2308	732	547	547
query77	241	258	214	214
query78	3914	3563	3188	3188
query79	2944	866	589	589
query80	1598	310	284	284
query81	514	156	130	130
query82	1148	120	94	94
query83	285	211	193	193
query84	299	107	89	89
query85	841	347	298	298
query86	480	182	155	155
query87	1001	961	885	885
query88	3072	2099	2095	2095
query89	295	191	173	173
query90	2067	129	127	127
query91	130	120	99	99
query92	93	65	71	65
query93	2110	1151	687	687
query94	638	258	219	219
query95	568	261	226	226
query96	807	564	273	273
query97	1065	1027	1002	1002
query98	189	140	132	132
query99	432	351	304	304
Total cold run time: 179901 ms
Total hot run time: 81967 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.66 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c649dac66a7478637b9678deab74b235fd16c34b, data reload: false

query1	0.00	0.01	0.00
query2	0.08	0.04	0.04
query3	0.24	0.11	0.10
query4	1.60	0.11	0.10
query5	0.17	0.15	0.16
query6	1.26	0.70	0.69
query7	0.04	0.00	0.01
query8	0.05	0.02	0.03
query9	0.29	0.22	0.21
query10	0.34	0.36	0.36
query11	0.17	0.12	0.13
query12	0.15	0.11	0.12
query13	0.31	0.31	0.30
query14	0.45	0.45	0.44
query15	0.36	0.35	0.36
query16	0.22	0.21	0.26
query17	0.64	0.68	0.67
query18	0.18	0.17	0.16
query19	1.18	1.08	1.18
query20	0.02	0.01	0.01
query21	15.46	0.16	0.11
query22	5.07	0.05	0.04
query23	16.17	0.26	0.10
query24	2.97	0.32	0.27
query25	0.10	0.05	0.03
query26	0.76	0.17	0.11
query27	0.04	0.03	0.04
query28	3.66	0.57	0.27
query29	12.42	3.20	2.56
query30	0.24	0.12	0.13
query31	2.76	0.37	0.18
query32	3.51	0.32	0.23
query33	1.52	1.48	1.50
query34	15.38	2.13	1.79
query35	1.77	1.71	1.74
query36	0.44	0.29	0.28
query37	0.07	0.04	0.04
query38	0.05	0.03	0.03
query39	0.04	0.02	0.03
query40	0.12	0.08	0.08
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.02	0.03
Total cold run time: 90.44 s
Total hot run time: 14.66 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 66.83% (139/208) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants