[Fix](test) make pyudf ut deterministic and meaningful by linrrzqqq · Pull Request #67352 · apache/doris

linrrzqqq · 2026-08-31T17:09:14Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Some Python UDF BE unit tests were flaky and provided limited verification value. Several tests launched shell-based fake Python processes that only created Unix socket files or stayed alive for a fixed duration. These tests exercised process-management branches but did not verify that Doris could start the production Python server, establish an Arrow Flight connection, or execute an actual UDF.

This PR:

Removes or rewrites low-value Python UDF tests that only exercised implementation details or line coverage.
Keeps meaningful coverage for process startup failures, shutdown, recovery, pool initialization, and concurrent process selection.
Fixes the race in the least-shared-process test by waiting until the complete process pool is available before checking process distribution.
Adds an end-to-end BE unit test that:
- Creates a Python runtime through the production venv scanner.
- Starts the real python_server.py.
- Connects through Arrow Flight.
- Executes a nullable inline scalar UDF.
- Verifies the returned values and null propagation.
Adds PYTHON_UDF_TEST_INTERPRETER as the only Python interpreter configuration for this test. It defaults to /usr/bin/python, matching the community P0 pipeline, and can be overridden locally in custom_env.sh.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Python server unit tests previously relied on shell fakes that only created socket files, so they could not verify that the configured runtime starts the production Python server, serves Arrow Flight requests, or executes UDF code correctly. Add config-driven test deployment setup and an end-to-end BE test that starts the production Python server and validates nullable inline UDF results over Arrow Flight. Keep short fake-process timeouts while allowing the real-server test to use production-like startup budgets. Also make the least-shared-process assertion wait for the full process pool to remove its initialization race. Use one Python interpreter environment variable for the real-server test, defaulting to the community P0 pipeline path and allowing local overrides through custom_env.sh. ### Release note None ### Check List (For Author) - Test: Unit Test - PYTHONPATH=/tmp/doris_python_udf_ut_deps PYTHON_UDF_TEST_INTERPRETER=/usr/local/bin/python3 ./run-be-ut.sh --run '--filter=Python*Test.*' -j 32 (111 tests passed) - build-support/check-build-hygiene.sh - build-support/check-format.sh - build-support/run-clang-tidy.sh --build-dir be/ut_build_ASAN --files be/src/udf/python/python_server.h be/src/udf/python/python_server.cpp be/test/udf/python/python_server_test.cpp - bash -n env.sh - bash -n run-be-ut.sh - git diff --check - Behavior changed: No - Does this need documentation: No

hello-stephen · 2026-08-31T17:09:19Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

linrrzqqq · 2026-09-01T06:31:23Z

run buildall

hello-stephen · 2026-09-01T08:01:25Z

TPC-H: Total hot run time: 16842 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6c5c1414fd09aa38718dcdd1c8f219471e0aa9e1, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17569	3004	2989	2989
q2	2095	249	221	221
q3	10253	927	519	519
q4	4675	248	211	211
q5	7666	566	387	387
q6	136	117	93	93
q7	531	507	384	384
q8	9239	905	859	859
q9	3433	2423	2406	2406
q10	6512	844	718	718
q11	400	198	184	184
q12	616	259	203	203
q13	18117	1530	1161	1161
q14	158	147	142	142
q15	q16	442	400	376	376
q17	1393	840	779	779
q18	3095	2290	2262	2262
q19	1121	889	779	779
q20	372	285	208	208
q21	5296	1733	1899	1733
q22	337	267	228	228
Total cold run time: 93456 ms
Total hot run time: 16842 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3352	3320	3274	3274
q2	511	437	365	365
q3	2203	2396	2192	2192
q4	1208	1205	915	915
q5	2203	2160	2113	2113
q6	181	125	90	90
q7	1081	943	861	861
q8	1608	1412	1404	1404
q9	3134	3099	3111	3099
q10	1875	1801	1632	1632
q11	358	266	249	249
q12	458	428	340	340
q13	1498	1536	1139	1139
q14	175	178	159	159
q15	q16	388	391	357	357
q17	3631	3421	3222	3222
q18	4865	4475	4878	4475
q19	877	931	878	878
q20	1000	959	844	844
q21	3871	3123	3258	3123
q22	407	344	335	335
Total cold run time: 34884 ms
Total hot run time: 31066 ms

hello-stephen · 2026-09-01T08:07:56Z

TPC-DS: Total hot run time: 82324 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6c5c1414fd09aa38718dcdd1c8f219471e0aa9e1, data reload: false

query5	4275	408	351	351
query6	396	149	125	125
query7	4911	438	236	236
query8	310	121	115	115
query9	8692	2867	2918	2867
query10	432	218	187	187
query11	5375	1039	909	909
query12	119	74	71	71
query13	1187	448	334	334
query14	6164	2213	2073	2073
query14_1	2014	1970	1957	1957
query15	170	119	110	110
query16	935	377	346	346
query17	791	451	383	383
query18	2334	334	243	243
query19	164	142	118	118
query20	73	73	77	73
query21	211	101	87	87
query22	5536	5473	5357	5357
query23	6852	6425	6042	6042
query23_1	6243	6263	6000	6000
query24	7250	1082	776	776
query24_1	763	768	775	768
query25	408	278	233	233
query26	1220	242	127	127
query27	2775	412	249	249
query28	4716	1518	1508	1508
query29	924	431	334	334
query30	254	153	131	131
query31	825	397	320	320
query32	132	74	74	74
query33	462	211	170	170
query34	988	853	484	484
query35	408	403	337	337
query36	572	570	543	543
query37	123	80	70	70
query38	1011	854	821	821
query39	483	500	487	487
query39_1	453	441	485	441
query40	202	89	78	78
query41	54	52	52	52
query42	78	70	70	70
query43	237	266	206	206
query44	1019	546	545	545
query45	110	107	109	107
query46	795	833	549	549
query47	767	741	711	711
query48	317	319	245	245
query49	536	234	184	184
query50	721	266	194	194
query51	8019	7849	7971	7849
query52	68	66	57	57
query53	200	202	152	152
query54	218	184	149	149
query55	70	62	63	62
query56	198	158	149	149
query57	754	689	656	656
query58	196	158	164	158
query59	1220	1222	1087	1087
query60	284	184	174	174
query61	110	128	112	112
query62	382	200	173	173
query63	170	144	143	143
query64	2691	673	591	591
query65	1723	1709	1632	1632
query66	1826	251	200	200
query67	10538	9907	9770	9770
query68	3039	1147	766	766
query69	345	226	201	201
query70	674	615	621	615
query71	254	191	171	171
query72	2491	1944	1726	1726
query73	630	570	350	350
query74	2004	1245	1145	1145
query75	1213	1148	983	983
query76	2352	775	567	567
query77	261	260	211	211
query78	3967	3689	3238	3238
query79	2354	857	603	603
query80	1611	334	286	286
query81	486	157	137	137
query82	611	134	97	97
query83	271	208	191	191
query84	295	105	87	87
query85	817	359	288	288
query86	381	177	177	177
query87	1020	968	897	897
query88	2778	2131	2127	2127
query89	280	198	179	179
query90	1998	132	128	128
query91	131	124	104	104
query92	76	70	63	63
query93	1638	1139	688	688
query94	624	257	236	236
query95	540	322	231	231
query96	786	580	266	266
query97	1070	1083	1021	1021
query98	159	135	132	132
query99	415	343	319	319
Total cold run time: 179480 ms
Total hot run time: 82324 ms

hello-stephen · 2026-09-01T08:12:19Z

ClickBench: Total hot run time: 14.66 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6c5c1414fd09aa38718dcdd1c8f219471e0aa9e1, data reload: false

query1	0.01	0.00	0.01
query2	0.07	0.04	0.04
query3	0.25	0.11	0.11
query4	1.60	0.10	0.09
query5	0.17	0.17	0.17
query6	1.25	0.72	0.68
query7	0.04	0.01	0.00
query8	0.05	0.02	0.03
query9	0.28	0.21	0.22
query10	0.35	0.35	0.35
query11	0.17	0.11	0.12
query12	0.15	0.12	0.12
query13	0.32	0.31	0.31
query14	0.44	0.44	0.45
query15	0.35	0.36	0.35
query16	0.25	0.23	0.24
query17	0.69	0.67	0.70
query18	0.18	0.16	0.16
query19	1.20	1.12	1.15
query20	0.01	0.02	0.03
query21	15.43	0.17	0.13
query22	5.07	0.05	0.05
query23	16.19	0.25	0.10
query24	3.00	0.30	0.27
query25	0.11	0.04	0.04
query26	0.82	0.17	0.11
query27	0.03	0.03	0.03
query28	3.67	0.54	0.27
query29	12.46	3.15	2.57
query30	0.26	0.11	0.12
query31	2.75	0.37	0.16
query32	3.54	0.31	0.24
query33	1.50	1.43	1.39
query34	15.37	2.27	1.77
query35	1.73	1.71	1.74
query36	0.45	0.30	0.29
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.04	0.02	0.02
query40	0.11	0.08	0.07
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.61 s
Total hot run time: 14.66 s

linrrzqqq added 2 commits August 31, 2026 17:22

[Fix](test) make pyudf ut deterministic and meaningful

9ebe467

Navigation Menu

Uh oh!

[Fix](test) make pyudf ut deterministic and meaningful - #67352

[Fix](test) make pyudf ut deterministic and meaningful#67352
linrrzqqq wants to merge 2 commits into
apache:masterfrom
linrrzqqq:pyudf-test

linrrzqqq commented Aug 31, 2026

Uh oh!

hello-stephen commented Aug 31, 2026

Uh oh!

linrrzqqq commented Sep 1, 2026

Uh oh!

hello-stephen commented Sep 1, 2026

Uh oh!

hello-stephen commented Sep 1, 2026

Uh oh!

hello-stephen commented Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

linrrzqqq commented Aug 31, 2026

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Aug 31, 2026

Uh oh!

linrrzqqq commented Sep 1, 2026

Uh oh!

hello-stephen commented Sep 1, 2026

Uh oh!

hello-stephen commented Sep 1, 2026

Uh oh!

hello-stephen commented Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants