[fix](nereids) deduplicate scope slots #58683

englefly · 2025-12-03T09:33:26Z

If the slots in the scope are not deduplicated, some slots will be bounded twice, and optimizer throw exception because of ambigious.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

…rted when executing the query: time_createdis ambiguous

hello-stephen · 2025-12-03T09:33:33Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

morrySnow · 2025-12-03T09:38:17Z

fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/Scope.java

    public Scope(Optional<Scope> outerScope, List<Slot> slots, List<Slot> asteriskSlots) {
        this.outerScope = Objects.requireNonNull(outerScope, "outerScope can not be null");
-        this.slots = Utils.fastToImmutableList(Objects.requireNonNull(slots, "slots can not be null"));
+        this.slots = Utils.fastToImmutableList(Objects.requireNonNull(


change parameters to Set<Slot> ?

调用处拿到的也是list, 所以这里把签名改了也没有用,相当于把这个转换放到调用者那里执行

englefly · 2025-12-03T10:59:56Z

run buildall

doris-robot · 2025-12-03T11:28:03Z

TPC-H: Total hot run time: 34435 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a0475bb9c7f3fbc6e60ca1b506a8eb9c9639701c, data reload: false

------ Round 1 ----------------------------------
q1	17676	5233	5008	5008
q2	2087	318	216	216
q3	10199	1300	740	740
q4	10215	828	322	322
q5	7535	2410	2147	2147
q6	187	174	146	146
q7	947	802	646	646
q8	9352	1414	1087	1087
q9	6969	5295	5284	5284
q10	6798	2178	1789	1789
q11	525	318	294	294
q12	337	368	228	228
q13	17763	3670	3080	3080
q14	247	236	228	228
q15	582	505	514	505
q16	892	887	824	824
q17	668	824	472	472
q18	7449	7201	7116	7116
q19	1104	961	590	590
q20	373	354	230	230
q21	3953	3362	2530	2530
q22	1039	1009	953	953
Total cold run time: 106897 ms
Total hot run time: 34435 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5000	4979	4947	4947
q2	333	385	312	312
q3	2186	2735	2302	2302
q4	1329	1737	1287	1287
q5	4250	4535	4650	4535
q6	212	170	130	130
q7	2030	2019	1833	1833
q8	2670	2662	2570	2570
q9	7473	7672	7394	7394
q10	3058	3321	2901	2901
q11	600	506	500	500
q12	703	775	613	613
q13	3675	3884	3335	3335
q14	291	318	294	294
q15	594	505	532	505
q16	897	927	921	921
q17	1208	1499	1528	1499
q18	7672	7701	7543	7543
q19	879	889	913	889
q20	1995	2072	1923	1923
q21	4962	4615	4557	4557
q22	1070	1077	978	978
Total cold run time: 53087 ms
Total hot run time: 51768 ms

doris-robot · 2025-12-03T11:39:28Z

TPC-DS: Total hot run time: 182790 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a0475bb9c7f3fbc6e60ca1b506a8eb9c9639701c, data reload: false

query1	1066	398	399	398
query2	6569	1213	1170	1170
query3	6736	236	226	226
query4	25527	23321	22831	22831
query5	5068	642	518	518
query6	350	257	243	243
query7	4669	544	312	312
query8	326	263	247	247
query9	8729	2667	2663	2663
query10	540	403	320	320
query11	15371	14940	15239	14940
query12	192	118	118	118
query13	1695	596	474	474
query14	10028	6105	6090	6090
query15	214	198	187	187
query16	7683	686	541	541
query17	1242	779	687	687
query18	2052	430	341	341
query19	216	209	176	176
query20	130	126	120	120
query21	215	137	113	113
query22	3864	3972	3846	3846
query23	33016	32181	31997	31997
query24	8425	2424	2437	2424
query25	620	503	464	464
query26	1242	278	180	180
query27	2717	506	337	337
query28	4333	2199	2174	2174
query29	835	637	501	501
query30	304	249	221	221
query31	832	710	614	614
query32	83	80	78	78
query33	609	388	345	345
query34	854	886	538	538
query35	795	842	750	750
query36	923	913	830	830
query37	128	116	106	106
query38	3894	3865	3779	3779
query39	1551	1436	1419	1419
query40	239	134	124	124
query41	71	66	63	63
query42	131	122	118	118
query43	440	449	434	434
query44	1362	778	756	756
query45	200	191	193	191
query46	909	1027	652	652
query47	1699	1723	1626	1626
query48	421	436	334	334
query49	787	509	416	416
query50	715	709	429	429
query51	3893	4017	3869	3869
query52	118	116	112	112
query53	244	273	189	189
query54	317	305	287	287
query55	96	99	93	93
query56	335	340	323	323
query57	1134	1169	1071	1071
query58	288	283	276	276
query59	2284	2400	2325	2325
query60	364	377	358	358
query61	166	168	166	166
query62	776	712	653	653
query63	239	202	192	192
query64	4526	1326	1030	1030
query65	4080	3976	3971	3971
query66	1086	471	341	341
query67	15243	15168	14918	14918
query68	8398	1007	639	639
query69	526	351	315	315
query70	1142	1024	985	985
query71	439	368	323	323
query72	5762	4937	4851	4851
query73	680	592	350	350
query74	8915	8887	8681	8681
query75	3053	3091	2558	2558
query76	3311	1164	753	753
query77	548	430	342	342
query78	9381	9633	8857	8857
query79	1477	872	591	591
query80	644	597	494	494
query81	514	272	235	235
query82	227	145	118	118
query83	282	267	253	253
query84	265	118	101	101
query85	903	481	538	481
query86	376	299	272	272
query87	4053	4040	4007	4007
query88	3070	2342	2333	2333
query89	397	344	312	312
query90	1882	237	233	233
query91	197	210	165	165
query92	82	81	72	72
query93	1222	1045	685	685
query94	725	487	346	346
query95	542	415	419	415
query96	563	562	295	295
query97	2607	2709	2587	2587
query98	241	215	209	209
query99	1298	1382	1297	1297
Total cold run time: 269528 ms
Total hot run time: 182790 ms

hello-stephen · 2025-12-03T13:49:11Z

FE Regression Coverage Report

Increment line coverage 100.00% (4/4) 🎉
Increment coverage report
Complete coverage report

englefly · 2025-12-04T01:53:00Z

run buildall

englefly · 2025-12-04T01:54:56Z

run buildall

doris-robot · 2025-12-04T02:14:29Z

TPC-H: Total hot run time: 34756 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d96ee4ae3e1052d5aa662f45a6524a1ac34f69a, data reload: false

------ Round 1 ----------------------------------
q1	17643	5067	4916	4916
q2	2074	318	207	207
q3	10241	1426	754	754
q4	10222	860	334	334
q5	7526	2396	2235	2235
q6	191	170	138	138
q7	937	808	634	634
q8	9355	1432	1113	1113
q9	7036	5305	5323	5305
q10	6796	2219	1796	1796
q11	524	321	307	307
q12	347	366	228	228
q13	17770	3709	3087	3087
q14	232	248	218	218
q15	571	520	513	513
q16	917	872	812	812
q17	679	779	599	599
q18	7293	7178	7094	7094
q19	1089	959	627	627
q20	368	354	226	226
q21	4034	3508	2657	2657
q22	1019	992	956	956
Total cold run time: 106864 ms
Total hot run time: 34756 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4961	4950	4907	4907
q2	336	390	316	316
q3	2135	2629	2299	2299
q4	1321	1758	1299	1299
q5	4244	4540	4625	4540
q6	214	174	126	126
q7	2067	1981	1835	1835
q8	2675	2482	2565	2482
q9	7614	7597	7518	7518
q10	3280	3236	2778	2778
q11	588	506	504	504
q12	737	789	630	630
q13	3553	3955	3389	3389
q14	302	307	263	263
q15	560	522	536	522
q16	893	940	910	910
q17	1221	1433	1413	1413
q18	8047	7741	7466	7466
q19	902	847	938	847
q20	2059	2101	1961	1961
q21	4982	4482	4178	4178
q22	1099	1031	956	956
Total cold run time: 53790 ms
Total hot run time: 51139 ms

doris-robot · 2025-12-04T02:25:49Z

TPC-DS: Total hot run time: 181781 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d96ee4ae3e1052d5aa662f45a6524a1ac34f69a, data reload: false

query5	5570	646	487	487
query6	340	246	233	233
query7	4666	520	324	324
query8	310	273	252	252
query9	8736	2623	2632	2623
query10	593	383	324	324
query11	15479	15129	14890	14890
query12	186	118	115	115
query13	1699	581	466	466
query14	7781	3382	3174	3174
query14_1	3003	3089	2995	2995
query15	269	206	186	186
query16	7806	697	504	504
query17	1609	810	655	655
query18	2050	441	329	329
query19	287	215	174	174
query20	134	123	124	123
query21	218	136	114	114
query22	3936	4004	3862	3862
query23	16672	16242	15904	15904
query23_1	16009	15907	16017	15907
query24	6517	1606	1206	1206
query24_1	1197	1239	1223	1223
query25	577	517	461	461
query26	1227	281	176	176
query27	2687	502	341	341
query28	4338	2178	2159	2159
query29	795	629	505	505
query30	314	243	219	219
query31	841	747	670	670
query32	87	77	70	70
query33	599	378	334	334
query34	838	902	546	546
query35	795	819	753	753
query36	892	942	848	848
query37	123	119	89	89
query38	3839	3908	3778	3778
query39	778	752	705	705
query39_1	707	702	699	699
query40	222	132	120	120
query41	71	65	64	64
query42	133	116	117	116
query43	456	452	433	433
query44	1360	775	761	761
query45	201	196	187	187
query46	929	1032	670	670
query47	1710	1727	1640	1640
query48	409	444	351	351
query49	809	528	432	432
query50	750	717	428	428
query51	3951	3861	3981	3861
query52	120	114	106	106
query53	249	255	191	191
query54	322	312	288	288
query55	94	99	97	97
query56	340	334	320	320
query57	1149	1164	1067	1067
query58	293	282	273	273
query59	2299	2442	2301	2301
query60	364	350	345	345
query61	164	162	172	162
query62	765	711	656	656
query63	244	196	197	196
query64	4498	1225	972	972
query65	4054	3945	3968	3945
query66	1045	437	335	335
query67	15359	14910	14697	14697
query68	4919	1024	663	663
query69	530	347	310	310
query70	1129	1050	1023	1023
query71	440	350	330	330
query72	5901	4884	4949	4884
query73	669	578	343	343
query74	8555	8796	8609	8609
query75	3031	3033	2492	2492
query76	3216	1156	765	765
query77	516	417	313	313
query78	9522	9703	8960	8960
query79	1294	876	582	582
query80	694	603	506	506
query81	488	276	250	250
query82	197	136	117	117
query83	272	276	257	257
query84	265	118	93	93
query85	888	497	455	455
query86	344	289	277	277
query87	4024	4101	3940	3940
query88	2945	2312	2248	2248
query89	377	330	295	295
query90	1852	230	230	230
query91	183	183	145	145
query92	75	73	70	70
query93	1188	1054	680	680
query94	675	446	355	355
query95	492	426	413	413
query96	544	546	287	287
query97	2615	2716	2549	2549
query98	249	219	216	216
query99	1344	1410	1275	1275
Total cold run time: 265044 ms
Total hot run time: 181781 ms

yujun777

LGTM

github-actions · 2025-12-04T03:28:08Z

PR approved by anyone and no changes requested.

hello-stephen · 2025-12-04T05:14:43Z

FE Regression Coverage Report

Increment line coverage 100.00% (4/4) 🎉
Increment coverage report
Complete coverage report

If the slots in the scope are not deduplicated, an error will be repo…

a0475bb

…rted when executing the query: time_createdis ambiguous

morrySnow reviewed Dec 3, 2025

View reviewed changes

englefly changed the title ~~[fix](nereids) dedeplicate scope slots~~ [fix](nereids) deduplicate scope slots Dec 3, 2025

add comments

9d96ee4

yujun777 approved these changes Dec 4, 2025

View reviewed changes

github-actions bot added the reviewed label Dec 4, 2025

[fix](nereids) deduplicate scope slots #58683

Are you sure you want to change the base?

[fix](nereids) deduplicate scope slots #58683

Conversation

englefly commented Dec 3, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Dec 3, 2025

Uh oh!

morrySnow Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

englefly Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

englefly commented Dec 3, 2025

Uh oh!

doris-robot commented Dec 3, 2025

Uh oh!

doris-robot commented Dec 3, 2025

Uh oh!

hello-stephen commented Dec 3, 2025

FE Regression Coverage Report

Uh oh!

englefly commented Dec 4, 2025

Uh oh!

englefly commented Dec 4, 2025

Uh oh!

doris-robot commented Dec 4, 2025

Uh oh!

doris-robot commented Dec 4, 2025

Uh oh!

yujun777 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

hello-stephen commented Dec 4, 2025

FE Regression Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

englefly Dec 3, 2025 •

edited

Loading