[fix](scheduler) Fix invalid access after freed #48168

Gabriel39 · 2025-02-21T02:21:31Z

What problem does this PR solve?

WRITE of size 1 at 0x6160007e86f0 thread T1983 (Pipe_normal [wo)
#0 0x55fc8065b975 in std::__atomic_base::store(bool, std::memory_order) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:457:2
#1 0x55fc8065b975 in std::__atomic_base::operator=(bool) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:349:2
#2 0x55fc8065b975 in std::atomic::operator=(bool) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:80:22
#3 0x55fc8065b975 in doris::pipeline::PipelineTask::set_running(bool) /root/doris/be/src/pipeline/pipeline_task.h:192:47
#4 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int)::$_0::operator()() const /root/doris/be/src/pipeline/task_scheduler.cpp:121:23
#5 0x55fc8065b975 in doris::Deferdoris::pipeline::TaskScheduler::_do_work(int)::$_0::~Defer() /root/doris/be/src/util/defer_op.h:37:16
#6 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:162:5
#7 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:608:24
#8 0x55fc4c55395e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
#9 0x55fc4c55395e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
#10 0x7f9ee3d25608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
#11 0x7f9ee3fd2132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x6160007e86f0 is located 624 bytes inside of 632-byte region [0x6160007e8480,0x6160007e86f8)
freed by thread T1981 (Pipe_normal [wo) here:
#0 0x55fc47aa680d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x3376e80d) (BuildId: 865149e62959581e)
#1 0x55fc8059db84 in std::default_deletedoris::pipeline::PipelineTask::operator()(doris::pipeline::PipelineTask*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
#2 0x55fc8059db84 in std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >::unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4
#3 0x55fc8059db84 in void std::destroy_at<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
#4 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
#5 0x55fc8059db84 in void std::_Destroy_aux::__destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
#6 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
#7 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
#8 0x55fc8059db84 in std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >::~vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2
#9 0x55fc8052571c in void std::destroy_at<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
#10 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
#11 0x55fc8052571c in void std::_Destroy_aux::__destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
#12 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
#13 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
#14 0x55fc8052571c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > > > >::_M_erase_at_end(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1796:6
#15 0x55fc8052571c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_deletedoris::pipeline::PipelineTask > > > > >::clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1499:9
#16 0x55fc8052571c in doris::pipeline::PipelineFragmentContext::PipelineFragmentContext() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:142:12
#17 0x55fc47ad30cc in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
#18 0x55fc80658d57 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
#19 0x55fc80658d57 in std::__shared_ptr<doris::TaskExecutionContext, (__gnu_cxx::_Lock_policy)2>::__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
#20 0x55fc80658d57 in doris::pipeline::close_task(doris::pipeline::PipelineTask*, doris::Status) /root/doris/be/src/pipeline/task_scheduler.cpp:100:1
#21 0x55fc8065aa17 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:160:36
#22 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:608:24
#23 0x55fc4c55395e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
#24 0x55fc4c55395e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
#25 0x7f9ee3d25608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-02-21T02:21:39Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Gabriel39 · 2025-02-21T02:23:55Z

run buildall

be/src/pipeline/task_scheduler.cpp

Gabriel39 · 2025-02-21T03:50:30Z

run buildall

doris-robot · 2025-02-21T04:11:09Z

TPC-H: Total hot run time: 31817 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f69aacd3e3d3752fa063b02dc9533b8274ce04fa, data reload: false

------ Round 1 ----------------------------------
q1	17601	5319	5191	5191
q2	2072	306	172	172
q3	10404	1363	749	749
q4	10279	1031	527	527
q5	8739	2447	2391	2391
q6	190	172	131	131
q7	914	764	621	621
q8	9320	1367	1187	1187
q9	4962	4646	4700	4646
q10	6864	2313	1879	1879
q11	483	288	274	274
q12	350	350	225	225
q13	17768	3690	3060	3060
q14	226	226	217	217
q15	503	459	458	458
q16	611	609	596	596
q17	613	858	323	323
q18	6761	6321	6209	6209
q19	1211	954	548	548
q20	308	329	195	195
q21	2761	2210	1918	1918
q22	359	330	300	300
Total cold run time: 103299 ms
Total hot run time: 31817 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5139	5136	5114	5114
q2	235	335	228	228
q3	2221	2714	2323	2323
q4	1415	1822	1392	1392
q5	4269	4158	4130	4130
q6	213	161	124	124
q7	1848	1831	1773	1773
q8	2654	2662	2562	2562
q9	7281	7115	7245	7115
q10	2980	3216	2760	2760
q11	585	504	502	502
q12	682	773	616	616
q13	3433	3883	3233	3233
q14	286	296	280	280
q15	501	452	461	452
q16	623	678	632	632
q17	1116	1662	1319	1319
q18	7611	7349	7398	7349
q19	798	809	875	809
q20	1995	2005	1891	1891
q21	5476	4893	4810	4810
q22	620	596	552	552
Total cold run time: 51981 ms
Total hot run time: 49966 ms

HappenLee · 2025-02-21T04:14:24Z

be/src/pipeline/task_scheduler.cpp

+                if (close) {
+                    task->fragment_context()->decrement_running_task(task->pipeline_id());
+                }
+            } else {


seens not need else, should alway set task->set_running(false);

HappenLee

LGTM

github-actions · 2025-02-21T04:20:57Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-21T04:21:00Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-21T04:22:19Z

TPC-DS: Total hot run time: 185255 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f69aacd3e3d3752fa063b02dc9533b8274ce04fa, data reload: false

query1	974	395	382	382
query2	4056	1896	1882	1882
query3	6764	221	210	210
query4	26017	23984	23464	23464
query5	2357	671	509	509
query6	304	209	186	186
query7	4609	516	315	315
query8	297	248	235	235
query9	8639	2776	2770	2770
query10	492	327	277	277
query11	15851	15116	14977	14977
query12	156	112	105	105
query13	1663	547	418	418
query14	8952	6492	7249	6492
query15	210	191	162	162
query16	6228	654	481	481
query17	1186	705	540	540
query18	1946	393	289	289
query19	185	180	155	155
query20	116	115	119	115
query21	196	126	100	100
query22	4294	4178	4193	4178
query23	33981	33076	33177	33076
query24	7796	2428	2421	2421
query25	497	448	388	388
query26	934	276	159	159
query27	2110	498	360	360
query28	3850	2502	2485	2485
query29	662	546	434	434
query30	263	184	159	159
query31	898	825	792	792
query32	72	63	66	63
query33	506	362	301	301
query34	784	856	518	518
query35	799	812	732	732
query36	1017	969	910	910
query37	131	104	71	71
query38	4111	4123	4176	4123
query39	1450	1402	1384	1384
query40	206	114	101	101
query41	54	57	52	52
query42	128	114	115	114
query43	493	514	480	480
query44	1290	841	826	826
query45	178	166	160	160
query46	889	1032	659	659
query47	1741	1830	1738	1738
query48	433	442	329	329
query49	810	571	433	433
query50	708	743	445	445
query51	4203	4151	4168	4151
query52	105	110	96	96
query53	233	255	185	185
query54	500	483	409	409
query55	82	86	89	86
query56	276	278	280	278
query57	1130	1175	1076	1076
query58	245	235	238	235
query59	2867	2846	2551	2551
query60	302	294	270	270
query61	123	112	114	112
query62	803	748	649	649
query63	233	203	194	194
query64	3969	1032	643	643
query65	3231	3131	3173	3131
query66	1096	418	314	314
query67	15748	15538	15225	15225
query68	2348	798	582	582
query69	409	305	272	272
query70	1186	1117	1108	1108
query71	359	300	289	289
query72	5858	3655	3708	3655
query73	657	768	378	378
query74	9190	9226	8915	8915
query75	3112	3192	2650	2650
query76	2212	1167	735	735
query77	341	362	359	359
query78	9941	10206	9375	9375
query79	1023	904	645	645
query80	589	562	477	477
query81	473	271	246	246
query82	479	130	96	96
query83	171	183	160	160
query84	238	97	78	78
query85	736	403	297	297
query86	304	336	306	306
query87	4505	4529	4276	4276
query88	3082	2405	2396	2396
query89	379	321	303	303
query90	1788	198	204	198
query91	146	135	110	110
query92	61	61	62	61
query93	1081	999	575	575
query94	524	418	294	294
query95	332	260	254	254
query96	522	548	283	283
query97	2761	2842	2729	2729
query98	235	211	208	208
query99	1308	1401	1385	1385
Total cold run time: 253779 ms
Total hot run time: 185255 ms

doris-robot · 2025-02-21T04:27:36Z

ClickBench: Total hot run time: 30.49 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f69aacd3e3d3752fa063b02dc9533b8274ce04fa, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.10
query5	0.41	0.41	0.40
query6	1.17	0.67	0.64
query7	0.02	0.02	0.02
query8	0.04	0.02	0.03
query9	0.58	0.51	0.53
query10	0.58	0.57	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.63	0.59	0.60
query14	2.72	2.68	2.71
query15	0.91	0.84	0.84
query16	0.36	0.38	0.38
query17	1.05	1.04	1.05
query18	0.22	0.19	0.19
query19	1.92	1.79	2.09
query20	0.01	0.01	0.01
query21	15.35	0.92	0.54
query22	0.75	1.06	0.63
query23	15.11	1.39	0.64
query24	7.35	1.75	0.63
query25	0.50	0.19	0.15
query26	0.73	0.16	0.14
query27	0.06	0.05	0.04
query28	8.75	0.89	0.44
query29	12.55	3.95	3.36
query30	0.25	0.08	0.06
query31	2.83	0.60	0.39
query32	3.22	0.56	0.47
query33	3.06	3.03	3.06
query34	15.88	5.14	4.51
query35	4.55	4.54	4.58
query36	0.67	0.49	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.14	0.13
query41	0.09	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 104.99 s
Total hot run time: 30.49 s

doris-robot · 2025-02-21T05:34:33Z

TeamCity be ut coverage result:
Function Coverage: 44.28% (11687/26395)
Line Coverage: 33.87% (97852/288905)
Region Coverage: 32.95% (50103/152051)
Branch Coverage: 28.61% (25184/88018)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f69aacd3e3d3752fa063b02dc9533b8274ce04fa_f69aacd3e3d3752fa063b02dc9533b8274ce04fa/report/index.html

WRITE of size 1 at 0x6160007e86f0 thread T1983 (Pipe_normal [wo) #0 0x55fc8065b975 in std::__atomic_base<bool>::store(bool, std::memory_order) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:457:2 apache#1 0x55fc8065b975 in std::__atomic_base<bool>::operator=(bool) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:349:2 apache#2 0x55fc8065b975 in std::atomic<bool>::operator=(bool) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:80:22 apache#3 0x55fc8065b975 in doris::pipeline::PipelineTask::set_running(bool) /root/doris/be/src/pipeline/pipeline_task.h:192:47 apache#4 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int)::$_0::operator()() const /root/doris/be/src/pipeline/task_scheduler.cpp:121:23 apache#5 0x55fc8065b975 in doris::Defer<doris::pipeline::TaskScheduler::_do_work(int)::$_0>::~Defer() /root/doris/be/src/util/defer_op.h:37:16 apache#6 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:162:5 apache#7 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:608:24 apache#8 0x55fc4c55395e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 apache#9 0x55fc4c55395e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 apache#10 0x7f9ee3d25608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 apache#11 0x7f9ee3fd2132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 0x6160007e86f0 is located 624 bytes inside of 632-byte region [0x6160007e8480,0x6160007e86f8) freed by thread T1981 (Pipe_normal [wo) here: #0 0x55fc47aa680d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x3376e80d) (BuildId: 865149e62959581e) apache#1 0x55fc8059db84 in std::default_delete<doris::pipeline::PipelineTask>::operator()(doris::pipeline::PipelineTask*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2 apache#2 0x55fc8059db84 in std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >::~unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4 apache#3 0x55fc8059db84 in void std::destroy_at<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 apache#4 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7 apache#5 0x55fc8059db84 in void std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6 apache#6 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*>(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7 apache#7 0x55fc8059db84 in void std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >(std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >*, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7 apache#8 0x55fc8059db84 in std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >::~vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2 apache#9 0x55fc8052571c in void std::destroy_at<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 apache#10 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7 apache#11 0x55fc8052571c in void std::_Destroy_aux<false>::__destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6 apache#12 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7 apache#13 0x55fc8052571c in void std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7 apache#14 0x55fc8052571c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > > >::_M_erase_at_end(std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1796:6 apache#15 0x55fc8052571c in std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > >, std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> >, std::allocator<std::unique_ptr<doris::pipeline::PipelineTask, std::default_delete<doris::pipeline::PipelineTask> > > > > >::clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1499:9 apache#16 0x55fc8052571c in doris::pipeline::PipelineFragmentContext::~PipelineFragmentContext() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:142:12 apache#17 0x55fc47ad30cc in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6 apache#18 0x55fc80658d57 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11 apache#19 0x55fc80658d57 in std::__shared_ptr<doris::TaskExecutionContext, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31 apache#20 0x55fc80658d57 in doris::pipeline::close_task(doris::pipeline::PipelineTask*, doris::Status) /root/doris/be/src/pipeline/task_scheduler.cpp:100:1 apache#21 0x55fc8065aa17 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:160:36 apache#22 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:608:24 apache#23 0x55fc4c55395e in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 apache#24 0x55fc4c55395e in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5 apache#25 0x7f9ee3d25608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8

[fix](scheduler) Fix invalid access after freed

1ebcdd9

BiteTheDDDDt reviewed Feb 21, 2025

View reviewed changes

be/src/pipeline/task_scheduler.cpp Outdated Show resolved Hide resolved

update

f69aacd

Gabriel39 force-pushed the fix_0221 branch from 4fc618a to f69aacd Compare February 21, 2025 03:50

HappenLee reviewed Feb 21, 2025

View reviewed changes

HappenLee approved these changes Feb 21, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 21, 2025

github-actions bot added the reviewed label Feb 21, 2025

BiteTheDDDDt approved these changes Feb 21, 2025

View reviewed changes

Gabriel39 merged commit b7c973e into apache:master Feb 21, 2025
24 of 28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](scheduler) Fix invalid access after freed #48168

[fix](scheduler) Fix invalid access after freed #48168

Gabriel39 commented Feb 21, 2025

hello-stephen commented Feb 21, 2025

Gabriel39 commented Feb 21, 2025

Gabriel39 commented Feb 21, 2025

doris-robot commented Feb 21, 2025

HappenLee Feb 21, 2025

HappenLee left a comment

github-actions bot commented Feb 21, 2025

github-actions bot commented Feb 21, 2025

doris-robot commented Feb 21, 2025

doris-robot commented Feb 21, 2025

doris-robot commented Feb 21, 2025

[fix](scheduler) Fix invalid access after freed #48168

[fix](scheduler) Fix invalid access after freed #48168

Conversation

Gabriel39 commented Feb 21, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

hello-stephen commented Feb 21, 2025

Gabriel39 commented Feb 21, 2025

Gabriel39 commented Feb 21, 2025

doris-robot commented Feb 21, 2025

HappenLee Feb 21, 2025

Choose a reason for hiding this comment

HappenLee left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 21, 2025

github-actions bot commented Feb 21, 2025

doris-robot commented Feb 21, 2025

doris-robot commented Feb 21, 2025

doris-robot commented Feb 21, 2025