dotnet-watch tests failing

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=629827
Build error leg or test failing: dotnet-watch.Tests.dll.1.WorkItemExecution
Pull request: #39888

Error Message

##[error].packages/microsoft.dotnet.helix.sdk/8.0.0-beta.24177.1/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(89,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item dotnet-watch.Tests.dll.1 in job 82728a56-9430-4356-ab82-1c21670792a7 has failed

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item dotnet-watch\\.Tests\\.dll\\.1 in job [a-z0-9\\-]+ has failed",
  "BuildRetry": true,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=629827
Error message validated: [Work item dotnet-watch\.Tests\.dll\.1 in job [a-z0-9\-]+ has failed]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/4/2024 9:51:10 AM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2732403	dotnet-sdk	Run FullFramework Tests	Log	#51011
2732285	dotnet-sdk	Run TestBuild Tests	Log	#50939
2732284	dotnet-sdk	Run TestBuild Tests	Log	#51000
2731748	dotnet-sdk	Run FullFramework Tests	Log	#50943
1070309	dotnet/sdk	Run TestBuild Tests	Log	#49326
1069918	dotnet/sdk	Run TestBuild Tests	Log	#49444
2731715	dotnet-sdk	Run FullFramework Tests	Log	#50938
1069692	dotnet/sdk	Run TestBuild Tests	Log	#49421
1069571	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49284
1066660	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49284
1068620	dotnet/sdk	Run TestBuild Tests	Log	#49408
1068588	dotnet/sdk	🟣 Run FullFramework Tests	Log	#49409
1068007	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49389
1067512	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49374
1067583	dotnet/sdk	Run TestBuild Tests	Log	#49393
2729237	dotnet-sdk	Run FullFramework Tests	Log	#50855
1066921	dotnet/sdk	Run TestBuild Tests	Log	#49375
1066696	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49374
1065747	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49290
1065794	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49357
1065782	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49288
1065766	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49306
1065706	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49352
1065689	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49329
1065749	dotnet/sdk	Run FullFramework Tests	Log	#49356
1065631	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49118
1064368	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49242
1060792	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49280
1065610	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49310
2728318	dotnet-sdk	🟣 Run ContainerBased Tests (debian12Amd64)	Log
1065477	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49332
1065487	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49306
1064445	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49290
1064363	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49239
1065543	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49288
1064753	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49306
1065101	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49288
1065102	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49330
1065083	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49336
1065159	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#48906
1065055	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49329
2728071	dotnet-sdk	🟣 Run TestBuild Tests	Log
1064899	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49348
1064897	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49346
1064858	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49308
1064808	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49315
1064765	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#48387
1064741	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49287
2727783	dotnet-sdk	🟣 Run TestBuild Tests	Log
1064729	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49311
1064585	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49306
1064373	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49335
1064242	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49287
1064372	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49336
1064474	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49306
1064303	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49166
1064298	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49334
2727354	dotnet-sdk	🟣 Run TestBuild Tests	Log
1064268	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49332
1064272	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49150
1064250	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49328
1064248	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49315
1064233	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49314
1064216	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48906
1064222	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49312
1064208	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49330
1064206	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49329
1064207	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49186
1064212	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49284
1064194	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48819
1064304	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49290
1063970	dotnet/sdk	Run TestBuild Tests	Log	#49057
1064090	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49323
2727284	dotnet-sdk	🟣 Run ContainerBased Tests (debian12Amd64)	Log
1063209	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49306
1063972	dotnet/sdk	Run FullFramework Tests	Log	#49061
1063176	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49290
1063880	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49315
1064038	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49316
1063879	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49315
1063868	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49314
1063854	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49312
1063853	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49312
1063978	dotnet/sdk	Run TestBuild Tests	Log	#49134
1063957	dotnet/sdk	Run TestBuild Tests	Log	#45548
1063902	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#48906
1063897	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49150
1063768	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49311
1063737	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49310
1062477	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49013
1063788	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49150
2727011	dotnet-sdk	🟣 Run ContainerBased Tests (debian12Amd64)	Log
1063683	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49309
1063686	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49310
1063536	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48387
1063600	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49309
1063418	dotnet/sdk	🟣 Run TestBuild Tests	Log	#48387
1063369	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#49308
1063370	dotnet/sdk	🟣 Run ContainerBased Tests	Log	#48387
1063334	dotnet/sdk	🟣 Run TestBuild Tests	Log	#49287
Displaying 100 of 189 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	18	189

[marcpopMSFT]

@tmat we still see this fairly frequently. Any luck? Here's a recent one from a codeflow PR:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-41616-merge-58511284b77c40778a/dotnet-watch.Tests.dll.1/1/console.418de7ca.log?helixlogtype=result

[tmat]

No luck yet. This might help: https://github.com/dotnet/sdk/blob/d45e40d665e647804de03d2b4eec2aae9c488cb5/test/dotnet-watch.Tests/xunit.runner.json

It will print out test info every 20 seconds a test is running.

[marcpopMSFT]

@tmat another failure this morning shows a bunch of build processes for watch apps still running... https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-sdk-refs-pull-41958-merge-84357342ee154a5088/dotnet-watch.Tests.dll.2/1/console.3dddc463.log?helixlogtype=result

[tmat]

Looks like that's in 8.0 branch. Porting test diagnostics from 9: #41963

[Forgind]

This has been making one of my PRs fail today, so I looked into it a bit. Based on when this issue was created and which tests are failing, I suspect this PR:
#39618

I asked jasonmalinowski whether the MSBuildWorkspace is supposed to support F# because I'd found a code snippet that suggested that it shouldn't, and he said it doesn't and never did. Did that PR make us use MSBuildWorkspace in a different way than we had before?

[Forgind]

The changes to HotReloadDotNetWatcher look most suspicious to me. It looks like a substantial change to how we call MSBuild.