Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Darwin Build_Release failure on Helix tests - Tests timing out #40074

Open
MiYanni opened this issue Apr 9, 2024 · 7 comments
Open

Darwin Build_Release failure on Helix tests - Tests timing out #40074

MiYanni opened this issue Apr 9, 2024 · 7 comments

Comments

@MiYanni
Copy link
Member

MiYanni commented Apr 9, 2024

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=635799
Build error leg or test failing: Microsoft.DotNet.MSBuildSdkResolver.Tests.dll.3.WorkItemExecution
Pull request: #40059

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "END OF WORK ITEM LOG: Command timed out, and was killed",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=635799
Error message validated: [END OF WORK ITEM LOG: Command timed out, and was killed]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/9/2024 5:21:23 PM UTC

Report

Build Definition Test Pull Request
869397 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44855
869322 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44834
869324 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44804
869306 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44803
869305 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44802
869059 dotnet/sdk dotnet.Tests.dll.15.WorkItemExecution #44850
869041 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44804
868896 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44757
868784 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44816
868773 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44815
868628 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44810
868411 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44803
867682 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44809
867645 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44814
867642 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44813
867632 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44812
867649 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44815
868410 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44802
868415 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44805
868202 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44828
868087 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44802
868044 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44824
867981 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44820
867734 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44805
867685 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44816
867420 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44764
867301 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44807
867379 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44802
867326 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44810
867321 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44803
867320 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44802
867221 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44805
866658 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44795
867202 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44570
867060 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44802
866618 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44791
866933 dotnet/sdk dotnet.Tests.dll.15.WorkItemExecution #44699
866598 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44785
866566 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44790
866495 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44786
866309 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.5.WorkItemExecution #44785
866190 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution #44764
866133 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44540
866136 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44779
866111 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #44761
866086 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44699
866026 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44751
865871 dotnet/sdk dotnet-watch.Tests.dll.5.WorkItemExecution #44691
865857 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44699
865846 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44774
865854 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44761
865756 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44603
865634 dotnet/sdk Microsoft.DotNet.MSBuildSdkResolver.Tests.dll.netfx.WorkItemExecution #44751
865474 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44768
865370 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44695
865316 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #44570
865273 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44710
865258 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44767
865246 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #44766
865147 dotnet/sdk dotnet-watch.Tests.dll.4.WorkItemExecution #44603
865079 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44763
865022 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44762
864884 dotnet/sdk dotnet-watch.Tests.dll.4.WorkItemExecution #44691
864943 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44603
864866 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44759
864823 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44749
864717 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44755
864712 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44753
864568 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44745
864671 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44748
864619 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44744
864620 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44743
864611 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44746
864562 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution #44744
864384 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.6.WorkItemExecution #44729
864433 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution #44721
864240 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.3.WorkItemExecution #44735
864318 dotnet/sdk Microsoft.NET.Sdk.Razor.Tool.Tests.dll.1.WorkItemExecution #44603
864358 dotnet/sdk Microsoft.NET.Build.Tests.dll.6.WorkItemExecution #44733
864320 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44740
864275 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution #44737
864273 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44729
863469 dotnet/sdk dotnet.Tests.dll.15.WorkItemExecution #44708
864271 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44728
864173 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #44699
864106 dotnet/sdk dotnet-watch.Tests.dll.4.WorkItemExecution #44691
864239 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44733
862705 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution #44688
864115 dotnet/sdk Microsoft.NET.ToolPack.Tests.dll.3.WorkItemExecution #44733
863986 dotnet/sdk dotnet-watch.Tests.dll.4.WorkItemExecution #44603
864069 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44611
863518 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44712
863759 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution #44603
863795 dotnet/sdk Microsoft.WebTools.AspireService.Tests.dll.WorkItemExecution #44726
863268 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution #44701
863726 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution #44723
863700 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution #44706
863553 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44715
863503 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution #44711
863481 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution #44707
Displaying 100 of 277 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
19 108 277
@nagilson
Copy link
Member

@tmat May you please look at this dotnet watch failure?

@tmat
Copy link
Member

tmat commented May 20, 2024

@nagilson I checked a few logs from above and it looks like the failures are in release/8.0.1xx branch. That wouldn't have various test infra fixes.

@nagilson
Copy link
Member

Thank you for checking :) @tmat, which test infra fixes are you referring to? It may be worth porting them over to 8.0.1xx. Any disagreements @marcpopMSFT?

@nagilson
Copy link
Member

We also just hit it in main, so there might be another existing problem: #41045

@baronfel
Copy link
Member

@tmat this is popping up again in the rc1 branches, when you get back can you take a look?

@tmat
Copy link
Member

tmat commented Aug 15, 2024

@baronfel I can take a look. Do you have a link to the failed run? The test log should be good enough to determine the issue on main/rc1 branches.

@baronfel
Copy link
Member

The helix timeouts are most of the failed tests in this pipeline, but you can't access them right now due to helix auth issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants