Build server warnings - Could not load file or assembly 'Microsoft.CodeAnalysis'

[C-Wal]

Describe the bug

I am getting a stack of these warnings on my build server and I believe it's breaking test discovery in my pipeline:

##[warning]CSC(0,0): Warning CS8032: An instance of analyzer MSTest.Analyzers.UseAttributeOnTestMethodAnalyzer cannot be created from path\to\packages\MSTest.Analyzers.3.3.1\analyzers\dotnet\cs\MSTest.Analyzers.dll : Could not load file or assembly 'Microsoft.CodeAnalysis, Version=3.11.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified..

With a line like this for a whole pile of analyzers in MSTest.Analyzers.

I do not have this issue locally in VS 2022.

Looking at the MSTest.Analyzers.3.3.1 nuget package it declares no dependencies so why is it looking for and failing to find Microsoft.CodeAnalysis?

Steps To Reproduce

.Net Framework 4.8 test project.
Add MSTest.TestFramework 3.3.1 nuget package.
Build on TFS with 2017 Enterprise build agent.

Expected behavior

No warnings, same as in VS 2022.

Actual behavior

Lots of warnings, tests not discovered.

Additional context

I am aware .Net Framework 4.8 and a VS 2017 build agent are old-hat but they are what I've got to work with on this.

I tried using the latest MSTest.TestFramework but it didn't fix it either. I also tried installing the Microsft.CodeAnalysis 3.11 package into my project and I still get exactly the same warnings about it being not found.

[Youssef1313]

I am getting a stack of these warnings on my build server and I believe it's breaking test discovery in my pipeline

These warnings should have no impact on test discovery. If you have issues related to discovery, please report them in a separate issue with some details.

The MSTest.Analyzers package requires Roslyn 3.11 which ships in VS 2019 16.11. So, it's expected this doesn't work in VS 2017. You can consume a newer version of the compiler by adding the following in Directory.Build.props:

<ItemGroup>
  <PackageReference Include="Microsoft.Net.Compilers.Toolset" Version="3.11.0" PrivateAssets="all" />
</ItemGroup>

But keep in mind that, as far as I know, using this package is not officially supported (cc @jaredpar to confirm).

Meanwhile, I'll see the possibility of supporting VS 2017 by the analyzers

[Youssef1313]

Downgrading Roslyn doesn't seem like a good option for us. So:

• Either explicitly update the compiler per the previews suggestion, keeping in mind it's not officially supported.

• Exclude analyzers in your CI:

  <ItemGroup>
    <PackageReference Include="MSTest.Analyzers" ExcludeAssets="all" Condition="'$(TF_BUILD)' == 'true'" />
  </ItemGroup>

[C-Wal]

Thanks. This information was enough to get rid of the warnings for me, by removing absolutely all references to any 'Analyzers' in the projects. My test discovery in the TFS build is still broken ('Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.'), but as you said, that must be a different issue so I'll close this one.