FileNotFoundException for System.Security.Cryptography.Csp version 5. .NET doesn't know to look in C:\Program Files\dotnet directory?

[briangru]

Description

Hey guys, please help me out. My CLR loader knowledge is from 2012, and I need to be reeducated. I'm happy to swing by campus if you guys still eat lunch in Building 18 or 25.

One of my .NET 5 executables worked earlier this week, but broke now due to a FileNotFoundException for System.Security.Cryptography.Csp version 5. In the intervening time we installed the latest Windows Update and VS builds, and I am sure one of them included a new .NET 5 update, version 5.0.10. This may have broken it.

There is no version 5 of this assembly outside of .NET 5 internal "packages", as far as I can tell. There is a version in this directory, I guess it's the new GAC for .NET Core, but only for reference assemblies?
C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0

Note - if we have installed something like .NET 5.0.10, should that update these libraries? There is no 5.0.10 reference assembly directory.

I copied the reference assembly (which has no code!) to my application's directory and of course the loader was able to find the file. My app even seemed to work. (I changed the .NET Framework's loader to explicitly throw an exception if we stupidly loaded a reference assembly for execution, as opposed to the reflection-only loader context, but that is exactly what I'm doing and it doesn't fail. I guess my error check didn't translate over to .NET Core, or the loader is now more complicated than I think.) But my file copying hack doesn't seem like a smart thing to do. And I don't know if the code was returning something intelligent, vs. maybe null or just being a noop.

I looked at the System.Security.Cryptography.Csp Nuget package and added a reference to it in my project. Building this was disappointing for two reasons:

1. The assembly version number in the Nuget package is 4:2:1:0
2. VS did not copy the DLL to the output directory. Perhaps it did this because the Nuget package contains NET 4.6, 4.61 and 4.6.2 versions of this binary ONLY. No .NET Standard 2.1 build exists, nor a .NET 5 version.

My code:

        private static String GenerateRandomToken()
        {
            byte[] bytes = new byte[Constants.TokenLength];
            using (RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider())
                rng.GetNonZeroBytes(bytes);
            return Convert.ToBase64String(bytes);
        }

Relevant parts of the stack are here:
System.IO.FileNotFoundException: Could not load file or assembly 'System.Security.Cryptography.Csp, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Security.Cryptography.Csp, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at FlexCharging.EnergyNet.AccountServerV3.GenerateRandomToken()

Configuration

This is a .NET 5 app running on Windows Server 2019 (an Azure Hyper-V image of Windows). WinVer says Version 1809, build 17763.2183.

Here's my .NET installation status. Tell me if there's some dotnet --downloadAllMissing.NETTeamPackages option I should regularly run, or a dotnet --deleteObsolete.NETCoreBuilds.

C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.401
 Commit:    4bef5f3dbf

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.401\

Host (useful for support):
  Version: 5.0.10
  Commit:  e1825b4928

.NET SDKs installed:
  5.0.401 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.19 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Regression?

Yes this worked earlier this week. Broken by either VS 16.11.3 or maybe .NET 5 build, version 5.0.10 perhaps?

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Hey guys, please help me out. My CLR loader knowledge is from 2012, and I need to be reeducated. I'm happy to swing by campus if you guys still eat lunch in Building 18 or 25.

One of my .NET 5 executables worked earlier this week, but broke now due to a FileNotFoundException for System.Security.Cryptography.Csp version 5. In the intervening time we installed the latest Windows Update and VS builds, and I am sure one of them included a new .NET 5 update, version 5.0.10. This may have broken it.

There is no version 5 of this assembly outside of .NET 5 internal "packages", as far as I can tell. There is a version in this directory, I guess it's the new GAC for .NET Core, but only for reference assemblies?
C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0

Note - if we have installed something like .NET 5.0.10, should that update these libraries? There is no 5.0.10 reference assembly directory.

I copied the reference assembly (which has no code!) to my application's directory and of course the loader was able to find the file. My app even seemed to work. (I changed the .NET Framework's loader to explicitly throw an exception if we stupidly loaded a reference assembly for execution, as opposed to the reflection-only loader context, but that is exactly what I'm doing and it doesn't fail. I guess my error check didn't translate over to .NET Core, or the loader is now more complicated than I think.) But my file copying hack doesn't seem like a smart thing to do. And I don't know if the code was returning something intelligent, vs. maybe null or just being a noop.

I looked at the System.Security.Cryptography.Csp Nuget package and added a reference to it in my project. Building this was disappointing for two reasons:

1. The assembly version number in the Nuget package is 4:2:1:0
2. VS did not copy the DLL to the output directory. Perhaps it did this because the Nuget package contains NET 4.6, 4.61 and 4.6.2 versions of this binary ONLY. No .NET Standard 2.1 build exists, nor a .NET 5 version.

My code:

        private static String GenerateRandomToken()
        {
            byte[] bytes = new byte[Constants.TokenLength];
            using (RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider())
                rng.GetNonZeroBytes(bytes);
            return Convert.ToBase64String(bytes);
        }

Relevant parts of the stack are here:
System.IO.FileNotFoundException: Could not load file or assembly 'System.Security.Cryptography.Csp, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Security.Cryptography.Csp, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at FlexCharging.EnergyNet.AccountServerV3.GenerateRandomToken()

Configuration

This is a .NET 5 app running on Windows Server 2019 (an Azure Hyper-V image of Windows). WinVer says Version 1809, build 17763.2183.

Here's my .NET installation status. Tell me if there's some dotnet --downloadAllMissing.NETTeamPackages option I should regularly run, or a dotnet --deleteObsolete.NETCoreBuilds.

C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.401
 Commit:    4bef5f3dbf

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.401\

Host (useful for support):
  Version: 5.0.10
  Commit:  e1825b4928

.NET SDKs installed:
  5.0.401 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.19 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Regression?

Yes this worked earlier this week. Broken by either VS 16.11.3 or maybe .NET 5 build, version 5.0.10 perhaps?

Author:	briangru
Assignees:	-
Labels:	area-System.Security, untriaged
Milestone:	-

[jeffschwMSFT]

cc @agocke

[bartonjs]

Aside from the obvious (inside joke of) replacing the entire body with throw new PrinterOnFireException();, things that can make the symptoms go away without actually identifying the problem:

        private static String GenerateRandomToken()
        {
            byte[] bytes = new byte[Constants.TokenLength];
-           using (RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider())
+           using (RandomNumberGenerator rng = RandomNumberGenerator.Create())
                rng.GetNonZeroBytes(bytes);
            return Convert.ToBase64String(bytes);
        }

Or, if you're targeting the .NET (nee Core) line specifically, you can get fancier:

        private static String GenerateRandomToken()
        {
            Span<byte> bytes = stackalloc byte[Constants.TokenLength];
            RandomNumberGenerator.Fill(bytes); // Of course, some zeroes can appear now.
            string ret = Convert.ToBase64String(bytes);
            bytes.Clear(); // if you care.
            return ret;
        }

[agocke]

I wouldn't try to work around this, because something fishy seems to be going on.

There shouldn't be anything fancy in assembly loading here -- the ref assembly should have a real runtime assembly that should go along with it in this case, no fancy type forwarding or facades.

Assuming you're using the shared runtime, the files in the .NET 5 shared runtime should be in C:\Program Files\dotnet\shared\Microsoft.NETCore.App\5.0.10\ directory. If I look in there I find System.Security.Cryptography.Csp.dll and it has assembly version 5.0.0 (patch releases don't change assembly version).

@briangru Can you check if that file is present on your machine as well? If it's not, I think your installation is just broken. If it is, your installation may still be broken, but it's more interesting. 😄

[agocke]

Ah, one more thing -- if you have a <appname>.deps.json next to your app that you can share, that may indicate the cause as well. If the assembly somehow got dropped, this might lead to failure to load.

[briangru]

Ah. I do have the right file in C:\Program Files\dotnet\shared\Microsoft.NETCore.App\5.0.10. That is a real binary with "real" IL instead of a reference assembly, and the version is correct. I do have a .deps.json file, which I'll attach here.
EnergyNetSchedulingServer.deps.ThisIsAJsonFile.txt

[briangru]

I will try one of Jeremy's suggestions, like maybe some span related code, or possibly replacing the entire method with a PrinterIsOnFireException. It would be nice to get to the bottom of this though.

Absolutely essential historical note: When Sun Microsystems was around in the 1990's, SunOS or Solaris defined several signals standard to Unix for interprocess communication from the command line to something running. These included SIG_HUP (for hang-up) and SIG_KILL (like sending a control-C, which can be caught & reacted to, like Console.Break). Sun also included a SIG_PRINTER_FIRE. PrinterIsOnFireException was obviously necessary for interoperability. I also added YouMoronException, because that just seemed necessary sometimes, and the ASP.NET team seemed to have used it.

And while I was not involved, to help out our PM, DateTime.Parse("Brad's Wedding") was added to ensure he didn't forget his anniversary.

[agocke]

Huh, this deps.json looks a lot like an app that was compiled for the desktop framework, not .NET 5. What's the TargetFramework in your project file?

[briangru]

It was originally compiled for .NET 4.5 or 4.6 in 2016, then upgraded to .NET 5.
net5.0

[agocke]

I see, in that case it may be an artifact of the porting that causes the problem. The best long-term strategy will be to take a look at your project file and make sure that everything's compatible with .NET 5 and that any extraneous NuGet references are removed. The deps.json file isn't supposed to be edited by hand, so if something is going wrong it's likely because the inputs to the SDK are somehow wrong. If you can share it, I might be able to eyeball it. Alternatively, if you can share a binlog I might be able to figure it out from that.

However, I'm curious about one more thing. I'd also expect a <app>.runtimeconfig.json next to your app, that should have content almost identical to the following:

{
  "runtimeOptions": {
    "tfm": "net5.0",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "5.0.0"
    }
  }
}

Do you have that as well?

[briangru]

The .runtimeconfig.json file exactly matches what you listed here.

There is one oddity - I am using some Nuget packages. I'm using PushSharp version 4.0.10, and I get a warning message saying it has been restored using a .NET Framework version between 4.6 and 4.8, but not net5.
There are other PushSharp Nuget packages, but it's not immediately obvious which to use. PushSharp.Core? PushSharp.NetCore2.1? PushSharp.NETStandard? Or PushSharpCore.Core? I realize that's my problem to solve, but you all may want to be aware of the level of fragmentation & confusion that got introduced into the Nuget package ecosystem. And while VS has a way of marking packages obsolete, there doesn't seem to be any good way of flagging "this package isn't what you should use for .NET 5 or higher", except the error message from VS above. And people seem to be on their own at guessing what the replacement is.

While this is something that needs to work at another level, have you considered a "Nuget package forwarder", even as a manual step so that VS can more aggressively detect & help people migrate? Kinda like the TypeForwardedToAttribute or a slightly more structured ObsoleteAttribute, only it would be on a Nuget package and work during Nuget package restoration time. No CLR loader support needed, but if done right, it might reduce your support burden by giving the ecosystem tools to succeed with less handholding.

[logicaloud]

Possibly related to #60144