Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky TransparentCompiler fuzzing test #18050

Open
majocha opened this issue Nov 22, 2024 · 3 comments
Open

Flaky TransparentCompiler fuzzing test #18050

majocha opened this issue Nov 22, 2024 · 3 comments
Milestone

Comments

@majocha
Copy link
Contributor

majocha commented Nov 22, 2024

Recently reenabled fuzzing test still needs some work:

[<Theory>]
[<InlineData(SignatureFiles.Yes)>]
[<InlineData(SignatureFiles.No)>]
[<InlineData(SignatureFiles.Some)>]
let Fuzzing signatureFiles =

It is sporadically failing in CI.

It takes a lot of iterations locally to trip it, but it is possible to repro.

A good way to get a repro locally, (and to stress test a selected test case in general) is to define a xUnit attrbute like this

type RepeatAttribute(count, [<ParamArray>] args: obj array) =
    inherit DataAttribute()
    override _.GetData _ = Seq.init count (fun i -> [| yield! args; i |])

and apply it like this:

[<Theory>]
[<Repeat(100, SignatureFiles.Yes)>]
[<Repeat(100, SignatureFiles.No)>]
[<Repeat(100, SignatureFiles.Some)>]
let Fuzzing signatureFiles _ =

Running the theory in parallel it can be repro'd in around 100 iterations.
Out of curiosity I tried to run this test with useTransparentCompiler = false and it also fails from time to time. It is possible that it reveals some concurrency issue unrelated to TransparentCompiler.

@0101
Copy link
Contributor

0101 commented Nov 22, 2024

Let's skip it for now then.

Out of curiosity I tried to run this test with useTransparentCompiler = false and it also fails from time to time. It is possible that it reveals some concurrency issue unrelated to TransparentCompiler.

That's quite interesting, does it fail with the same error?

@majocha
Copy link
Contributor Author

majocha commented Nov 22, 2024

That's quite interesting, does it fail with the same error?

I thought so, but I'm not so sure now as I can't get that result anymore.

@0101 0101 mentioned this issue Nov 22, 2024
@majocha
Copy link
Contributor Author

majocha commented Nov 22, 2024

Yes, occasional Expected no errors, but there were some:
when useTransparentCompiler = false in fuzzingTest

Assert.Equal() Failure: Collections differ
Expected: []
Actual:   [Tuple (AbortedChecking
  "Expected no errors, but there were some: 
[|E:\repos\fsharp\artifacts\bin\FSharp.Compiler.ComponentTests\Release\net9.0\test-projects\TestProject_680b94c6\FileF012.fs (3,5)-(3,6) typecheck error Module 'TestProject_680b94c6.ModuleF012' contains
    val f: x: 'a -> (ModuleF001.TF001V_1<'a> * 'a0 * ModuleF004.TF004V_1<'a>) * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * ModuleF006.TF006V_1<'a>) * (ModuleF001.TF001V_1<'a> * ((ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * 'b) * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * 'c) * ModuleF004.TF004V_1<'a>) * ModuleF007.TF007V_1<'a>) * ModuleF010.TF010V_1<'a>) * TF012V_1<'a>    
but its signature specifies
    val f: x: 'a -> (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * 'a0) * ModuleF004.TF004V_1<'a>) * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * ModuleF006.TF006V_1<'a>) * (ModuleF001.TF001V_1<'a> * ((ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * 'b) * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * (ModuleF001.TF001V_1<'a> * ModuleF002.TF002V_1<'a>) * 'c) * ModuleF004.TF004V_1<'a>) * ModuleF007.TF007V_1<'a>) * ModuleF010.TF010V_1<'a>) * TF012V_1<'a>    
The types differ|]", 1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: New
Development

No branches or pull requests

3 participants