Improve performance of weeder when type-class-roots = false is set.

[pranaysashank]

For analysing evidence uses we collect evidence uses in,

requestedEvidence :: Map Declaration (Set Name)

In analyseEvidenceUses, we loop over all the names in all the sets of the map, to construct dependency graph after calling getEvidenceTree on the name. However, these names in sets across different declarations are duplicated a lot. In one example in a repo at work, we have 16961625 names in which only 200330 are unique. So now, we instead pre-construct an evidence trees map Map Name [Declaration] for all the unique name and perform a lookup in this map to construct the graph.

In a private repo, the times before this change and after

❯ find . -name '*.hie'  | wc -l
   1097

❯ time result/bin/weeder # weeder from master

real    5m53.707s
user    5m50.350s
sys     0m2.206s

 ❯ time result/bin/weeder # weeder from this branch

real    0m34.008s
user    0m31.716s
sys     0m2.196s

I have a failing test in ./test/Spec/ModuleRoot, but this is failing on master as well.

[ocharles]

Wow, what an incredible speed up! I'll take a look! @ryndubei could you also have a look if you have time?

[ryndubei]

Thanks! I don't have a large enough project on hand at the moment that I can quickly test the performance with, but the changes look good to me. I only have a few minor, mostly-style-related comments.

[pranaysashank]

@ryndubei I addressed all of your comments

one minor thing I have noticed is there seems to be a slight performance loss with the usage of foldMap & Map.fromSet. Core shows that fromSet doesn't get inlined and foldMap generates recursive code with (++) operators instead as opposed to when we hand roll (concat (map (declMap Map.!) (Set.toList v))) & use Set.foldl'. The same is the case with fromSet (turns out even foldl' has the same O(n) complexity). A quick look at fromSet shows it doesn't have the inline pragma or the usual haskell trick where the recursion is extracted to a separate go function.

Since the performance impact only seems to be at most 1-2 seconds, I think we can prefer code clarity

[pranaysashank]

also, why don't we link the executable with -rtsopts? it's useful to change some parameters to figure out how the performance characteristics change or limit the memory used with +RTS -M

[ocharles]

Before...

[root@internal:~]# time nix-store -r /nix/store/sc5pfnj0la...-weeder.drv --builders '' --check
real	6m1.956s
user	6m5.595s
sys	0m2.792s

After

[root@internal:~]# time nix-store -r /nix/store/kip37...-weeder.drv --builders '' --check
real	3m5.243s
user	3m11.471s
sys	0m3.609s

Hard to argue with that!

Thanks for you hard work, @pranaysashank!