Unwanted duplicate packages for Intel unified-dev environment on Orion #1477
Comments
|
I was hitting this a lot when working on orion (and hercules), with
|
|
Thanks for the tip @rickgrubin-noaa! I poked around a bit and at least part of the issue is that the Here's the spack-stack/configs/common/packages.yaml Lines 251 to 253 in ece9d20 And here's the excerpt from the esmf There doesn't appear to be any restriction on the py-numpy version in the esmf Any thoughts? |
|
@mathomp4, @climbfuji, @rickgrubin-noaa, @alexrichert any thoughts about the behavior I mentioned above? Thanks! |
|
Also noticed that intel MKL is not used in the Orion intel configuration. Is this correct? Not sure MKL would be related to this issue, but thought I should double check on this. Thanks! |
That's because NOAA doesn't want to move (back) to MKL from openblas. it used to be MKL with hpc-stack, but when we moved to spack-stack we had openblas first (due to issues with mkl in the stack in the early days of spack/spack-stack). And now it seems we are stuck with openblas on the RDHCPS systems (until someone actually tests MKL vs openblas and finds out that it is safe to switch back). |
I don't know how this can happen, unless a |
Here's what I learned this week when asking the same question: It boils down to what |
|
Thanks for the info. Let's keep openblas on the RDHPCS platforms (and continue to carry around all the special handling in the site configs). |
Building results in no duplicates. |
|
Ahh of course, this makes sense. We had to use |
|
This works for my case too! Thanks @rickgrubin-noaa! I'm seeing only the expected duplicates now: I'll create a PR for the Orion Intel config. Should that be based on the release/1.9.0 branch? |
|
Yes please. I guess we need the same change for each and every RDHPCS system that has an override of the py-numpy requirements in the site config ( @rickgrubin-noaa @AlexanderRichert-NOAA Since we expect b4b differences for the UFS when switching to the "full" OneAPI compilers ( |
|
It's a question for UWM devs, I have little preference (it's one more source of possible numerical differences that will get investigated by devs from time to time, but I don't mind if the devs don't). How was UWM getting linked to MKL previously? |
In hpc-stack, I don't recall the details. These were all manually configured build scripts. |
|
PR #1482 resolved this issue, so I will close as completed. |
Describe the bug
The intel unified-dev environment for the Intel compiler set is producing unwanted duplicates in the concretize step. Here is the output of show_duplicate_packages.py
I'm getting this from the feature branches for the libirc PR #1435. I'm also seeing similar behavior from the Intel build the @RatkoVasic-NOAA recently did:
There seems to be an issue with skylake vs skylake-avx512 architecture.
It's not clear if the PR #1435 introduced this behavior. I suspect not and this behavior has likely been around for a while. I don't think this issue should hold up PR #1435.
I think this is a specific Orion/Intel configuration issue meaning that we should not hold up the 1.9.0 installation and testing on other platforms.
To Reproduce
Follow the instructions for building a local environment here: https://spack-stack.readthedocs.io/en/latest/PreConfiguredSites.html#create-local-environment, and select orion, unified-dev, and intel compiler.
Use the feature branches from #1435, but I suspect you can use the 1.9.0 release branches or the develop braches as well.
Then run
spack concretizeorspack concretize --fresh.Expected behavior
The concretize step should only produce the expected duplicates (esmf, crtm)
System:
What system(s) are you running the code on?
Orion, Intel unified-dev environment
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: