"Failed to locate Java for JDK adopt:1.12.0.2" error using courier v2.1.6 on ARM mac

[huonw]

Describe the bug

Using Pants 2.18.0 in its default configuration in a JVM repo on macOS arm64 seems to hit errors like:

ValueError: Failed to locate Java for JDK `adopt:1.12.0.2`:
JVM adopt:1.12.0.2 not found in index: JVM adopt not found
/bin/bash: /bin/java: No such file or directory

Git bisect narrows it down to #19940. It appears to be Coursier v2.1.6 specifically: reverting back to the previous default version works fine.

[coursier]
version = "v2.1.0-M5-18-gfebf9838c"

Minimal reproducer:

cd $(mktemp -d)

cat > pants.toml <<EOF
[GLOBAL]
pants_version = "2.18.0"
backend_packages = [
  "pants.backend.experimental.java",
]
EOF

echo 'junit_tests(name="tests", jdk="adopt:1.12.0.2")' > BUILD
touch ExampleTest.java

# BUG: with default coursier (v2.1.6)
pants test ::

cat >> pants.toml <<EOF
[coursier]
version = "v2.1.0-M5-18-gfebf9838c"
EOF

# "OKAY" (same behaviour as 2.17): with original coursier
pants test ::

With 2.18.0, this prints (with my comments)

# BUG:
09:32:53.98 [ERROR] 1 Exception encountered:

Engine traceback:
  in `test` goal

ValueError: Failed to locate Java for JDK `adopt:1.12.0.2`:
JVM adopt:1.12.0.2 not found in index: JVM adopt not found
/bin/bash: /bin/java: No such file or directory

# "OKAY":
09:32:55.07 [ERROR] 1 Exception encountered:
...
IntrinsicError: Unmatched glob from The resolve `jvm-default` from `[jvm].resolves`: "3rdparty/jvm/default.lock"
...

NB. The "OKAY" test still has an error because this repo isn't configured to actually run these tests, but the problem is demonstrated earlier enough. Running that same code with 2.17.0 gives the second error for both runs.

Pants version

2.18.0

OS
macOS

Additional info

https://pantsbuild.slack.com/archives/C0D7TNJHL/p1699843811602729?thread_ts=1699829181.582269&cid=C0D7TNJHL

[gauthamnair]

Filed issue VirtusLab/coursier-m1#79 with Virtus labs, made a PR #20179 to remove 2.1.6 support in pants.

I don't think this will be otherwise fixable in time. Apologies again for the shallow testing the first time around. The VirtustLabs M1 coursier is able to do other functions like resolve and fetch dependencies (which I took as sufficient indication that it was working), but the jvm downloading function appears to be currently broken.

[gauthamnair]

Ok, so slight false alarm here. The actual issue is that the adopt jdk is not available for M1 for the versions we ask for, not that the M1 coursier 2.1.6 is broken.

Changing the test to be:

scalatest_tests(
    # These tests are `parametrize`d, so that they will run against two different
    # JDK versions.
    jdk=parametrize(temurin_11="temurin:1.11", temurin_17="temurin:1.17"),
)

makes them pass on my m1 against pants_version = "2.18.0" with:

✓ tests/jvm/org/pantsbuild/example/lib/ExampleLibSpec.scala@jdk=temurin_11 succeeded in 0.47s.
✓ tests/jvm/org/pantsbuild/example/lib/ExampleLibSpec.scala@jdk=temurin_17 succeeded in 0.38s.

It may be that all we need to do is change the JDKs requested in example-jvm sot that it is compatible with all platforms.

[gauthamnair]

You can see the behavior here, using the new default in main on an M1 and comparing to the old default in pants main:

[Exit:1] coursier_dls $ ./cs-aarch64-apple-darwin-2.1.6-Virtus java --jvm adopt:1.11 -version
JVM adopt:1.11 not found in index: JVM adopt not found


[Exit:1] coursier_dls $ ./cs-aarch64-apple-darwin-2.1.6-Virtus java --jvm temurin:1.11 -version
openjdk version "11.0.21" 2023-10-17
OpenJDK Runtime Environment Temurin-11.0.21+9 (build 11.0.21+9)
OpenJDK 64-Bit Server VM Temurin-11.0.21+9 (build 11.0.21+9, mixed mode)


coursier_dls $ ./cs-x86_64-apple-darwin-2.1.0-M5-18-gfebf9838c java --jvm adopt:1.11 -version
openjdk version "11" 2018-09-25
OpenJDK Runtime Environment AdoptOpenJDK (build 11+28)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11+28, mixed mode)


coursier_dls $ ./cs-x86_64-apple-darwin-2.1.0-M5-18-gfebf9838c java --jvm temurin:1.11 -version
openjdk version "11.0.21" 2023-10-17
OpenJDK Runtime Environment Temurin-11.0.21+9 (build 11.0.21+9)
OpenJDK 64-Bit Server VM Temurin-11.0.21+9 (build 11.0.21+9, mixed mode)

The reason we did not get build failures in pants itself I think is because we are set up to use temurin:1.11 by default, which is available for all platforms.

pants/src/python/pants/jvm/subsystems.py

Line 51 in e70fe05

default="temurin:1.11",

Here are the lists of available versions for M1:

$ ./cs-aarch64-apple-darwin-2.1.6-Virtus java --available | grep 1.11
adoptium:1.11.0.15
...
temurin:1.11.0.15
...

but on x86 you have:

$ ./cs-x86_64-apple-darwin-2.1.0-M5-18-gfebf9838c java --available | grep 1.11
adopt:1.11
...
adoptium:1.11.0.12
...
temurin:1.11.0.12

@benjyw @huonw I think we can change the jvm versions in the example-jvm repo and we are fine with the existing pants 2.18.0.

[gauthamnair]

Big thanks to the Virtuslab maintainer

[huonw]

Is this fixed by the combination of these?

• Upgrade to pants 2.18.0 and M1-compatible jdk example-jvm#29
• pants: upgrade Pants to 2.18.1 example-kotlin#15
• Use a Zulu JDK 8 to run the Kotlin analyzer. #20184

[gauthamnair]

@huonw I think so, let me try to convince myself by trying to argue for it.

Stepping back, the question is what is pants' spec here, rather than its behavior. There is a change in behavior which will manifest as a failure to build for some users who have set things up in a certain way, but is that a bug?

Proposed spec

• Pants strives to download and use tools that are native to the local cpu architecture because of the increase in performance.
• When a tool version is not available for your architecture, pants may choose either to:
  • quietly use a tool from another architecture known to work via emulation (Rosetta) as a stop-gap
  • or, if other variants (versions) of the tool exist with cross-platform support, fail the build. The user will be able to change the version.
  • The default versions specified by pants will not result in a failed build, when taken together.

The last bit contains an or and proposes that it is ok to break workflows that worked via emulation if it has become possible to have similar workflows work natively.

The specifics of coursier, jvm, kotlin.

Before adding the M1/arm coursier versions and bumping the default to v2.1.6 we were quietly using emulation for all of these:

• coursier itself
• any jdk requested by coursier
• including the pants-hard-coded jdk used for the kotlin analyzer

After #19940 (and #20121) which went into pants 2.18.0 we got:

• coursier default is M1
• and therefore requested jdks need to be available on M1. The user will see an error if they previously requested a jdk not available for M1, but they can change it via the options system or within targets. This is fine and within the proposed spec. It was working before via emulation, now broken, but the user can fix it and go truly cross-platform. example-jvm had to be changed in the same way as a user would have to adjust in their repo. The default jdk in pants JvmToolBase subsystem is temurin:1.11 and is already M1-compatible.
• Broken: The kotlin analyzer is broken. It was working via emulation but now the defaults fail. Moreover, the user had no ability to change the jdk used since it is hard-coded.

#20184 Fixed the Kotlin analyzer issue, and example-kotlin was also fixed, so now we should be ok.

[huonw]

Sounds good, thanks