JDK 21 toolchain #876

sgammon · 2025-01-06T04:39:43Z

Summary

This PR updates dependencies and build process for Pkl to bring GraalVM and Truffle support up to their latest versions. Today, Pkl builds against Java 17+, and requires Java 17+ to run.

After merging this PR, Pkl would build against Java 21, test against Java 21, and impose no change to lib consumers (i.e. still only requiring Java 17+ to run or transitively compile against Pkl).

Test downstream here (updated), via Github Actions. On that fork PR, the change set is expressed in full, with fixups.

PR Tree

chore: upgrade build tooling → jdk21 pkl-project-commons#6
fix: downstream native-image builds #914 (split out into independent PR)
Kotlin ABI checks #919 (split out into independent PR)
JSR305 integration for kotlinc #920 (split out into independent PR)

Rationale

Gradle Toolchains: Uses Gradle's toolchains feature to provision and use different versions of the JDK.
JDK 21 Toolchain: Builds (and tests) occur under JDK 21, with bytecode targeting JVM 17. Build bytecode adopts JVM 21 to accelerate build tooling with no change to lib consumers.
Truffle SVM Dependency: Certain superclasses used by Pkl (notably, AbstractTruffleException and TruffleFeature) have moved to the new org.graalvm.nativeimage:truffle-runtime-svm coordinate.
Multi-JDK testing: Adds stronger guarantees for backward compatibility with regard to Pkl and JVM execution (i.e. java -jar ... with a fat JAR). Tests against versions 18-23 with test suites and java -jar ... executions.

Known Issues

Failing Tests (pkl-gradle)
Exports on JVM17: Running against newer Truffle on JVM17 may need --add-exports=...

Pre-merge Checklist

Refresh Gradle lockfiles and review

Changelog

feat: support for jvm21+ toolchain
feat: support for gradle toolchains
feat: pass -PnativeArch=native to build with -march=native
test: multi-jdk testing support
test: support for jvm-test-suite plugin
test: add tasks to run jpkl eval on multiple jdks
test: make jdk exec tests respect multi-jdk flags and ranges
fix: remove mrjar classes at >jvm17 from fatjars
fix: use jdk21 to run the tests (needed for Unsafe.ensureInitialized)
fix: truffle svm dependency is required after graalvm 24.0.0
fix: warnings for gvm flag usage, renamed truffle svm macro
fix: build with --add-modules=jdk.unsupported where needed
fix: don't use gu tool for modern graalvm versions
fix: catch Throwable instead of deprecated-for-removal ThreadDeath
chore: buildinfo changes for JVM targets, toolchains
chore: enforce testing at exactly jdk21
chore: enforce build tooling at jdk21+
chore: bump graalvm/truffle libs → 24.1.2
chore: toolchains for buildSrc

buildSrc/src/main/kotlin/BuildInfo.kt

buildSrc/src/main/kotlin/InstallGraalVm.kt

gradlew

pkl-cli/pkl-cli.gradle.kts

pkl-core/src/test/files/LanguageSnippetTests/input/errors/analyzeInvalidHttpModule.inert

pkl-executor/pkl-executor.gradle.kts

bioball · 2025-01-06T17:21:27Z

This is great, thanks, Sam!

I will do a review of this by next week.

FYI: Pkl today supports Java 21. The range enabled here is Java 22+ (updated your PR title).

settings.gradle.kts

StefMa · 2025-01-06T20:20:52Z

@sgammon feel free to extract the toolchain support into another PR. Indeed, my PR is quited dated. I will close it...

sgammon · 2025-01-06T20:31:18Z

@StefMa roger, no worries :) I am a well known offender in that regard

bioball

Thanks for this!

Also, general note: we'll need to set up our CI to use Java 21 to compile (for various compileJava tasks), and ideally to test with all of JDK 17/21/22+. I actually think toolchains probably makes sense for this, so, I'd be okay with the toolchain changes coming back into this PR.

With toolchains, maybe the version of Java used for tasks.test can come from a build flag.

buildSrc/src/main/kotlin/pklJavaLibrary.gradle.kts

pkl-core/src/main/resources/META-INF/native-image/org.pkl-lang/native-image.properties

gradlew

buildSrc/src/main/kotlin/pklFatJar.gradle.kts

gradle/libs.versions.toml

pkl-cli/pkl-cli.gradle.kts

buildSrc/src/main/kotlin/pklFatJar.gradle.kts

pkl-cli/pkl-cli.gradle.kts

buildSrc/src/main/kotlin/BuildInfo.kt

buildSrc/src/main/kotlin/pklJavaLibrary.gradle.kts

buildSrc/src/main/kotlin/pklKotlinLibrary.gradle.kts

settings.gradle.kts

buildSrc/src/main/kotlin/pklFatJar.gradle.kts

settings.gradle.kts

buildSrc/src/main/kotlin/pklJvmToolchain.gradle.kts

pkl-cli/pkl-cli.gradle.kts

pkl-core/src/test/kotlin/org/pkl/core/LanguageSnippetTestsEngine.kt

pkl-executor/pkl-executor.gradle.kts

buildSrc/src/main/kotlin/pklJavaLibrary.gradle.kts

buildSrc/src/main/kotlin/BuildInfo.kt

pkl-commons/gradle.lockfile

pkl-core/src/test/kotlin/org/pkl/core/LanguageSnippetTestsEngine.kt

buildSrc/src/main/kotlin/BuildInfo.kt

buildSrc/build.gradle.kts

buildSrc/src/main/kotlin/BuildInfo.kt

pkl-cli/gradle.lockfile

pkl-cli/pkl-cli.gradle.kts

sgammon · 2025-02-02T00:49:03Z

@bioball This is ready for final review again :) I've left a self-review detailing the cleanups I've applied. Thank you for your patience while I figured out that GVM 17 bug.

bioball

Thanks, Sam! This looks really good. Thanks for your help here!

Can you also do the following:

Update the .java-version file to 21
Update DEVELOPMENT.adoc instructions (change mentions of Java 17 to 21)
Add -DpklMultiJdkTesting=true to our CircleCI definition. Should be okay to just add "-DpklMultiJdkTesting=true" to fixed gradleArgs in GradleJob.pkl.

buildSrc/src/main/kotlin/BuildInfo.kt

sgammon · 2025-02-03T21:51:25Z

@bioball No worries :) changes applied!

- feat: support for jvm21+ toolchain - feat: support for gradle toolchains - feat: pass `-PnativeArch=native` to build with `-march=native` - test: multi-jdk testing support - test: support for `jvm-test-suite` plugin - test: add tasks to run `jpkl eval` on multiple jdks - test: make jdk exec tests respect multi-jdk flags and ranges - fix: remove mrjar classes at >jvm17 from fatjars - fix: use jdk21 to run the tests (needed for `Unsafe.ensureInitialized`) - fix: truffle svm dependency is required after graalvm `24.0.0` - fix: warnings for gvm flag usage, renamed truffle svm macro - fix: build with `--add-modules=jdk.unsupported` where needed - fix: don't use `gu` tool for modern graalvm versions - fix: catch `Throwable` instead of deprecated-for-removal `ThreadDeath` - chore: buildinfo changes for JVM targets, toolchains - chore: enforce testing at exactly jdk21 - chore: enforce build tooling at jdk21+ - chore: bump graalvm/truffle libs → `24.1.2` - chore: toolchains for `buildSrc` Signed-off-by: Sam Gammon <[email protected]>

- chore: update to jdk21 in circle jobs - chore: re-gen circle jobs Signed-off-by: Sam Gammon <[email protected]>

bioball

LGTM, solid work, thank you!

Checksums taken during apple#876 ended up out of date due to the release of GraalVM `21.1.2` (the PR was built against `21.1.1`). These checksums are calculated from the GraalVM download page directly: https://www.graalvm.org/downloads/ On Linux: ``` sha256sum graalvm-... ``` Signed-off-by: Sam Gammon <[email protected]>

sgammon force-pushed the feat/jvm21plus-upstream branch from 8c2a05f to dc45756 Compare January 6, 2025 04:52

sgammon mentioned this pull request Jan 6, 2025

[WIP] feat: java/graalvm 21+ support elide-dev/pkl#3

Draft

13 tasks

sgammon changed the title ~~feat: jvm 21+ support~~ JVM 21+ support Jan 6, 2025

sgammon mentioned this pull request Jan 6, 2025

Upstream: Build Upgrades #204

Closed

sgammon commented Jan 6, 2025

View reviewed changes

bioball changed the title ~~JVM 21+ support~~ JVM 22+ support Jan 6, 2025

bioball reviewed Jan 6, 2025

View reviewed changes

settings.gradle.kts Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

StefMa mentioned this pull request Jan 6, 2025

Add Gradle toolchain support #262

Closed

sgammon force-pushed the feat/jvm21plus-upstream branch from dc45756 to 1982880 Compare January 6, 2025 22:00

This comment was marked as outdated.

Sign in to view

bioball requested changes Jan 21, 2025

View reviewed changes

sgammon changed the title ~~JVM 22+ support~~ JDK 21+ toolchain Jan 22, 2025

sgammon force-pushed the feat/jvm21plus-upstream branch from 1982880 to a314e92 Compare January 22, 2025 06:12

sgammon commented Jan 22, 2025

View reviewed changes

sgammon requested a review from bioball January 22, 2025 06:26

This comment was marked as outdated.

Sign in to view

sgammon changed the title ~~JDK 21+ toolchain~~ JDK 21 toolchain Jan 22, 2025

This comment was marked as outdated.

Sign in to view

sgammon commented Jan 22, 2025

View reviewed changes

buildSrc/src/main/kotlin/pklFatJar.gradle.kts Show resolved Hide resolved

sgammon force-pushed the feat/jvm21plus-upstream branch from f9daed3 to 263588b Compare January 23, 2025 05:19

This comment was marked as outdated.

Sign in to view

sgammon mentioned this pull request Jan 23, 2025

chore: upgrade build tooling → jdk21 apple/pkl-project-commons#6

Merged

bioball reviewed Jan 23, 2025

View reviewed changes

settings.gradle.kts Outdated Show resolved Hide resolved

bioball reviewed Jan 23, 2025

View reviewed changes

buildSrc/src/main/kotlin/pklJvmToolchain.gradle.kts Outdated Show resolved Hide resolved

bioball reviewed Jan 23, 2025

View reviewed changes

sgammon force-pushed the feat/jvm21plus-upstream branch 3 times, most recently from 8ad9dd5 to 465337a Compare January 29, 2025 00:05

This comment was marked as outdated.

Sign in to view

sgammon force-pushed the feat/jvm21plus-upstream branch from 465337a to 5f98585 Compare January 30, 2025 22:06

This comment was marked as outdated.

Sign in to view

sgammon force-pushed the feat/jvm21plus-upstream branch 2 times, most recently from 69750d3 to d6a8801 Compare January 31, 2025 03:18

bioball reviewed Jan 31, 2025

View reviewed changes

sgammon force-pushed the feat/jvm21plus-upstream branch from d6a8801 to e85b259 Compare January 31, 2025 22:22

This comment was marked as resolved.

Sign in to view

sgammon force-pushed the feat/jvm21plus-upstream branch from e85b259 to 8d9c22c Compare February 2, 2025 00:21

sgammon commented Feb 2, 2025

View reviewed changes

sgammon force-pushed the feat/jvm21plus-upstream branch from 8d9c22c to a14ef41 Compare February 2, 2025 00:47

sgammon requested a review from bioball February 2, 2025 00:48

sgammon mentioned this pull request Feb 2, 2025

Kotlin ABI checks #919

Open

7 tasks

bioball requested changes Feb 3, 2025

View reviewed changes

buildSrc/src/main/kotlin/BuildInfo.kt Show resolved Hide resolved

sgammon force-pushed the feat/jvm21plus-upstream branch from a14ef41 to 596aa0a Compare February 3, 2025 21:51

sgammon requested a review from bioball February 3, 2025 21:51

sgammon added 2 commits February 3, 2025 13:53

chore: update circleci jobs for multi-jdk testing

87a03af

- chore: update to jdk21 in circle jobs - chore: re-gen circle jobs Signed-off-by: Sam Gammon <[email protected]>

sgammon force-pushed the feat/jvm21plus-upstream branch from 596aa0a to 87a03af Compare February 3, 2025 21:53

bioball approved these changes Feb 3, 2025

View reviewed changes

bioball merged commit 408242a into apple:main Feb 3, 2025
3 checks passed

sgammon mentioned this pull request Feb 3, 2025

fix: upgrade ci jobs to jdk 21 #926

Merged

sgammon mentioned this pull request Feb 4, 2025

fix: native gvm ci #929

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JDK 21 toolchain #876

JDK 21 toolchain #876

sgammon commented Jan 6, 2025 •

edited

Loading

bioball commented Jan 6, 2025

This comment was marked as outdated.

StefMa commented Jan 6, 2025

sgammon commented Jan 6, 2025

This comment was marked as outdated.

This comment was marked as outdated.

bioball left a comment

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as resolved.

sgammon commented Feb 2, 2025

bioball left a comment •

edited

Loading

sgammon commented Feb 3, 2025

bioball left a comment

JDK 21 toolchain #876

JDK 21 toolchain #876

Conversation

sgammon commented Jan 6, 2025 • edited Loading

Summary

PR Tree

Rationale

Known Issues

Pre-merge Checklist

Changelog

bioball commented Jan 6, 2025

This comment was marked as outdated.

StefMa commented Jan 6, 2025

sgammon commented Jan 6, 2025

This comment was marked as outdated.

This comment was marked as outdated.

bioball left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as resolved.

sgammon commented Feb 2, 2025

bioball left a comment • edited Loading

Choose a reason for hiding this comment

sgammon commented Feb 3, 2025

bioball left a comment

Choose a reason for hiding this comment

sgammon commented Jan 6, 2025 •

edited

Loading

bioball left a comment •

edited

Loading