8353786: Migrate Vector API math library support to FFM API #24462

iwanowww · 2025-04-04T22:52:24Z

Migrate Vector API math library (SVML and SLEEF) linkage from native code (in JVM) to Java FFM API.

Since FFM API doesn't support vector calling conventions yet, migration affects only symbol lookup for now. But it still enables significant simplifications on JVM side.

The patch consists of the following parts:

on-demand symbol lookup in Java code replaces eager lookup from native code during JVM startup;
2 new VM intrinsics for vector calls (support unary and binary shapes) (code separated from unary/binary vector operations);
new internal interface to query supported CPU ISA extensions (jdk.incubator.vector.CPUFeatures) used for CPU dispatching.

java.lang.foreign API is used to perform symbol lookup in vector math library, then the address is cached and fed into corresponding JVM intrinsic, so C2 can turn it into a direct vector call in generated code.

Once java.lang.foreign supports vectors & vector calling conventions, VM intrinsics can go away.

Performance is on par with original implementation (tested with microbenchmarks on linux-x64 and macosx-aarch64).

Testing: hs-tier1 - hs-tier6, microbenchmarks (on linux-x64 and macosx-aarch64)

Thanks!

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8353786: Migrate Vector API math library support to FFM API (Enhancement - P3)

Reviewers

Vladimir Kozlov (@vnkozlov - Reviewer) Review applies to bb1a11db
Paul Sandoz (@PaulSandoz - Reviewer) Review applies to a288cbbf
Xiaohong Gong (@XiaohongGong - Committer) Review applies to 88eacc48
Jorn Vernee (@JornVernee - Reviewer) Review applies to 88eacc48
Hamlin Li (@Hamlin-Li - Reviewer) Review applies to 585312ae
Jatin Bhateja (@jatin-bhateja - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24462/head:pull/24462
$ git checkout pull/24462

Update a local copy of the PR:
$ git checkout pull/24462
$ git pull https://git.openjdk.org/jdk.git pull/24462/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 24462

View PR using the GUI difftool:
$ git pr show -t 24462

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24462.diff

Using Webrev

Link to Webrev Comment

bridgekeeper · 2025-04-04T22:53:18Z

👋 Welcome back vlivanov! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-04-04T22:53:50Z

@iwanowww This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8353786: Migrate Vector API math library support to FFM API

Reviewed-by: jbhateja, kvn, psandoz, xgong, jvernee, mli

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 144 new commits pushed to the master branch:

4b88029: 8355439: Some hotspot/jtreg/serviceability/sa/* tests fail on static JDK due to explicit checks for shared libraries in process memory map
d8f012e: 8305186: Reference.waitForReferenceProcessing should be more accessible to tests
ac05002: 8354877: DirectClassBuilder default flags should include ACC_SUPER
... and 141 more: https://git.openjdk.org/jdk/compare/4eae9b5ba61bfe262b43346a7499c98c1a54d2fe...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-04-04T22:54:18Z

@iwanowww
The core-libs label was successfully added.

The hotspot-compiler label was successfully added.

liach · 2025-04-04T23:27:11Z

Moving vector API library selection to Java code looks like a right step to me.

iwanowww · 2025-04-05T02:19:06Z

/cc hotspot

openjdk · 2025-04-05T02:20:13Z

@iwanowww
The hotspot label was successfully added.

mlbridge · 2025-04-05T02:42:36Z

Webrevs

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java

vnkozlov

Few questions?

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/CPUFeatures.java

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/CPUFeatures.java

XiaohongGong

Looks good to me. Thanks for your updating!

JornVernee

Very interesting! Looks mostly good to me. Left a few inline notes.

JornVernee · 2025-04-18T16:09:32Z

src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java

+    <V extends VectorPayload, E>
+    V libraryBinaryOp(long addr, Class<? extends V> vClass, Class<E> eClass, int length, String debugName,
+                      V v1, V v2,
+                      BinaryOperation<V,?> defaultImpl) {


I notice that the bound of V differs between libraryUnaryOp, which uses Vectory<E> and this method, which uses VectorPayload. Not sure if this is intentional?

JornVernee · 2025-04-18T16:16:28Z

src/hotspot/share/prims/vectorSupport.cpp

+  ThreadToNativeFromVM ttn(thread);
+  return env->NewStringUTF(features_string);


Isn't there a way to do this without the extra transition?

How about:

oop result = java_lang_String::create_oop_from_str((char*) bytes, CHECK_NULL); return (jstring) JNIHandles::make_local(THREAD, result);

Fair enough.

JornVernee · 2025-04-18T16:53:50Z

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java

+
+    @ForceInline
+    /*package-private*/ static
+    <E, V extends Vector<E>>


Here you're using Vector instead of VectorPayload for the binary op, so there seems to be a discrepancy with VectorSupport.

I don't have a strong preference, but I kept it aligned with unaryOp/binaryOp intrinsics.

JornVernee · 2025-04-18T17:01:31Z

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java

+            try {
+                MemorySegment addr = LOOKUP.findOrThrow(symbol);
+                debug("%s %s => 0x%016x\n", op, symbol, addr.address());
+                T impl = implSupplier.apply(opc); // TODO: should call the very same native implementation eventually (once FFM API supports vectors)


FWIW, one current barrier I see to implementing the vector calling convention in the linker, is that the FFM linker (currently) transmits register values to the downcall stub use Java primitive types. So, in order to support vector calling conventions, we would need to add some kind of 'primitive' that can hold the entire vector value, and preferably gets passed in the right register.

However, I think in the case of these math libraries in particular, speed of the fallback implementation is not that much of an issue, since there is also an intrinsic. So alternatively, we could split a vector value up into smaller integral types (int, long) -> pass them to the downcall stub in that form -> and then reconstruct the full vector value in its target register. (I used the same trick when I was experimenting with FP80 support, which also requires splitting up the 80 bit value up into 2 longs).

IMO an in-memory representation for vectors is preferred when it comes to FFM linker calling conventions. 512-bit vector requires 8 longs, so some of them will end up passed on stack for any non-trivial case. And with in-memory representation, VM can elide vector store/load once FFM linker stub is intrinsified.

luhenry · 2025-04-22T14:46:21Z

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java

+        static String getDefaultName() {
+            return switch (StaticProperty.osArch()) {
+                case "amd64", "x86_64" -> SVML;
+                case "aarch64" -> SLEEF;


We should be supporting SLEEF on riscv64. Was there a specific motivation not to include it here?

Good catch, fixed.

Hamlin-Li

I just ran some basic tests on riscv, seems there are some issues, also have some comments.

Hamlin-Li · 2025-04-22T13:45:26Z

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/CPUFeatures.java

+    public static Set<String> features() {
+        return features;
+    }
+}


Maybe an extra line needed at the end of this file?

Hamlin-Li · 2025-04-22T16:34:21Z

src/hotspot/share/runtime/abstract_vm_version.cpp

+                                                         size_t features_offset) {
+  assert(features_offset <= cpu_info_string_len, "");
+  if (features_offset < cpu_info_string_len) {
+    assert(cpu_info_string[features_offset + 0] == ',', "");


This assert fails on riscv.

A simple fix could be:

diff --git a/src/hotspot/os_cpu/linux_riscv/vm_version_linux_riscv.cpp b/src/hotspot/os_cpu/linux_riscv/vm_version_linux_riscv.cpp index 484a2a645aa..a785dc65c9e 100644 --- a/src/hotspot/os_cpu/linux_riscv/vm_version_linux_riscv.cpp +++ b/src/hotspot/os_cpu/linux_riscv/vm_version_linux_riscv.cpp @@ -196,25 +196,12 @@ void VM_Version::setup_cpu_available_features() { _cpu_info_string = os::strdup(buf); - _features_string = extract_features_string(_cpu_info_string, - strnlen(_cpu_info_string, sizeof(buf)), - features_offset); + _features_string = _cpu_info_string; }

Alternatively, it's fine for now to completely drop extract_features_string call on linux-riscv (as on some other platforms) and fix it separately. Then VectorSupport.getCPUFeatures() returns empty string. VectorMathLibrary doesn't rely on CPUFeatures on RISC-V.

Let me know how you prefer to handle it.

I think we still need this or similar thing on riscv. Please check my new comments below about rvv extension on riscv.
On the other hand, it's also good to have it on riscv for consistency, and there is a log output of "cpu features" in VectorMathLibrary.java

Hamlin-Li · 2025-04-22T16:37:43Z

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/CPUFeatures.java

+    private static Set<String> getCPUFeatures() {
+        String featuresString = VectorSupport.getCPUFeatures();
+        debug(featuresString);
+        String[] features = featuresString.toLowerCase(Locale.ROOT).split(", "); // ", " is used as a delimiter


On riscv, it's splitted by " ", for the fix please refer to CPUInfo.java in test.

Hamlin-Li · 2025-04-22T18:58:18Z

src/hotspot/share/runtime/abstract_vm_version.hpp

@@ -58,6 +58,8 @@ class Abstract_VM_Version: AllStatic {
  static uint64_t _features;
  static const char* _features_string;

+  static const char* _cpu_info_string;


Not quite sure the reason to introduce _cpu_info_string.
Seems to me you could just use _features_string, and remove _cpu_info_string and its related code, e.g. extract_features_string. Please check the code in test/lib/jdk/test/whitebox/cpuinfo/CPUInfo.java

Mayber in CPUFeatures, could use the similar code as CPUInfo to split the cpu string into cpu features?

The intention is to align _features_string with _features which enumerates well-known CPU capabilities JVM manages. As of now, _features_string contains more information, so I introduced _cpu_info_string to keep it.

Speaking of test/lib/jdk/test/whitebox/cpuinfo/CPUInfo.java, the approach chosen there may be fine for a test library, but we need a more stable API between JVM and JDK.

I'm fine with this.
But it might be better to change the spliting regex of _features_string in CPUFeatures.java to support riscv cpu features format.

Ok, I pushed an update. Let me know what you think about it.

Hamlin-Li

Found another possible issue on riscv.

Hamlin-Li · 2025-04-23T08:40:44Z

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java

+    V unaryMathOp(Unary op, int opc, VectorSpecies<E> vspecies,
+                  IntFunction<VectorSupport.UnaryOperation<V,?>> implSupplier,
+                  V v) {
+        var entry = lookup(op, opc, vspecies, implSupplier);


Seems there is another issue for riscv here.
If the rvv extension is not supported on the running machine, it will still generate the code using rvv, this should lead to a crash at runtime?

In previous code, we use UseRVV to detect if rvv extension is supported.

On the other hand, user can choose to disable UseRVV if they want even if rvv extension is supported on the running machine. In this sense, there could be similar issue on other platforms?

Does the following check catch UseRVV == false case on RISC-V?

public boolean isSupported(Operator op, VectorSpecies<?> vspecies) { ... int maxLaneCount = VectorSupport.getMaxLaneCount(vspecies.elementType()); if (vspecies.length() > maxLaneCount) { return false; // lacking vector support } ...

FTR both VectorSupport.getMaxLaneCount() and CPUFeatures don't rely on raw list of ISA extensions CPU supports, but only those reported by the JVM. So, if some feature support is disabled on JVM side, it won't be reported by VM_Version and, hence, CPUFeatures.

Thank you for updating! Looks good for riscv. I have ran some basic tests for vector API, passed. I did not ran benchmark, as riscv & aarch64 share the same way to bridge from java to sleef.

Does the following check catch UseRVV == false case on RISC-V?

Yes. If you don't mind, an explicit comment might be helpful. As to me "lacking vector support" here means the vector length is not large enough, but it's quite subjective, so you are on the call.

FTR both VectorSupport.getMaxLaneCount() and CPUFeatures don't rely on raw list of ISA extensions CPU supports, but only those reported by the JVM. So, if some feature support is disabled on JVM side, it won't be reported by VM_Version and, hence, CPUFeatures.

I'm fine with this.

Thanks, I added some clarifications in the comments.

jatin-bhateja · 2025-04-24T18:57:11Z

src/hotspot/share/opto/vectorIntrinsics.cpp

+    char* buf = NEW_ARENA_ARRAY(C->comp_arena(), char, buflen);
+    debug_name = debug_name_oop->const_oop()->as_instance()->java_lang_String_str(buf, buflen);
+  }
+  Node* vcall = make_runtime_call(RC_VECTOR,


By generating an upfront CallLeafVectorNode, we may miss out on performing any GVN-style optimization for trigonometric identities like the following. do you think creating a macro node which can lazily be expanded to call node during macro expansion will help.

arcsin(sin(x)) => x
arccos(cos(x)) => x
sin(arcsin(x) => x
cos(arccos(x) => x

It does look attractive, but macro expansion-based solution requires JVM to internalize such operations and their properties.

IMO a higher-level solution based on more generic JVM primitives would enable libraries to properly annotate their operations in Java bytecodes/class files, so C2 can perform such type of transformations without the need to intrinsify each individual operation first. (Think of JDK-8218414 / JDK-8347901 on steroids.)

I agree, this is a typical graph transform which cannot be applied currently because we are generating CallLeafVectorNode upfront during parsing, If we prevent intrinsification then compiler will attempt inlining, generating a much complex graph shape which may not be reducible.

I don't see any insurmountable problems performing such transformations on chains of CallLeafVector nodes (or any other call nodes). But the missing piece is information about the algebraic properties of native functions JVM can't derive on its own.

src/hotspot/cpu/riscv/riscv.ad

jatin-bhateja

Looks good to me.
Best Regards

iwanowww · 2025-04-25T21:19:51Z

Thanks for the reviews!

/integrate

openjdk · 2025-04-25T21:22:38Z

Going to push as commit e57fd71.
Since your change was applied there have been 146 commits pushed to the master branch:

5db62ab: 8315719: Adapt AOTClassLinking test case for dynamic CDS archive
2785570: 8355366: Fix the wrong usage of PassFailJFrame.forcePass() in some manual tests
4b88029: 8355439: Some hotspot/jtreg/serviceability/sa/* tests fail on static JDK due to explicit checks for shared libraries in process memory map
... and 143 more: https://git.openjdk.org/jdk/compare/4eae9b5ba61bfe262b43346a7499c98c1a54d2fe...master

Your commit was automatically rebased without conflicts.

openjdk · 2025-04-25T21:22:47Z

@iwanowww Pushed as commit e57fd71.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

iwanowww added 12 commits April 4, 2025 15:40

Remove vector math intrinsics

50210bd

Vector math library

57487c8

VM intrinsics

a92a2a8

VectorMathLib: Migrate to lambdas

714e4d0

cleanup

02802d0

SLEEF improvements

0734e25

fixes

def3434

Update templates

2efdac9

SVML fixes

c74739e

TODO list

2a7006b

Cleanup

70cdd15

CPU features support

0a1bba8

openjdk bot added core-libs [email protected] hotspot-compiler [email protected] labels Apr 4, 2025

Misc fixes and cleanups

fc27aee

iwanowww force-pushed the vector.math.01.java branch from 03c5a8c to fc27aee Compare April 5, 2025 01:30

iwanowww changed the title ~~XXXXXXX: ???~~ 8353786: Migrate Vector API math library support to FFM API Apr 5, 2025

openjdk bot added the hotspot [email protected] label Apr 5, 2025

iwanowww marked this pull request as ready for review April 5, 2025 02:38

openjdk bot added the rfr Pull request is ready for review label Apr 5, 2025

minborg reviewed Apr 7, 2025

View reviewed changes

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java Outdated Show resolved Hide resolved

vnkozlov reviewed Apr 7, 2025

View reviewed changes

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/CPUFeatures.java Show resolved Hide resolved

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/VectorMathLibrary.java Show resolved Hide resolved

liach reviewed Apr 7, 2025

View reviewed changes

src/jdk.incubator.vector/share/classes/jdk/incubator/vector/CPUFeatures.java Outdated Show resolved Hide resolved

openjdk bot removed the ready Pull request is ready to be integrated label Apr 17, 2025

iwanowww added 2 commits April 17, 2025 10:50

RVV and SVE adjustments

e2b762e

Merge remote-tracking branch 'origin/master' into vector.math.01.java

88eacc4

XiaohongGong approved these changes Apr 18, 2025

View reviewed changes

JornVernee approved these changes Apr 18, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 18, 2025

luhenry reviewed Apr 22, 2025

View reviewed changes

Hamlin-Li reviewed Apr 22, 2025

View reviewed changes

riscv fix

3d1adff

openjdk bot removed the ready Pull request is ready to be integrated label Apr 23, 2025

Avoid thread state transition in VectorSupport_GetCPUFeatures

42ed9ba

Hamlin-Li reviewed Apr 23, 2025

View reviewed changes

CPUFeatures: RISC-V support

585312a

Hamlin-Li approved these changes Apr 24, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 24, 2025

jatin-bhateja reviewed Apr 24, 2025

View reviewed changes

Improve comments

541c4d7

openjdk bot removed the ready Pull request is ready to be integrated label Apr 24, 2025

RealFYang reviewed Apr 25, 2025

View reviewed changes

src/hotspot/cpu/riscv/riscv.ad Show resolved Hide resolved

Remove UseVectorStubs usage in riscv.ad

f4373e4

jatin-bhateja approved these changes Apr 25, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Apr 25, 2025

openjdk bot added the integrated Pull request has been integrated label Apr 25, 2025

openjdk bot closed this Apr 25, 2025

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Apr 25, 2025

graalvmbot mentioned this pull request May 5, 2025

[GR-64700] Update labsjdk to 25+21-jvmci-b01 oracle/graal#11129

Merged

		ThreadToNativeFromVM ttn(thread);
		return env->NewStringUTF(features_string);

8353786: Migrate Vector API math library support to FFM API #24462

8353786: Migrate Vector API math library support to FFM API #24462

Uh oh!

Conversation

iwanowww commented Apr 4, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Apr 4, 2025

Uh oh!

openjdk bot commented Apr 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Apr 4, 2025

Uh oh!

liach commented Apr 4, 2025

Uh oh!

iwanowww commented Apr 5, 2025

Uh oh!

openjdk bot commented Apr 5, 2025

Uh oh!

mlbridge bot commented Apr 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

Uh oh!

vnkozlov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

XiaohongGong left a comment

Choose a reason for hiding this comment

Uh oh!

JornVernee left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JornVernee Apr 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iwanowww Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iwanowww Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hamlin-Li left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iwanowww commented Apr 4, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Apr 4, 2025 •

edited

Loading

mlbridge bot commented Apr 5, 2025 •

edited

Loading

JornVernee Apr 18, 2025 •

edited

Loading

iwanowww Apr 23, 2025 •

edited

Loading

iwanowww Apr 23, 2025 •

edited

Loading

Hamlin-Li Apr 23, 2025 •

edited

Loading

iwanowww Apr 24, 2025 •

edited

Loading

iwanowww Apr 25, 2025 •

edited

Loading